Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bx8Nb-0002Xi-0O for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 08:07:11 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1bx8Na-0001ai-Iz for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 08:07:10 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1bx8NL-0000sI-C3 for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 08:06:55 +0000 Received: from mail-lf0-x229.google.com ([2a00:1450:4010:c07::229]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1bx8Jm-0000rZ-Ft for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 08:03:18 +0000 Received: by mail-lf0-x229.google.com with SMTP id x79so74053139lff.0 for ; Thu, 20 Oct 2016 01:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=7f/sbZVo6mJgUIEp6SBiH3azneuJzrxXqMJeK9+bSPc=; b=dR6pksJrHyat7NWDDI2AaXg1TpY6u1ybg0SAgR0llAbVZJK1uHnTyUydycAy8uLdTo eLK569T/bfsV2OsdGb3NShhCMH3E153keMlXNxo8Ys7/A2JXwHKj6aeRgWTBe/PUbFLf gDBgQYOq/zq9ci372YHOiI8Yr0swM6jv9moYbWVEN2BpDv7zCCKC2beTT/l3gbG3mrrc mn8ePPGKHsiZnaSJBSVsX05na4AjjhvX93czfugA4koea+ETv8BOSqaZ2qnmxhQDDTlL vmZG2jPtC91m3E4XLaFGpiJu0FH+XIDwgyPnxkbJPycDR79WNVVFM5QVDicmIy1M4o85 qf8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=7f/sbZVo6mJgUIEp6SBiH3azneuJzrxXqMJeK9+bSPc=; b=fjays10YaNCbWxFX/28tHbALwSdGYb1RW28DLRYDoeo1M1pL3xOGWRZFA3C5W0nGqh P+WedT3wyWeyJ/cBU4ALrtn0JA9A8ZU6fpTngfmcNxps/rNTP8T6Jpuc55ntzWVbVhd/ P5/PYJb7MTaqgqvij4xmaa2Gw/KAk3xu23QREDi8a1qiPzMMh063hX8Pir9cnA5b6r6x sClre6KjjcsLBW65Syk8ij/kGEB8Lft2lxvlPeTkqmEEa1DOn1lGYillMhbQWH7RGIR5 BTQaoKGKcJX2u8HueMb/YBbSVsHaMHd3vkSKUTDWfejsdB2MrxYmWJ68vMqAjFuN6kZn W/BQ== X-Gm-Message-State: AA6/9Rk2l5WmVlulRsHbGhGPVMxIMRWyz890vTrMFfRP9rLTYE59NoSyK4xu04EL276QpNbSGbPSi4fzzvjTRM4I X-Received: by 10.25.26.73 with SMTP id a70mr11018884lfa.108.1476950593662; Thu, 20 Oct 2016 01:03:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.92.219 with HTTP; Thu, 20 Oct 2016 01:03:11 -0700 (PDT) In-Reply-To: References: From: Fahar Abbas Date: Thu, 20 Oct 2016 13:03:11 +0500 Message-ID: Subject: Re: RM1849: Auto-generating security keys To: Ashesh Vashi Cc: Murtuza Zabuawala , Dave Page , pgadmin-hackers , Josh Berkus , =?UTF-8?B?RGV2cmltIEfDnE5Ew5xa?= , Magnus Hagander , Sandeep Thakkar , Hamid Quddus Akhtar Content-Type: multipart/alternative; boundary=001a1140278c7c885e053f475bad X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgadmin-hackers Precedence: bulk Sender: pgadmin-hackers-owner@postgresql.org --001a1140278c7c885e053f475bad Content-Type: text/plain; charset=UTF-8 Thanks Ashesh for your steps and will also test with new setup( on fresh machine) and our customers can face this problem with new setup and will update the status accordingly. On Thu, Oct 20, 2016 at 12:53 PM, Ashesh Vashi < ashesh.vashi@enterprisedb.com> wrote: > You were missing the other places, before the function - do_setup(..) > call, where we are generating the other keys automatically. > I've checked-in the code. > > Fahar, > > If you want to test the issue properly, please follow the below steps: > - git checkout c4f1b8eb112e99d228c40a412020fa616dbe44f6 > This was the commit-id, where we have the configuration schema version > '13'. > - Delete the existing pgadmin4.db > - Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in > config_local.py. > - Execute the command - 'python setup.py'. > - Now - you've configuration file with old schema. > - 'git checkout master' to go to the latest code. > - Make sure - you've latest code. 'git pull'. > - Now - rerun 'python setup.py/pgAdmin4.py'. It should work. > > > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > > > > *http://www.linkedin.com/in/asheshvashi* > > > On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabuawala enterprisedb.com> wrote: > >> Hi, >> >> PFA patch to fix the issue for Pyhton3. >> RM#1849 >> >> -- >> Regards, >> Murtuza Zabuawala >> EnterpriseDB: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >> fahar.abbas@enterprisedb.com> wrote: >> >>> Hi Dave, >>> >>> I have reopened following RM: >>> ================================ >>> https://redmine.postgresql.org/issues/1849 >>> >>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page wrote: >>> >>>> Patch applied. >>>> >>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>> ashesh.vashi@enterprisedb.com> wrote: >>>> >>>>> Hi Fahar, >>>>> >>>>> Please log the case on redmine. >>>>> Please find the attached patch, please apply it locally, and test it. >>>>> >>>>> And, please update the case, and this mail chain accordingly. >>>>> >>>>> -- >>>>> >>>>> Thanks & Regards, >>>>> >>>>> Ashesh Vashi >>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>> >>>>> >>>>> >>>>> *http://www.linkedin.com/in/asheshvashi* >>>>> >>>>> >>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>> fahar.abbas@enterprisedb.com> wrote: >>>>> >>>>>> Here is the output of if we copy config_local.py and execute python >>>>>> setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> User can not do any setup for web based now. >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> >>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>> >>>>>>> Dave, >>>>>>> >>>>>>> Testing Environment >>>>>>> >>>>>>> Ubuntu 16.04 Linux 64: >>>>>>> -------------------------------- >>>>>>> >>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>> >>>>>>> >>>>>>> 1) Install GIT >>>>>>> >>>>>>> sudo apt-get install git >>>>>>> >>>>>>> 2) Install pip3 >>>>>>> >>>>>>> sudo apt-get install python3-pip >>>>>>> >>>>>>> 3) Install virtualenv >>>>>>> >>>>>>> sudo pip3 install virtualenv >>>>>>> >>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>> pycrypto module >>>>>>> >>>>>>> sudo apt-get install libpq-dev >>>>>>> >>>>>>> sudo apt-get install python3-dev >>>>>>> >>>>>>> 5) Create virtual environment >>>>>>> >>>>>>> virtualenv -p python3 venv >>>>>>> >>>>>>> 6) Create mkdir Projects >>>>>>> >>>>>>> 7) Clone git repo in Projects >>>>>>> >>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>> >>>>>>> 8) activate virtual environment >>>>>>> >>>>>>> source venv/bin/activate >>>>>>> >>>>>>> 9) Install modules >>>>>>> >>>>>>> pip3 install -r requirements_py3.txt >>>>>>> >>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>> Projects\pgAdmin4\web * >>>>>>> >>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>> python setup.py >>>>>>> >>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>> >>>>>>> Here is the output: >>>>>>> ------------------------- >>>>>>> >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>> >>>>>>> >>>>>>> Is this expected? >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>>> >>>>>>>> Sure, >>>>>>>> >>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>> >>>>>>>> Kind Regards, >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Patch applied. >>>>>>>>> >>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>> >>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>> >>>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Dave, >>>>>>>>>> >>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Friday, October 14, 2016, Dave Page >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>>> happy with it? >>>>>>>>>>>>>> >>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>> >>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>>> 'web' mode. >>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>> >>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>> >>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>>> the Security object. >>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hmm. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>> >>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>> problem. >>>>>>>>>>>> >>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>> >>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>> installations/databases. >>>>>>>>>>> >>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>> >>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>> >>>>>>>>>> Please find the updated patch. >>>>>>>>>> >>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks & Regards, >>>>>>>>>> >>>>>>>>>> Ashesh Vashi >>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>> >>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>> >>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>>> appreciated)! >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dave Page >>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>> Twitter: @pgsnake >>>>>>>>> >>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com --001a1140278c7c885e053f475bad Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Ashesh for your steps and will also test with new s= etup( on fresh machine) and our customers can face this problem with new se= tup and will update the status accordingly.

On Thu, Oct 20, 2016 at 12:53 PM, Ashe= sh Vashi <ashesh.vashi@enterprisedb.com> wrote:<= br>
You were missing the oth= er places, before the function - do_setup(..) call, where we are generating= the other keys automatically.
I've checked-in the code.
=
Fahar,

If you want to test the issu= e properly, please follow the below steps:
- git checkout c4f1b8e= b112e99d228c40a412020fa616dbe44f6
=C2=A0 This was the commit= -id, where we have the configuration schema version '13'.
- Delete the existing pgadmin4.db
- Set CSRF_SESSION_KEY, SECRET_KEY, S= ECURITY_PASSWORD_SALT in config_local.py.
- Execute the command -= 'python setup.py'.
- Now - you've configuration file= with old schema.
- 'git checkout master' to go to the la= test code.
- Make sure - you've latest code. 'git pull= 9;.
- Now - rerun 'python setup.py/pgAdmin4.py'. It should work.
=


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company

<= br>

<= a href=3D"http://www.linkedin.com/in/asheshvashi" target=3D"_blank">http= ://www.linkedin.com/in/asheshvashi


On Thu, Oct 20= , 2016 at 11:15 AM, Murtuza Zabuawala <murtuza.zabuawala@= enterprisedb.com> wrote:
Hi,

PFA patch to fix the issue for P= yhton3.
RM#1849

= --
Regards,
Murtuza Zabuawala=
EnterpriseDB:=C2=A0http://www.ent= erprisedb.com
The Enterprise PostgreSQL Company

On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abba= s <fahar.abbas@enterprisedb.com> wrote:
Hi Dave,

I have reopened f= ollowing RM:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
https://redmine.postgresql.or= g/issues/1849

On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <dpage@pgadmin= .org> wrote:
Patch applied.
On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi = <ashesh.vashi@enterprisedb.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">
Hi Fahar,
Please log the case on redmine.
Please find the attached patch, = please apply it locally, and test it.

And, please = update the case, and this mail chain accordingly.

--

Th= anks & Regards,

Ashesh Vashi
Enterpri= seDB INDIA: Enterprise PostgreSQL Company

<= br>

<= a href=3D"http://www.linkedin.com/in/asheshvashi" target=3D"_blank">http= ://www.linkedin.com/in/asheshvashi


On= Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas <fahar.abbas@enterpr= isedb.com> wrote:
Here is the output of if we copy config_local.py and execute = python setup.py
pgAdmin 4 - Application Initialisation
=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D


The configuration databas= e - '/home/fahar/.pgadmin/pgadmin4.db' does not exist.
Ente= ring initial setup mode...
NOTE: Configuring authentication for SERVER m= ode.


=C2=A0=C2=A0=C2=A0 Enter the email address and password to = use for the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 account:

Em= ail address: fahar.abbas@enterprisedb.com
Password:
Retype password:
T= raceback (most recent call last):
=C2=A0 File "setup.py", line= 449, in <module>
=C2=A0=C2=A0=C2=A0 do_setup(app)
=C2=A0 File = "setup.py", line 96, in do_setup
=C2=A0=C2=A0=C2=A0 password = =3D encrypt_password(p1)
=C2=A0 File "/home/fahar/venv/lib/python3.= 5/site-packages/flask_security/utils.py", line 150, in encry= pt_password
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac(password).decode('= ;ascii')
=C2=A0 File "/home/fahar/venv/lib/python3.5/= site-packages/flask_security/utils.py", line 108, in get_hmac
= =C2=A0=C2=A0=C2=A0 'set to "%s"' % _security.password_has= h)
RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must n= ot be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf= 2_sha512"
python setup.py
pgAdmin 4 - Application Initialisation=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

User= can not do any setup for web based now.


The configur= ation database - '/home/fahar/.pgadmin/pgadmin4.db' does not e= xist.
Entering initial setup mode...
NOTE: Configuring authentication= for SERVER mode.


=C2=A0=C2=A0=C2=A0 Enter the email address and= password to use for the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 accou= nt:

Email address: fahar.abbas@enterprisedb.com
Password:
Retype p= assword:
Traceback (most recent call last):
=C2=A0 File "setup.p= y", line 449, in <module>
=C2=A0=C2=A0=C2=A0 do_setup(app)=C2=A0 File "setup.py", line 96, in do_setup
=C2=A0=C2=A0=C2= =A0 password =3D encrypt_password(p1)
=C2=A0 File "/home/fahar/venv= /lib/python3.5/site-packages/flask_security/utils.py", line = 150, in encrypt_password
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac(password= ).decode('ascii')
=C2=A0 File "/home/fahar/venv/lib/py= thon3.5/site-packages/flask_security/utils.py", line 108, in= get_hmac
=C2=A0=C2=A0=C2=A0 'set to "%s"' % _security= .password_hash)
RuntimeError: The configuration value `SECURITY_PASSWORD= _SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set t= o "pbkdf2_sha512"

On Wed, O= ct 19, 2016 at 3:03 PM, Fahar Abbas <fahar.abbas@enterprisedb.c= om> wrote:
Dave,

Testing Environment
=C2=A0
<= /div>Ubuntu 16.04 Linux 64:
--------------------------------

pg-AdminIV Development Enviro= nment Setup for Ubuntu=C2=A0 :


1)= Install GIT

sudo apt-get install git

2) Install pip3

sudo apt-get install python3-pip

3) Install virtualenv=

sudo pip3 install virtual= env

4) insta= ll below dependency as it is required for psycopg2 & pycrypto module

sudo apt-get i= nstall libpq-dev

sudo apt-get install python3-dev

5) Create virtual environment

<= p dir=3D"ltr" style=3D"line-height:1.38;margin-top:0pt;margin-bottom:0pt"><= font size=3D"2">virtualenv -p python3 venv

6) Create mkdir Projects

7) Clone g= it repo in Projects

git clone http://git.postgresql.org/git/pgadmin4.git

8) activate virtual environment

source venv/bin/activate

9) Install modules

pip3 install -r= requirements_py3.txt

10) Edit the config.py file to co= nfig_local.py =C2=A0resides in Projects\pg= Admin4\web=C2=A0=C2=A0

= 11)Now run setup.py f= ile =C2=A0(\Projects\pgAdmin4\web)

=C2=A0 =C2=A0 python setup<= /span>.py

If user does not create config_local.py and do Py= thon setup.py for new Development then SECURITY_PASSWORD_SALT message is al= so displayed:

Here is the output:
--------------------= -----

python setup.py
pgAdmin 4 - Application Initial= isation
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D


The con= figuration database - '/home/fahar/.pgadmin/pgadmin4.db' does = not exist.
Entering initial setup mode...
NOTE: Configuring authentic= ation for SERVER mode.


=C2=A0=C2=A0=C2=A0 Enter the email addres= s and password to use for the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 = account:

Email address: fahar.abbas@enterprisedb.com
Password:
Ret= ype password:
Traceback (most recent call last):
=C2=A0 File "se= tup.py", line 449, in <module>
=C2=A0=C2=A0=C2=A0 do_setup(ap= p)
=C2=A0 File "setup.py", line 96, in do_setup
=C2=A0=C2= =A0=C2=A0 password =3D encrypt_password(p1)
=C2=A0 File "/home/faha= r/venv/lib/python3.5/site-packages/flask_security/utils.py",= line 150, in encrypt_password
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac(pa= ssword).decode('ascii')
=C2=A0 File "/home/fahar/venv/= lib/python3.5/site-packages/flask_security/utils.py", line 1= 08, in get_hmac
=C2=A0=C2=A0=C2=A0 'set to "%s"' % _se= curity.password_hash)
RuntimeError: The configuration value `SECURITY_PA= SSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is= set to "pbkdf2_sha512"
(venv) fahar@fahar-virtual-machine:~/<= wbr>Projects/pgadmin4/web$


Is this expected?

On Wed, Oct 19, 2016 at 1:37 PM= , Fahar Abbas <fahar.abbas@enterprisedb.com> wrot= e:
Sure,
Will test this thoroughly after complete investigation.

Kind Regards,

On Wed, Oct 19, 2016 at 1:27 PM, Dave Page= <dpage@pgadmin.org> wrote:
Patch applied.

Fahar, can you please = test this thoroughly in desktop and server modes, with both fresh and upgra= ded installations?


Packagers: This change means th= at packages are no longer forced to create a config_local.py file, and ther= e is no longer any need to explicitly set=C2=A0SECURITY_PASSWORD_SALT,=C2=A0S= ECURITY_KEY and=C2=A0CSRF_SESSION_KEY in the config (in fact, they sho= uld be removed for new installations, if you have included them in 1.0)

Thanks.


On We= d, Oct 19, 2016 at 6:46 AM, Ashesh Vashi <ashesh.vashi@enterpr= isedb.com> wrote:
Hi Dave,

On Sat, Oct 15, 2016 at 8:02 AM, Dave Page= <dpage@pgadmin.org> wrote:
Hi


On Friday, October= 14, 2016, Dave Page <dpage@pgadmin.org> wrote:
Hi

On Thursday, October 13, 2016, Ashesh Vashi <<= a>ashesh.vashi@enterprisedb.com> wrote:
Hi Dave,

On Tue, Oct 11, 2016 at 9:= 10 PM, Dave Page <dpage@pgadmin.org> = wrote:
Hi Ashesh,
=
Can you please review the attached patch, and apply if you&#= 39;re happy with it?
Overall the patch looked good t= o me.
But - I encounter an issue in 'web' mode, which won= t happen with 'runtime'.

Steps for reprodu= ction on existing pgAdmin 4 environment with 'web' mode.
= - Apply the patch
- Start the pgAdmin4 application (stand alone a= pplication).
- Open pgAdmin home page.
- Log out (if al= ready login).

And, you will see an exception.

I have figure out the issue with the patch.
= We were setting the SECURITY_PASSWORD_SALT, after initializing the Security= object.
Hence - it could not set the SECURITY_KEY, and SECURITY_= PASSWORD_SALT properly.

=
Hmm.
=C2=A0

I had moved the Security object initialization after= fetching these configurations from the database.
I have attached= a addon patch for the same.

<= /div>
OK, thanks.
=C2=A0

Now - I run into another issue.
<= div>Because - the existing password was hashed using the old SECURITY_PASSW= ORD_SALT, I am no more able to login to pgAdmin 4.

I think - we need to think about different strategy for upgrading the conf= iguration file in the 'web' mode.
I was thinking - we can= store the existing security configurations in the database during upgrade = process in 'web' mode.
My concern with that is that we'll likely be storing the de= fault config values in many cases, thus for those users, perpetuating the p= roblem.

I guess what we need to do is re-encrypt t= he password during the upgrade - however, that makes me think; we then have= both the key and the encrypted passwords in the same database which is cle= arly not a good idea. Sigh... Needs more thought.=C2=A0
<= div>
OK, so I've been thinking about this and= experimenting for a couple of hours, as well as annoying the crap out of M= agnus by thinking out loud in his general direction, and it looks like this= isn't a major problem as from what I can see, =C2=A0SECURITY_PASSWORD_= SALT is (aside from really being a key not a salt) not the only salting tha= t's done.=C2=A0

It looks like it's used sy= stem-wide as the key to generate an HMAC of the users password, which is th= en passed to passlib which salts and hashes it. I did some testing, and fou= nd that two users with the same password end up with different hashes in th= e database, so clearly there is also per-user salting happening. I also cre= ated two users, then dropped the database and created the same user account= s with the same passwords again, and found that the resulting hashes were d= ifferent in both databases - thus there is something else ensuring the hash= es are unique across different installations/databases.

So, I believe we can do as you suggest and migrate existing values fo= r SECURITY_PASSWORD_SALT, given that there's clearly some other per use= r and per installation/database salting going on anyway. New installations = can have the random value for SECURITY_PASSWORD_SALT.
We do not need to generate the random SECURITY_PASSWORD_SALT = during upgrade mode, which was wrong added in my addon patch.
Please find the updated patch.

Otherwi= se - looks good to me.
Please commit the new patch (if you're= ok with the change).

--

Thanks & Regards,

A= shesh Vashi
EnterpriseDB INDIA:=C2=A0Enterprise PostgreSQL Company



I don't believe SECURITY_K= EY and=C2=A0CSRF_SESSION_KEY are issues either, as they're used for pur= poses that are essentially ephemeral, and thus can be changed during an upg= rade.

Adding Magnus as I'd appreciate any thou= ghts he may have.

Patch attached - please review (= Ashesh, but others too would be appreciated)!

Than= ks.


--
Dave = Page
Blog: htt= p://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: <= a href=3D"http://www.enterprisedb.com" target=3D"_blank">http://www.enterpr= isedb.com
The Enterprise PostgreSQL Company





--
Dave Page=
Blog: http://= pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprised= b.com
The Enterprise PostgreSQL Company



--
= Syed Fahar Abbas
Quality Management Group

Enter= priseDB Corporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51= -8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas<= br>Website: www.e= nterprisedb.com



--
Syed Fah= ar Abbas
Quality Management Group

EnterpriseDB C= orporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803<= /a>
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Websit= e: www.enterprise= db.com



--
Syed Fahar Abbas
Quality Management Group

EnterpriseDB Corporation
Phone O= ffice: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-3= 33-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com
<= /div>




-- =
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

Enterp= riseDB UK: http:/= /www.enterprisedb.com
The Enterprise PostgreSQL Company



--
Syed Fahar Abbas
Quality Management G= roup

EnterpriseDB Corporation
Phone Office: +92-51-835-8874=
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skyp= e ID: syed.fahar.abbas
Website: www.enterprisedb.com




--
Syed Fahar= Abbas
Quality Management Group

EnterpriseDB Cor= poration
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com=
--001a1140278c7c885e053f475bad--