Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bx8pq-0004tJ-1B for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 08:36:22 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1bx8pp-0000QY-EA for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 08:36:21 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1bx8pY-0000Ap-Cn for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 08:36:04 +0000 Received: from mail-lf0-x229.google.com ([2a00:1450:4010:c07::229]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1bx8pT-0000Sq-KK for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 08:36:03 +0000 Received: by mail-lf0-x229.google.com with SMTP id b81so72249327lfe.1 for ; Thu, 20 Oct 2016 01:35:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=LgSxXoBfqbpFjMx3EVadm9jZfMpfDaRNVzxdl+SVNp4=; b=R50H7xvXLTFbS7OJknfMn1ngNTO84MtP+X3DOlDwk1IbkKGsqmPnHFHtturlitTN8+ SumHNT2T3w5+eIULfb8SQKZS0+JepN8BZE4gUCSUyz1X5IwRhp1QbzYbAxRxeFSi8XmB 7fvbo0FIrSUboa+UGcLn0gcLJPnOFKyk1xudnV/o9vVLztdgNDPY7YyOHUUX9+WDEFM5 aWDmA62COxuFPpjAynPyEp6NXW7IXBIyFCkpC0BwqdXFlUcmRQmUoPc6vhCN92ZKKQaN wzLAOQRZFDqE1fXfDVvkLU30bi4jqhiCYAHs8MpGyHxI2nkav0wzb9dV/iX3XH00gDFu AP1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=LgSxXoBfqbpFjMx3EVadm9jZfMpfDaRNVzxdl+SVNp4=; b=W5anvTnltva/9bzWxsLGVULV+3pL7Dq+r3SqDkAcmTzHJubnZoTkN8AJbCg0mY2BoA r9ZvleCstb6JAb8hm8uM+yhTdp0Sb3pJJAFEqWObk0edUhZ9rRDWA9Mw4h3nEi6Ti3/A CZ2WvnZ4qL8y6/YShgRk6h3UxcWdb7x3wvLZ6W9uQktKZaPNDifaejLgA/L1IGR5TalR jQbkLH0lkNNnjQIKND9Es+vE0MToBG6miF+gKlraT2GDMo2hQ+EJq3PFXmnYnw7IWB+s 92hmJ5+HP5NurZaV1WW/Gvzwib4kZmq0eshFZ53KOlVP+rIodBuEi1VD8x1E7PaPU5zK MH9w== X-Gm-Message-State: AA6/9RnkQUHbHwT5kMMs7jyZ3/nKrRQU9+NwnfXkmYrPi4AKsKAkjzloAroLitxScvs3Zet9mfAFMvgsXd83AKCa X-Received: by 10.25.190.69 with SMTP id o66mr11312747lff.109.1476952555179; Thu, 20 Oct 2016 01:35:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.92.219 with HTTP; Thu, 20 Oct 2016 01:35:53 -0700 (PDT) In-Reply-To: References: From: Fahar Abbas Date: Thu, 20 Oct 2016 13:35:53 +0500 Message-ID: Subject: Re: RM1849: Auto-generating security keys To: Ashesh Vashi Cc: Murtuza Zabuawala , Dave Page , pgadmin-hackers , Josh Berkus , =?UTF-8?B?RGV2cmltIEfDnE5Ew5xa?= , Magnus Hagander , Sandeep Thakkar , Hamid Quddus Akhtar Content-Type: multipart/alternative; boundary=94eb2c1a1e0a66e24e053f47d05c X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgadmin-hackers Precedence: bulk Sender: pgadmin-hackers-owner@postgresql.org --94eb2c1a1e0a66e24e053f47d05c Content-Type: text/plain; charset=UTF-8 Thanks pgAdmin4 Development team! User can launch pgAdmin4 with web browser with no issues with fresh setup, i tried on Ubuntu 16.04 Linux 64(Python 3.5). Will also try on OS X with Python 3 and will let you know. On Thu, Oct 20, 2016 at 1:03 PM, Fahar Abbas wrote: > Thanks Ashesh for your steps and will also test with new setup( on fresh > machine) and our customers can face this problem with new setup and will > update the status accordingly. > > On Thu, Oct 20, 2016 at 12:53 PM, Ashesh Vashi < > ashesh.vashi@enterprisedb.com> wrote: > >> You were missing the other places, before the function - do_setup(..) >> call, where we are generating the other keys automatically. >> I've checked-in the code. >> >> Fahar, >> >> If you want to test the issue properly, please follow the below steps: >> - git checkout c4f1b8eb112e99d228c40a412020fa616dbe44f6 >> This was the commit-id, where we have the configuration schema version >> '13'. >> - Delete the existing pgadmin4.db >> - Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in >> config_local.py. >> - Execute the command - 'python setup.py'. >> - Now - you've configuration file with old schema. >> - 'git checkout master' to go to the latest code. >> - Make sure - you've latest code. 'git pull'. >> - Now - rerun 'python setup.py/pgAdmin4.py'. It should work. >> >> >> -- >> >> Thanks & Regards, >> >> Ashesh Vashi >> EnterpriseDB INDIA: Enterprise PostgreSQL Company >> >> >> >> *http://www.linkedin.com/in/asheshvashi* >> >> >> On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabuawala < >> murtuza.zabuawala@enterprisedb.com> wrote: >> >>> Hi, >>> >>> PFA patch to fix the issue for Pyhton3. >>> RM#1849 >>> >>> -- >>> Regards, >>> Murtuza Zabuawala >>> EnterpriseDB: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >>> fahar.abbas@enterprisedb.com> wrote: >>> >>>> Hi Dave, >>>> >>>> I have reopened following RM: >>>> ================================ >>>> https://redmine.postgresql.org/issues/1849 >>>> >>>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page wrote: >>>> >>>>> Patch applied. >>>>> >>>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>> >>>>>> Hi Fahar, >>>>>> >>>>>> Please log the case on redmine. >>>>>> Please find the attached patch, please apply it locally, and test it. >>>>>> >>>>>> And, please update the case, and this mail chain accordingly. >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & Regards, >>>>>> >>>>>> Ashesh Vashi >>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>> >>>>>> >>>>>> >>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>> >>>>>> >>>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>> >>>>>>> Here is the output of if we copy config_local.py and execute python >>>>>>> setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> User can not do any setup for web based now. >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>>> >>>>>>>> Dave, >>>>>>>> >>>>>>>> Testing Environment >>>>>>>> >>>>>>>> Ubuntu 16.04 Linux 64: >>>>>>>> -------------------------------- >>>>>>>> >>>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>>> >>>>>>>> >>>>>>>> 1) Install GIT >>>>>>>> >>>>>>>> sudo apt-get install git >>>>>>>> >>>>>>>> 2) Install pip3 >>>>>>>> >>>>>>>> sudo apt-get install python3-pip >>>>>>>> >>>>>>>> 3) Install virtualenv >>>>>>>> >>>>>>>> sudo pip3 install virtualenv >>>>>>>> >>>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>>> pycrypto module >>>>>>>> >>>>>>>> sudo apt-get install libpq-dev >>>>>>>> >>>>>>>> sudo apt-get install python3-dev >>>>>>>> >>>>>>>> 5) Create virtual environment >>>>>>>> >>>>>>>> virtualenv -p python3 venv >>>>>>>> >>>>>>>> 6) Create mkdir Projects >>>>>>>> >>>>>>>> 7) Clone git repo in Projects >>>>>>>> >>>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>>> >>>>>>>> 8) activate virtual environment >>>>>>>> >>>>>>>> source venv/bin/activate >>>>>>>> >>>>>>>> 9) Install modules >>>>>>>> >>>>>>>> pip3 install -r requirements_py3.txt >>>>>>>> >>>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>>> Projects\pgAdmin4\web * >>>>>>>> >>>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>>> python setup.py >>>>>>>> >>>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>>> >>>>>>>> Here is the output: >>>>>>>> ------------------------- >>>>>>>> >>>>>>>> python setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>>> >>>>>>>> >>>>>>>> Is this expected? >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>>>> >>>>>>>>> Sure, >>>>>>>>> >>>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>>> >>>>>>>>> Kind Regards, >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Patch applied. >>>>>>>>>> >>>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>>> >>>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>>> >>>>>>>>>> Packagers: This change means that packages are no longer forced >>>>>>>>>> to create a config_local.py file, and there is no longer any need to >>>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Dave, >>>>>>>>>>> >>>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Friday, October 14, 2016, Dave Page >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page >>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can you please review the attached patch, and apply if >>>>>>>>>>>>>>> you're happy with it? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>>>> 'web' mode. >>>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>>> >>>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after >>>>>>>>>>>>>> initializing the Security object. >>>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hmm. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> OK, thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>>> problem. >>>>>>>>>>>>> >>>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>>> >>>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>>> installations/databases. >>>>>>>>>>>> >>>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>>> >>>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>>> >>>>>>>>>>> Please find the updated patch. >>>>>>>>>>> >>>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> Thanks & Regards, >>>>>>>>>>> >>>>>>>>>>> Ashesh Vashi >>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>>> >>>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>>> >>>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>>>> appreciated)! >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Dave Page >>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>> >>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Syed Fahar Abbas >>>>>>>>> Quality Management Group >>>>>>>>> >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>> Website: www.enterprisedb.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >> > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com --94eb2c1a1e0a66e24e053f47d05c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks pgAdmin4 Development team!

User can launch p= gAdmin4 with web browser with no issues with fresh setup, i tried on Ubuntu= 16.04 Linux 64(Python 3.5). Will also try on OS X with Python 3 and will l= et you know.

On Thu, Oct 20, 2016 at 1:03 PM, Fahar Abbas <fahar.abbas@= enterprisedb.com> wrote:
Thanks Ashesh for your steps and will also test with new set= up( on fresh machine) and our customers can face this problem with new setu= p and will update the status accordingly.
<= div class=3D"h5">

= On Thu, Oct 20, 2016 at 12:53 PM, Ashesh Vashi <ashesh.vashi@e= nterprisedb.com> wrote:
You were missing the other places, before the function -= do_setup(..) call, where we are generating the other keys automatically.I've checked-in the code.

Fahar,
=
If you want to test the issue properly, please follow the be= low steps:
- git checkout c4f1b8eb112e99d228c40a412020fa616d= be44f6
=C2=A0 This was the commit-id, where we have the configura= tion schema version '13'.
- Delete the existing pgadmin4.= db
- Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in config_= local.py.
- Execute the command - 'python setup.py'.
- Now - you've configuration file with old schema.
- &#= 39;git checkout master' to go to the latest code.
- Make sure= - you've latest code. 'git pull'.
- Now - rerun '= ;python setup.py/= pgAdmin4.py'. It should work.


--

Thanks & Regards,

Ashesh= Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company

<= br>

<= a href=3D"http://www.linkedin.com/in/asheshvashi" target=3D"_blank">http= ://www.linkedin.com/in/asheshvashi


On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:
Hi,

PFA patch t= o fix the issue for Pyhton3.
RM#1849

--
Regar= ds,
Murtuza Zabuawala
EnterpriseDB:=C2=A0http://www.enterprisedb.com
The Enterpr= ise PostgreSQL Company

On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abba= s <fahar.abbas@enterprisedb.com> wrote:
Hi Dave,

I have reopened f= ollowing RM:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
https://redmine.postgresql.or= g/issues/1849

On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <dpage@pgadmin= .org> wrote:
Patch applied.

On We= d, Oct 19, 2016 at 11:55 AM, Ashesh Vashi <ashesh.vashi@enterp= risedb.com> wrote:
Hi Fahar,

Please log the case on red= mine.
Please find the attached patch, please apply it locally, and test= it.

And, please update the case, and this mail ch= ain accordingly.

--

Thanks & R= egards,

Ashesh Vashi
EnterpriseDB INDIA: <= /span>Enterprise = PostgreSQL Company

<= br>

<= a href=3D"http://www.linkedin.com/in/asheshvashi" target=3D"_blank">http= ://www.linkedin.com/in/asheshvashi


On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas <fahar.abbas@enterprisedb.com> wrote:
Here is the output of if we copy config= _local.py and execute python setup.py
pgAdmin 4 - Application Init= ialisation
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D


The = configuration database - '/home/fahar/.pgadmin/pgadmin4.db' do= es not exist.
Entering initial setup mode...
NOTE: Configuring authen= tication for SERVER mode.


=C2=A0=C2=A0=C2=A0 Enter the email add= ress and password to use for the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2= =A0 account:

Email address: fahar.abbas@enterprisedb.com
Password: Retype password:
Traceback (most recent call last):
=C2=A0 File &quo= t;setup.py", line 449, in <module>
=C2=A0=C2=A0=C2=A0 do_setu= p(app)
=C2=A0 File "setup.py", line 96, in do_setup
=C2=A0= =C2=A0=C2=A0 password =3D encrypt_password(p1)
=C2=A0 File "/home/f= ahar/venv/lib/python3.5/site-packages/flask_security/utils.py&quo= t;, line 150, in encrypt_password
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac= (password).decode('ascii')
=C2=A0 File "/home/fahar/ve= nv/lib/python3.5/site-packages/flask_security/utils.py", lin= e 108, in get_hmac
=C2=A0=C2=A0=C2=A0 'set to "%s"' % = _security.password_hash)
RuntimeError: The configuration value `SECURITY= _PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH`= is set to "pbkdf2_sha512"
python setup.py
pgAdmin 4 - Appl= ication Initialisation
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=
User can not do any setup for web based now.


The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist.
Entering initial setup mode...
NOTE: Confi= guring authentication for SERVER mode.


=C2=A0=C2=A0=C2=A0 Enter = the email address and password to use for the initial pgAdmin user=C2=A0=C2= =A0=C2=A0=C2=A0 account:

Email address: fahar.abbas@enterprisedb.com
P= assword:
Retype password:
Traceback (most recent call last):
=C2= =A0 File "setup.py", line 449, in <module>
=C2=A0=C2=A0= =C2=A0 do_setup(app)
=C2=A0 File "setup.py", line 96, in do_se= tup
=C2=A0=C2=A0=C2=A0 password =3D encrypt_password(p1)
=C2=A0 File = "/home/fahar/venv/lib/python3.5/site-packages/flask_security= /utils.py", line 150, in encrypt_password
=C2=A0=C2=A0=C2=A0 signed= =3D get_hmac(password).decode('ascii')
=C2=A0 File "/= home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.= py", line 108, in get_hmac
=C2=A0=C2=A0=C2=A0 'set to "%s&= quot;' % _security.password_hash)
RuntimeError: The configuration va= lue `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_P= ASSWORD_HASH` is set to "pbkdf2_sha512"
<= div class=3D"m_1636870599681870228m_-3974466794599420138m_-7225415612496190= 839m_-5631392044300219617m_2500240099394767488m_-6843424418728108180h5">
On Wed, Oct 19, 2016= at 3:03 PM, Fahar Abbas <fahar.abbas@enterprisedb.com><= /span> wrote:
= Dave,

Testing Environment
=C2=A0
Ubuntu= 16.04 Linux 64:
--------------------------------

pg-AdminIV Development Environment Setup for Ubuntu=C2=A0 :


= 1) Install GIT

sudo apt-get install git

<= p dir=3D"ltr" style=3D"line-height:1.38;margin-top:0pt;margin-bottom:0pt"><= font size=3D"2">2) Install pip3<= /p>

sudo apt-get install pytho= n3-pip

3) In= stall virtualenv

sudo pip3 install virtualenv

4) install below dependency as it is required for psy= copg2 & pycrypto module

sudo apt-get install libpq-dev

sudo apt-get install python3-dev

5) Create virt= ual environment

virtualenv -p python3 venv

6) Create mkdir Projects

7) Clone git repo in Projects

git clone http://git.postgre= sql.org/git/pgadmin4.git

8) activate virtual environment

source venv/bin/activate

9) Install modules

pip3 install -r requirements_py3= .txt

10) Edit the config.py file to config_local.py =C2= =A0resides in Projects\pgAdmin4\web=C2=A0=C2=A0

11)Now run setup.py file =C2=A0(\Projects\pgAdmin4\web)

=C2=A0 =C2=A0 = python setup.py

If user does not create config_local.py and do Python set= up.py for new Development then SECURITY_PASSWORD_SALT message is also displ= ayed:

Here is the output:
-------------------------

python setup.py
pgAdmin 4 - Application Initialisation<= br>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D


The configuratio= n database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist= .
Entering initial setup mode...
NOTE: Configuring authentication for= SERVER mode.


=C2=A0=C2=A0=C2=A0 Enter the email address and pas= sword to use for the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 account:<= br>
Email address: fahar.abbas@enterprisedb.com
Password:
Retype passw= ord:
Traceback (most recent call last):
=C2=A0 File "setup.py&qu= ot;, line 449, in <module>
=C2=A0=C2=A0=C2=A0 do_setup(app)
=C2= =A0 File "setup.py", line 96, in do_setup
=C2=A0=C2=A0=C2=A0 p= assword =3D encrypt_password(p1)
=C2=A0 File "/home/fahar/venv/lib/= python3.5/site-packages/flask_security/utils.py", line 150, = in encrypt_password
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac(password).dec= ode('ascii')
=C2=A0 File "/home/fahar/venv/lib/python3= .5/site-packages/flask_security/utils.py", line 108, in get_= hmac
=C2=A0=C2=A0=C2=A0 'set to "%s"' % _security.pass= word_hash)
RuntimeError: The configuration value `SECURITY_PASSWORD_SALT= ` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to &qu= ot;pbkdf2_sha512"
(venv) fahar@fahar-virtual-machine:~/Project= s/pgadmin4/web$


Is this expected?

=
On Wed, Oct 19, 2016 = at 1:37 PM, Fahar Abbas <fahar.abbas@enterprisedb.com> wrote:
S= ure,

Will test this thoroughly after complete investigation.
Kind Regards,

On Wed, Oct= 19, 2016 at 1:27 PM, Dave Page <dpage@pgadmin.org> wrote:
Patch applied.

Fahar, can you please test this thoroughly in desktop and server m= odes, with both fresh and upgraded installations?


= Packagers: This change means that packages are no longer forced to create a= config_local.py file, and there is no longer any need to explicitly set=C2= =A0SECURITY_PASSWORD_SALT,=C2=A0SECURITY_KEY and=C2=A0CSRF_SESSION_KEY i= n the config (in fact, they should be removed for new installations, if you= have included them in 1.0)

Thanks.


On Wed, Oct 19, 2016 at 6:4= 6 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com>= wrote:
Hi Dave,

On Sat, Oct 15, 2016 at 8:02 AM, Dave Page = <dpage@pgadmin.org> wrote:
Hi

On Friday, October 14, 2016, Dave Page <dpage@pgadmin.org> wrote:
Hi

On Thursday, October 13, 2016= , Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
=
Hi Dave,=

On Tue= , Oct 11, 2016 at 9:10 PM, Dave Page <dpage@pgadmin= .org> wrote:
Hi Ashesh,

Can you please review the attached patch= , and apply if you're happy with it?
Overall the= patch looked good to me.
But - I encounter an issue in 'web&= #39; mode, which wont happen with 'runtime'.

Steps for reproduction on existing pgAdmin 4 environment with 'web&#= 39; mode.
- Apply the patch
- Start the pgAdmin4 applic= ation (stand alone application).
- Open pgAdmin home page.
<= div>- Log out (if already login).

And, you will se= e an exception.

I have figure out the issue with t= he patch.
We were setting the SECURITY_PASSWORD_SALT, after initi= alizing the Security object.
Hence - it could not set the SECURIT= Y_KEY, and SECURITY_PASSWORD_SALT properly.

Hmm.
=C2=A0

I had moved the Security object = initialization after fetching these configurations from the database.
=
I have attached a addon patch for the same.

OK, thanks.
=C2=A0

Now - I run into ano= ther issue.
Because - the existing password was hashed using the = old SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4.
<= div>
I think - we need to think about different strategy for = upgrading the configuration file in the 'web' mode.
I was= thinking - we can store the existing security configurations in the databa= se during upgrade process in 'web' mode.

My concern with that is that we'll likely= be storing the default config values in many cases, thus for those users, = perpetuating the problem.

I guess what we need to = do is re-encrypt the password during the upgrade - however, that makes me t= hink; we then have both the key and the encrypted passwords in the same dat= abase which is clearly not a good idea. Sigh... Needs more thought.=C2=A0

OK, so I've been think= ing about this and experimenting for a couple of hours, as well as annoying= the crap out of Magnus by thinking out loud in his general direction, and = it looks like this isn't a major problem as from what I can see, =C2=A0= SECURITY_PASSWORD_SALT is (aside from really being a key not a salt) not th= e only salting that's done.=C2=A0

It looks lik= e it's used system-wide as the key to generate an HMAC of the users pas= sword, which is then passed to passlib which salts and hashes it. I did som= e testing, and found that two users with the same password end up with diff= erent hashes in the database, so clearly there is also per-user salting hap= pening. I also created two users, then dropped the database and created the= same user accounts with the same passwords again, and found that the resul= ting hashes were different in both databases - thus there is something else= ensuring the hashes are unique across different installations/databases.

So, I believe we can do as you suggest and migrate = existing values for SECURITY_PASSWORD_SALT, given that there's clearly = some other per user and per installation/database salting going on anyway. = New installations can have the random value for SECURITY_PASSWORD_SALT.
We do not need to generate the random SECUR= ITY_PASSWORD_SALT during upgrade mode, which was wrong added in my addon pa= tch.

Please find the updated patch.

=
Otherwise - looks good to me.
Please commit the new pa= tch (if you're ok with the change).


--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA:=C2=A0Enterprise PostgreSQL Company


I don't believe SE= CURITY_KEY and=C2=A0CSRF_SESSION_KEY are issues either, as they're used= for purposes that are essentially ephemeral, and thus can be changed durin= g an upgrade.

Adding Magnus as I'd appreciate = any thoughts he may have.

Patch attached - please = review (Ashesh, but others too would be appreciated)!

<= div>Thanks.


--
Dave Page
Blog: http://pgsnake.blogspot.= com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The = Enterprise PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

E= nterpriseDB UK: h= ttp://www.enterprisedb.com
The Enterprise PostgreSQL Company



--
Syed Fahar Abbas
Q= uality Management Group

EnterpriseDB Corporation
Phone Offi= ce: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-33= 3-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com



--
Syed Fahar Abbas
Quality Management Group
=
EnterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone = Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed= .fahar.abbas
Website: www.enterprisedb.com



--
Syed Fah= ar Abbas
Quality Management Group

EnterpriseDB C= orporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803<= /a>
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Websit= e: www.enterprise= db.com




--
Dave Page
Blog: http://pgsnake.blogspot.= com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The = Enterprise PostgreSQL Company


--
Syed Fahar= Abbas
Quality Management Group

EnterpriseDB Cor= poration
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website:= www.enterprisedb= .com




--
Syed Fahar Abbas
Quality Management Group

<= /div>EnterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone Dir= ect: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.= fahar.abbas
Website: www.enterprisedb.com



--
Syed Fahar Abbas
Quality Management Group

E= nterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone Direct: += 92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
W= ebsite: www.enter= prisedb.com
--94eb2c1a1e0a66e24e053f47d05c--