Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bxAzv-0000L9-NU for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 10:54:56 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1bxAzv-0002zX-A3 for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 10:54:55 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1bxAze-0002kz-PN for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 10:54:38 +0000 Received: from mail-lf0-x232.google.com ([2a00:1450:4010:c07::232]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1bxAzZ-00040X-Ue for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 10:54:38 +0000 Received: by mail-lf0-x232.google.com with SMTP id x79so81678815lff.0 for ; Thu, 20 Oct 2016 03:54:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=hJ7wi92oDQsdi9uxFKcQKTVtqAQ0O6hFwEWMMHsbLXI=; b=CqEzoNpBdejjl609WTSuyhK436+q5Yudd3uGxgWaJxmuR1aWIDjIiJs1D7rWjqQCuF 6KzQ9d98WjrFDv4mLhxImZw5nUxWUFX0lnwVyKMnu8FWpPi3I65X/rA6/ybMXMBnRvLR /4hqM5XBygXNdElnGLp4khMtsv5hNyF0J+DnLrEI+TTQyoOEmXAm2CDIWOMp1owuajvQ ozL3MJ9MFgvOmeyrekPrDz7uDoAz484ioTlc1cLBkXIeoyObMxjGjA3iu5wbvat2Zqi7 elEjfZ1wq0pWFi/qjJ86JrlIeAdxMa50YLiJr2CNxsKmtczjCACiCW+ntPcsXU2lTO3i JG8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=hJ7wi92oDQsdi9uxFKcQKTVtqAQ0O6hFwEWMMHsbLXI=; b=FYPoDApVFUnbbfWbs/Gzwn7eeVXGRV27tAmC0LUUKww1xd6fwKWwSbHbd9f47YX3vK MunbEkVodb4yEUY2dyFB6tHx/t57ecWfZE51W/VP3fk46xb/dzFsxvHCPWMu5cJONwsz klwVHdzFyMTMJkn+CE+pzyiPejqkbfObkJ/gIPZl4RTsB6oqNzC/chSw2NlwIoeLDnCs xi4hoWqJHwCOxWoVCdUu3T48j0vi6wAf4RDYWE1ncYbtT/QVzSZjIFuuzyEsY2hUEf7T lMcYR2X56bMWx40oeABiG7YfIEEZ26o7qSf3MxSyAA9pUMSMIoHqGixtbHU4AJiihuoa uAMw== X-Gm-Message-State: ABUngvfqwNyf13vHnFWaJLEo1+v11+Sy1brUDkE8fDtvVNhjU/Ff4oH2+5quDelmlcCO0D+RaUq4n2CEsxB9MEsq X-Received: by 10.25.198.209 with SMTP id w200mr611431lff.175.1476960872315; Thu, 20 Oct 2016 03:54:32 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.92.219 with HTTP; Thu, 20 Oct 2016 03:54:30 -0700 (PDT) In-Reply-To: References: From: Fahar Abbas Date: Thu, 20 Oct 2016 15:54:30 +0500 Message-ID: Subject: Re: RM1849: Auto-generating security keys To: Ashesh Vashi Cc: Murtuza Zabuawala , Dave Page , pgadmin-hackers , Josh Berkus , =?UTF-8?B?RGV2cmltIEfDnE5Ew5xa?= , Magnus Hagander , Sandeep Thakkar , Hamid Quddus Akhtar Content-Type: multipart/alternative; boundary=94eb2c1a089e245b91053f49c054 X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgadmin-hackers Precedence: bulk Sender: pgadmin-hackers-owner@postgresql.org --94eb2c1a089e245b91053f49c054 Content-Type: text/plain; charset=UTF-8 All, I have not seen any issue in pgAdmin4 on MAC OS X(Python 3.5) Platform as well hence 1849 is resolved. Kind Regards, On Thu, Oct 20, 2016 at 1:35 PM, Fahar Abbas wrote: > Thanks pgAdmin4 Development team! > > User can launch pgAdmin4 with web browser with no issues with fresh setup, > i tried on Ubuntu 16.04 Linux 64(Python 3.5). Will also try on OS X with > Python 3 and will let you know. > > On Thu, Oct 20, 2016 at 1:03 PM, Fahar Abbas > wrote: > >> Thanks Ashesh for your steps and will also test with new setup( on fresh >> machine) and our customers can face this problem with new setup and will >> update the status accordingly. >> >> On Thu, Oct 20, 2016 at 12:53 PM, Ashesh Vashi < >> ashesh.vashi@enterprisedb.com> wrote: >> >>> You were missing the other places, before the function - do_setup(..) >>> call, where we are generating the other keys automatically. >>> I've checked-in the code. >>> >>> Fahar, >>> >>> If you want to test the issue properly, please follow the below steps: >>> - git checkout c4f1b8eb112e99d228c40a412020fa616dbe44f6 >>> This was the commit-id, where we have the configuration schema version >>> '13'. >>> - Delete the existing pgadmin4.db >>> - Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in >>> config_local.py. >>> - Execute the command - 'python setup.py'. >>> - Now - you've configuration file with old schema. >>> - 'git checkout master' to go to the latest code. >>> - Make sure - you've latest code. 'git pull'. >>> - Now - rerun 'python setup.py/pgAdmin4.py'. It should work. >>> >>> >>> -- >>> >>> Thanks & Regards, >>> >>> Ashesh Vashi >>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>> >>> >>> >>> *http://www.linkedin.com/in/asheshvashi* >>> >>> >>> On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabuawala < >>> murtuza.zabuawala@enterprisedb.com> wrote: >>> >>>> Hi, >>>> >>>> PFA patch to fix the issue for Pyhton3. >>>> RM#1849 >>>> >>>> -- >>>> Regards, >>>> Murtuza Zabuawala >>>> EnterpriseDB: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>>> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >>>> fahar.abbas@enterprisedb.com> wrote: >>>> >>>>> Hi Dave, >>>>> >>>>> I have reopened following RM: >>>>> ================================ >>>>> https://redmine.postgresql.org/issues/1849 >>>>> >>>>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page wrote: >>>>> >>>>>> Patch applied. >>>>>> >>>>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>> >>>>>>> Hi Fahar, >>>>>>> >>>>>>> Please log the case on redmine. >>>>>>> Please find the attached patch, please apply it locally, and test it. >>>>>>> >>>>>>> And, please update the case, and this mail chain accordingly. >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> >>>>>>> Ashesh Vashi >>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>> >>>>>>> >>>>>>> >>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>> >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>>> >>>>>>>> Here is the output of if we copy config_local.py and execute python >>>>>>>> setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> python setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> User can not do any setup for web based now. >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>>>> >>>>>>>>> Dave, >>>>>>>>> >>>>>>>>> Testing Environment >>>>>>>>> >>>>>>>>> Ubuntu 16.04 Linux 64: >>>>>>>>> -------------------------------- >>>>>>>>> >>>>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>>>> >>>>>>>>> >>>>>>>>> 1) Install GIT >>>>>>>>> >>>>>>>>> sudo apt-get install git >>>>>>>>> >>>>>>>>> 2) Install pip3 >>>>>>>>> >>>>>>>>> sudo apt-get install python3-pip >>>>>>>>> >>>>>>>>> 3) Install virtualenv >>>>>>>>> >>>>>>>>> sudo pip3 install virtualenv >>>>>>>>> >>>>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>>>> pycrypto module >>>>>>>>> >>>>>>>>> sudo apt-get install libpq-dev >>>>>>>>> >>>>>>>>> sudo apt-get install python3-dev >>>>>>>>> >>>>>>>>> 5) Create virtual environment >>>>>>>>> >>>>>>>>> virtualenv -p python3 venv >>>>>>>>> >>>>>>>>> 6) Create mkdir Projects >>>>>>>>> >>>>>>>>> 7) Clone git repo in Projects >>>>>>>>> >>>>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>>>> >>>>>>>>> 8) activate virtual environment >>>>>>>>> >>>>>>>>> source venv/bin/activate >>>>>>>>> >>>>>>>>> 9) Install modules >>>>>>>>> >>>>>>>>> pip3 install -r requirements_py3.txt >>>>>>>>> >>>>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>>>> Projects\pgAdmin4\web * >>>>>>>>> >>>>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>>>> python setup.py >>>>>>>>> >>>>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>>>> >>>>>>>>> Here is the output: >>>>>>>>> ------------------------- >>>>>>>>> >>>>>>>>> python setup.py >>>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>>> ====================================== >>>>>>>>> >>>>>>>>> >>>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>>> does not exist. >>>>>>>>> Entering initial setup mode... >>>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>>> >>>>>>>>> >>>>>>>>> Enter the email address and password to use for the initial >>>>>>>>> pgAdmin user account: >>>>>>>>> >>>>>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>>>>> Password: >>>>>>>>> Retype password: >>>>>>>>> Traceback (most recent call last): >>>>>>>>> File "setup.py", line 449, in >>>>>>>>> do_setup(app) >>>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>>> password = encrypt_password(p1) >>>>>>>>> File "/home/fahar/venv/lib/python3. >>>>>>>>> 5/site-packages/flask_security/utils.py", line 150, in >>>>>>>>> encrypt_password >>>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>>> File "/home/fahar/venv/lib/python3. >>>>>>>>> 5/site-packages/flask_security/utils.py", line 108, in get_hmac >>>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` >>>>>>>>> must not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>>> "pbkdf2_sha512" >>>>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>>>> >>>>>>>>> >>>>>>>>> Is this expected? >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>>>>> >>>>>>>>>> Sure, >>>>>>>>>> >>>>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>>>> >>>>>>>>>> Kind Regards, >>>>>>>>>> >>>>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Patch applied. >>>>>>>>>>> >>>>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>>>> >>>>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>>>> >>>>>>>>>>> Packagers: This change means that packages are no longer forced >>>>>>>>>>> to create a config_local.py file, and there is no longer any need to >>>>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave, >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Friday, October 14, 2016, Dave Page >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page < >>>>>>>>>>>>>>> dpage@pgadmin.org> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Can you please review the attached patch, and apply if >>>>>>>>>>>>>>>> you're happy with it? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment >>>>>>>>>>>>>>> with 'web' mode. >>>>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after >>>>>>>>>>>>>>> initializing the Security object. >>>>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hmm. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I had moved the Security object initialization after >>>>>>>>>>>>>>> fetching these configurations from the database. >>>>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> OK, thanks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>>>> problem. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>>>> >>>>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>>>> installations/databases. >>>>>>>>>>>>> >>>>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>>>> >>>>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>>>> >>>>>>>>>>>> Please find the updated patch. >>>>>>>>>>>> >>>>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> Thanks & Regards, >>>>>>>>>>>> >>>>>>>>>>>> Ashesh Vashi >>>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>>>> >>>>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>>>> >>>>>>>>>>>>> Patch attached - please review (Ashesh, but others too would >>>>>>>>>>>>> be appreciated)! >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Dave Page >>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>>> >>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Syed Fahar Abbas >>>>>>>>>> Quality Management Group >>>>>>>>>> >>>>>>>>>> EnterpriseDB Corporation >>>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>>> Website: www.enterprisedb.com >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Syed Fahar Abbas >>>>>>>>> Quality Management Group >>>>>>>>> >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>> Website: www.enterprisedb.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com --94eb2c1a089e245b91053f49c054 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
All,

I have not seen any issue in pgAdmin4 on = MAC OS X(Python 3.5) Platform as well hence 1849 is resolved.

= Kind Regards,

On Thu, Oct 20, 2016 at 1:35 PM, Fahar Abbas <fahar.abbas= @enterprisedb.com> wrote:
<= div dir=3D"ltr">Thanks pgAdmin4 Development team!

User can launch pg= Admin4 with web browser with no issues with fresh setup, i tried on Ubuntu = 16.04 Linux 64(Python 3.5). Will also try on OS X with Python 3 and will le= t you know.

On Thu, Oct 20, 2016 at 1:03 PM= , Fahar Abbas <fahar.abbas@enterprisedb.com> wrot= e:
Thanks Ashesh for you= r steps and will also test with new setup( on fresh machine) and our custom= ers can face this problem with new setup and will update the status accordi= ngly.

On Thu, Oct 20, 2016 at 12:53 PM, Ashesh Vashi &l= t;ashesh= .vashi@enterprisedb.com> wrote:
You were missing the other places, before the fu= nction - do_setup(..) call, where we are generating the other keys automati= cally.
I've checked-in the code.

Fahar,

If you want to test the issue properly, please follo= w the below steps:
- git checkout c4f1b8eb112e99d228c40a412020fa<= wbr>616dbe44f6
=C2=A0 This was the commit-id, where we have the c= onfiguration schema version '13'.
- Delete the existing p= gadmin4.db
- Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in= config_local.py.
- Execute the command - 'python setup.py= 9;.
- Now - you've configuration file with old schema.
<= div>- 'git checkout master' to go to the latest code.
- M= ake sure - you've latest code. 'git pull'.
- Now - re= run 'python s= etup.py/pgAdmin4.py'. It should work.



On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabu= awala <murtuza.zabuawala@enterprisedb.com> wrote:
Hi,

=
PFA patch to fix the issue for Pyhton3.
RM#1849
<= /div>

= --
Regards,
Murtuza Zabuawa= la
EnterpriseDB:=C2=A0http:= //www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abba= s <fahar.abbas@enterprisedb.com> wrote:
Hi Dave,

I have reopened f= ollowing RM:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
https://redmine.postgresql.or= g/issues/1849

On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <dpage@pgadmin= .org> wrote:
Patch applied.

On Wed, Oct 19, 2016 at 11:55 AM, As= hesh Vashi <ashesh.vashi@enterprisedb.com> = wrote:
Hi Fahar,

Please log the case on redmine.
Please find the attac= hed patch, please apply it locally, and test it.

A= nd, please update the case, and this mail chain accordingly.

--

Thanks & Regards,=

Ashesh Vashi
EnterpriseDB INDIA: <= a href=3D"http://www.enterprisedb.com" target=3D"_blank">Enterprise Postgre= SQL Company

<= br>

<= a href=3D"http://www.linkedin.com/in/asheshvashi" target=3D"_blank">http= ://www.linkedin.com/in/asheshvashi


On Wed, Oct 19, 2016 at 3:47 PM, F= ahar Abbas <fahar.abbas@enterprisedb.com> wrote:<= br>
Here is the output = of if we copy config_local.py and execute python setup.py
pgAdmin = 4 - Application Initialisation
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D


The configuration database - '/home/fahar/.pgadmin/pg= admin4.db' does not exist.
Entering initial setup mode...
NO= TE: Configuring authentication for SERVER mode.


=C2=A0=C2=A0=C2= =A0 Enter the email address and password to use for the initial pgAdmin use= r=C2=A0=C2=A0=C2=A0=C2=A0 account:

Email address: fahar.abbas@enterprisedb.co= m
Password:
Retype password:
Traceback (most recent call last= ):
=C2=A0 File "setup.py", line 449, in <module>
=C2= =A0=C2=A0=C2=A0 do_setup(app)
=C2=A0 File "setup.py", line 96,= in do_setup
=C2=A0=C2=A0=C2=A0 password =3D encrypt_password(p1)
=C2= =A0 File "/home/fahar/venv/lib/python3.5/site-packages/flask_secu= rity/utils.py", line 150, in encrypt_password
=C2=A0=C2=A0=C2= =A0 signed =3D get_hmac(password).decode('ascii')
=C2=A0 Fi= le "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac
=C2=A0=C2=A0=C2=A0 'set to= "%s"' % _security.password_hash)
RuntimeError: The config= uration value `SECURITY_PASSWORD_SALT` must not be None when the value of `= SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512"
python setup= .py
pgAdmin 4 - Application Initialisation
=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D

User can not do any setup for web= based now.


The configuration database - '/home/f= ahar/.pgadmin/pgadmin4.db' does not exist.
Entering initial set= up mode...
NOTE: Configuring authentication for SERVER mode.


= =C2=A0=C2=A0=C2=A0 Enter the email address and password to use for the init= ial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 account:

Email address: fahar.abbas@= enterprisedb.com
Password:
Retype password:
Traceback (most r= ecent call last):
=C2=A0 File "setup.py", line 449, in <mod= ule>
=C2=A0=C2=A0=C2=A0 do_setup(app)
=C2=A0 File "setup.py&q= uot;, line 96, in do_setup
=C2=A0=C2=A0=C2=A0 password =3D encrypt_passw= ord(p1)
=C2=A0 File "/home/fahar/venv/lib/python3.5/site-packa= ges/flask_security/utils.py", line 150, in encrypt_password
= =C2=A0=C2=A0=C2=A0 signed =3D get_hmac(password).decode('ascii'= ;)
=C2=A0 File "/home/fahar/venv/lib/python3.5/site-packages/f= lask_security/utils.py", line 108, in get_hmac
=C2=A0=C2=A0=C2= =A0 'set to "%s"' % _security.password_hash)
RuntimeEr= ror: The configuration value `SECURITY_PASSWORD_SALT` must not be None when= the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512"<= br>

On Wed, Oct 19, 2016 at 3:= 03 PM, Fahar Abbas <fahar.abbas@enterprisedb.com>= wrote:
Dave,<= br>

Testing Environment
=C2=A0
Ubuntu 16.04= Linux 64:
--------------------------------

pg-AdminIV Development Environment Setup for= Ubun= tu=C2=A0 :

<= br>

1) Insta= ll GIT

sudo = apt-get install git

2) Install pip3

sudo apt-get install python3-pip

3) Install virtualenv

sudo pip3 install virtualenv

4) install belo= w dependency as it is required for psycopg2 & pycrypto module

sudo apt-get install = libpq-dev

s= udo apt-get install python3-dev

5) Create virtual environment

virtualenv -p python3 venv

6) Create mkdir Projects

7) Clone git re= po in Projects

git clone http://git.postgresql.org/git/pgadmin4.git

8) activate virtu= al environment

source venv/bin/activate

9) Install modules

pip3 install -r requirements_py3.txt

10) Edit the config.py file to config_local.py =C2=A0resides in Pro= jects\pgAdmin4\web=C2=A0=C2=A0

11)Now run setup.py file =C2=A0(\Projects\= pgAdmin4\web)


If user does not create config_local.py and do Python setup= .py for new Development then SECURITY_PASSWORD_SALT message is also display= ed:

Here is the output:
-------------------------
<= /div>

python setup.py
pgAdmin 4 - Application Initialisation=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D


The configuratio= n database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist= .
Entering initial setup mode...
NOTE: Configuring authentication for= SERVER mode.


=C2=A0=C2=A0=C2=A0 Enter the email address and pas= sword to use for the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 account:<= br>
Email address: fahar.abbas@enterprisedb.com
Password:
Retype passw= ord:
Traceback (most recent call last):
=C2=A0 File "setup.py&qu= ot;, line 449, in <module>
=C2=A0=C2=A0=C2=A0 do_setup(app)
=C2= =A0 File "setup.py", line 96, in do_setup
=C2=A0=C2=A0=C2=A0 p= assword =3D encrypt_password(p1)
=C2=A0 File "/home/fahar/venv/lib/= python3.5/site-packages/flask_security/utils.py", line 150, = in encrypt_password
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac(password).dec= ode('ascii')
=C2=A0 File "/home/fahar/venv/lib/python3= .5/site-packages/flask_security/utils.py", line 108, in get_= hmac
=C2=A0=C2=A0=C2=A0 'set to "%s"' % _security.pass= word_hash)
RuntimeError: The configuration value `SECURITY_PASSWORD_SALT= ` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to &qu= ot;pbkdf2_sha512"
(venv) fahar@fahar-virtual-machine:~/Project= s/pgadmin4/web$


Is this expected?

On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas <fahar.abbas@ente= rprisedb.com> wrote:
Sure,

Will test this thoroughly after com= plete investigation.

Kind Regards,

On Wed, Oct 19, 2016 at 1:27 PM, Dave Page= <dpage@pgadmin.org> wrote:
Patch applied.

Fahar, can you please = test this thoroughly in desktop and server modes, with both fresh and upgra= ded installations?


Packagers: This change means th= at packages are no longer forced to create a config_local.py file, and ther= e is no longer any need to explicitly set=C2=A0SECURITY_PASSWORD_SALT,=C2=A0S= ECURITY_KEY and=C2=A0CSRF_SESSION_KEY in the config (in fact, they sho= uld be removed for new installations, if you have included them in 1.0)

Thanks.


On Wed, Oct 19, 2016 at 6:46 AM, Ashe= sh Vashi <ashesh.vashi@enterprisedb.com> wr= ote:
= Hi Dave,

On Sat, Oct 15, 2016 at 8:02 AM= , Dave Page <dpage@pgadmin.org> wrote:
Hi


On Friday, October 14, 2016, Dave Page= <dpage@pgadmin.o= rg> wrote:
Hi
On Thursday, October 13, 2016, Ashesh Vashi <ashesh.vashi@enter= prisedb.com> wrote:
Hi Dave,

On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi Ashesh,

Can = you please review the attached patch, and apply if you're happy with it= ?
Overall the patch looked good to me.
But= - I encounter an issue in 'web' mode, which wont happen with '= runtime'.

Steps for reproduction on existing p= gAdmin 4 environment with 'web' mode.
- Apply the patch
- Start the pgAdmin4 application (stand alone application).
<= div>- Open pgAdmin home page.
- Log out (if already login).
=

And, you will see an exception.

I have figure out the issue with the patch.
We were setting the= SECURITY_PASSWORD_SALT, after initializing the Security object.
= Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT prope= rly.

Hmm.
=C2=A0

<= div>I had moved the Security object initialization after fetching these con= figurations from the database.
I have attached a addon patch for = the same.

OK, thank= s.
=C2=A0
=

Now - I run into another issue.
Because - the exi= sting password was hashed using the old SECURITY_PASSWORD_SALT, I am no mor= e able to login to pgAdmin 4.

I think - we need to= think about different strategy for upgrading the configuration file in the= 'web' mode.
I was thinking - we can store the existing s= ecurity configurations in the database during upgrade process in 'web&#= 39; mode.

My concer= n with that is that we'll likely be storing the default config values i= n many cases, thus for those users, perpetuating the problem.
I guess what we need to do is re-encrypt the password during th= e upgrade - however, that makes me think; we then have both the key and the= encrypted passwords in the same database which is clearly not a good idea.= Sigh... Needs more thought.=C2=A0

<= /div>
OK, so I've been thinking about this and experimenting for a = couple of hours, as well as annoying the crap out of Magnus by thinking out= loud in his general direction, and it looks like this isn't a major pr= oblem as from what I can see, =C2=A0SECURITY_PASSWORD_SALT is (aside from r= eally being a key not a salt) not the only salting that's done.=C2=A0

It looks like it's used system-wide as the key = to generate an HMAC of the users password, which is then passed to passlib = which salts and hashes it. I did some testing, and found that two users wit= h the same password end up with different hashes in the database, so clearl= y there is also per-user salting happening. I also created two users, then = dropped the database and created the same user accounts with the same passw= ords again, and found that the resulting hashes were different in both data= bases - thus there is something else ensuring the hashes are unique across = different installations/databases.

So, I believe w= e can do as you suggest and migrate existing values for SECURITY_PASSWORD_S= ALT, given that there's clearly some other per user and per installatio= n/database salting going on anyway. New installations can have the random v= alue for SECURITY_PASSWORD_SALT.
We do n= ot need to generate the random SECURITY_PASSWORD_SALT during upgrade mode, = which was wrong added in my addon patch.

Please fi= nd the updated patch.

Otherwise - looks good to me= .
Please commit the new patch (if you're ok with the change).=


--

Thanks & Regards,
Ashesh Vash= i
EnterpriseDB INDIA:=C2=A0Enterprise PostgreSQL Company



<= div>I don't believe SECURITY_KEY and=C2=A0CSRF_SESSION_KEY are issues e= ither, as they're used for purposes that are essentially ephemeral, and= thus can be changed during an upgrade.

Adding Mag= nus as I'd appreciate any thoughts he may have.

Patch attached - please review (Ashesh, but others too would be appreciat= ed)!

Thanks.


--
Dave Page
Blog: http://pgsnake.blogspot.comTwitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterpris= e PostgreSQL Company





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twit= ter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise Post= greSQL Company



-= -
Syed Fahar Abbas
Quality Management Group

<= /div>EnterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone Dir= ect: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.= fahar.abbas
Website: www.enterprisedb.com



--
Syed Fahar Abbas
Quality M= anagement Group

EnterpriseDB Corporation
Phone Office: +92-= 51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707<= /a>
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com



--
Syed Fahar Abbas
Quality Management Group
=
EnterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone = Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed= .fahar.abbas
Website: www.enterprisedb.com




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


--
Syed Fahar Abbas
Quality Ma= nagement Group

EnterpriseDB Corporation
Phone Office: +92-5= 1-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com




--
Syed Fahar Abbas
Quality Man= agement Group

EnterpriseDB Corporation
Phone Office: +92-51= -835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com



--
Syed Fahar Abbas
Quality Management = Group

EnterpriseDB Corporation
Phone Office: +92-51-835-887= 4
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Sky= pe ID: syed.fahar.abbas
Website: www.enterprisedb.com



--
Syed Fahar Abbas
Quality Management Group

E= nterpriseDB Corporation
Phone Office: +92-51-835-8874
Phone Direct: += 92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
W= ebsite: www.enter= prisedb.com
--94eb2c1a089e245b91053f49c054--