Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bx8B2-0001XR-Q4 for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 07:54:13 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1bx8B2-0003gk-A4 for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 07:54:12 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1bx8An-0003SU-MM for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 07:53:58 +0000 Received: from mail-it0-x236.google.com ([2607:f8b0:4001:c0b::236]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1bx8Aj-0000gf-3k for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 07:53:56 +0000 Received: by mail-it0-x236.google.com with SMTP id 4so158209658itv.0 for ; Thu, 20 Oct 2016 00:53:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=zZXRriS/irpqH6pdyF3ZnUIgyhejGZvTRCCutgOCIcs=; b=ztbnc+veaZ7pwzkACNSk6dzl6JOjNwFlLvBcO81kXvibwk42bbCexrf6nAPSpRi38B D1Afr0qj0kE5Y+3DR/iAuHcLoPSxqCPptdqiKxHoNLloFfl0EgFpXxTDXmLBy64ORIWf aj6/aqdkqh2TXWE0TWvmGxxLqmSSvrcAVr2V9ouw7izESFA7ctClt7yRkDkrfGLPOYEy dcfk84Gq/27iz4InvKwlpa/aY7Mk1IFcl7Fl3/7VmlHI5k+tQKedjG7yhyHk3aIWGR50 Vqihy4qTNi7Ow22YSgft6bucm0AqCGdZ3e+90MksRsPyydN6U4EPPQKFrngGJIzubvNE QLtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=zZXRriS/irpqH6pdyF3ZnUIgyhejGZvTRCCutgOCIcs=; b=Es4Tv4up/DkVttvZ5OZV6jOPF8MmzxZnkIRJ5KqlsqHWXH9bBHXUsbbItR+dYAmP2g DGLxuxHSkc+ZaclDd/A106Tx5bpc19OeU0NfWu68S3RyJiVroS5GijI1xxMj+/rCgISG Z44Z5Iuz50+G6lWJglOfjfO5+hmxvKZqkFBxs16l8qRZ4r4c+KwtwiVroduQ2hFAHifA ZESrO83O0gMODrr2sgZCLekhVN8GZmiKpS0eqsYM2Mufh7y7EeZXWjzHEBuwyZ8aTfgK WfLuQpSwOYmYt5DNm+j5USWDa/yl8k99KnxpKGkTuz5QFHByZllQ/lS5t5FPzTWUI6EL 1JwQ== X-Gm-Message-State: AA6/9RmUM9ic7m18O3ufRKX7ZB6t+SM01m8FVgvWsYC5arRpg1QDgfZRd0A4PIDcBM+WZPca2kd1Bv8zpJL8IQ5W X-Received: by 10.107.58.70 with SMTP id h67mr9030773ioa.115.1476950031041; Thu, 20 Oct 2016 00:53:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.25.139 with HTTP; Thu, 20 Oct 2016 00:53:30 -0700 (PDT) In-Reply-To: References: From: Ashesh Vashi Date: Thu, 20 Oct 2016 13:23:30 +0530 Message-ID: Subject: Re: RM1849: Auto-generating security keys To: Murtuza Zabuawala , Fahar Abbas Cc: Dave Page , pgadmin-hackers , Josh Berkus , =?UTF-8?B?RGV2cmltIEfDnE5Ew5xa?= , Magnus Hagander , Sandeep Thakkar , Hamid Quddus Akhtar Content-Type: multipart/alternative; boundary=001a114a8ef6f39f53053f473910 X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgadmin-hackers Precedence: bulk Sender: pgadmin-hackers-owner@postgresql.org --001a114a8ef6f39f53053f473910 Content-Type: text/plain; charset=UTF-8 You were missing the other places, before the function - do_setup(..) call, where we are generating the other keys automatically. I've checked-in the code. Fahar, If you want to test the issue properly, please follow the below steps: - git checkout c4f1b8eb112e99d228c40a412020fa616dbe44f6 This was the commit-id, where we have the configuration schema version '13'. - Delete the existing pgadmin4.db - Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in config_local.py. - Execute the command - 'python setup.py'. - Now - you've configuration file with old schema. - 'git checkout master' to go to the latest code. - Make sure - you've latest code. 'git pull'. - Now - rerun 'python setup.py/pgAdmin4.py'. It should work. -- Thanks & Regards, Ashesh Vashi EnterpriseDB INDIA: Enterprise PostgreSQL Company *http://www.linkedin.com/in/asheshvashi* On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabuawala < murtuza.zabuawala@enterprisedb.com> wrote: > Hi, > > PFA patch to fix the issue for Pyhton3. > RM#1849 > > -- > Regards, > Murtuza Zabuawala > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < > fahar.abbas@enterprisedb.com> wrote: > >> Hi Dave, >> >> I have reopened following RM: >> ================================ >> https://redmine.postgresql.org/issues/1849 >> >> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page wrote: >> >>> Patch applied. >>> >>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>> ashesh.vashi@enterprisedb.com> wrote: >>> >>>> Hi Fahar, >>>> >>>> Please log the case on redmine. >>>> Please find the attached patch, please apply it locally, and test it. >>>> >>>> And, please update the case, and this mail chain accordingly. >>>> >>>> -- >>>> >>>> Thanks & Regards, >>>> >>>> Ashesh Vashi >>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>> >>>> >>>> >>>> *http://www.linkedin.com/in/asheshvashi* >>>> >>>> >>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>> fahar.abbas@enterprisedb.com> wrote: >>>> >>>>> Here is the output of if we copy config_local.py and execute python >>>>> setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: fahar.abbas@enterprisedb.com >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> python setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> User can not do any setup for web based now. >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: fahar.abbas@enterprisedb.com >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> >>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>> fahar.abbas@enterprisedb.com> wrote: >>>>> >>>>>> Dave, >>>>>> >>>>>> Testing Environment >>>>>> >>>>>> Ubuntu 16.04 Linux 64: >>>>>> -------------------------------- >>>>>> >>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>> >>>>>> >>>>>> 1) Install GIT >>>>>> >>>>>> sudo apt-get install git >>>>>> >>>>>> 2) Install pip3 >>>>>> >>>>>> sudo apt-get install python3-pip >>>>>> >>>>>> 3) Install virtualenv >>>>>> >>>>>> sudo pip3 install virtualenv >>>>>> >>>>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>>>> module >>>>>> >>>>>> sudo apt-get install libpq-dev >>>>>> >>>>>> sudo apt-get install python3-dev >>>>>> >>>>>> 5) Create virtual environment >>>>>> >>>>>> virtualenv -p python3 venv >>>>>> >>>>>> 6) Create mkdir Projects >>>>>> >>>>>> 7) Clone git repo in Projects >>>>>> >>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>> >>>>>> 8) activate virtual environment >>>>>> >>>>>> source venv/bin/activate >>>>>> >>>>>> 9) Install modules >>>>>> >>>>>> pip3 install -r requirements_py3.txt >>>>>> >>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>> Projects\pgAdmin4\web * >>>>>> >>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>> python setup.py >>>>>> >>>>>> If user does not create config_local.py and do Python setup.py for >>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>> >>>>>> Here is the output: >>>>>> ------------------------- >>>>>> >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>> >>>>>> >>>>>> Is this expected? >>>>>> >>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>> >>>>>>> Sure, >>>>>>> >>>>>>> Will test this thoroughly after complete investigation. >>>>>>> >>>>>>> Kind Regards, >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page >>>>>>> wrote: >>>>>>> >>>>>>>> Patch applied. >>>>>>>> >>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>> >>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>> >>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>> installations, if you have included them in 1.0) >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>> >>>>>>>>> Hi Dave, >>>>>>>>> >>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Friday, October 14, 2016, Dave Page wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave, >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>> >>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>> happy with it? >>>>>>>>>>>>> >>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>> >>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>> 'web' mode. >>>>>>>>>>>> - Apply the patch >>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>> >>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>> >>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>> the Security object. >>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hmm. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>> >>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>>>> >>>>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>>>> Sigh... Needs more thought. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>> >>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>> installations/databases. >>>>>>>>>> >>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>> >>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>> >>>>>>>>> Please find the updated patch. >>>>>>>>> >>>>>>>>> Otherwise - looks good to me. >>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks & Regards, >>>>>>>>> >>>>>>>>> Ashesh Vashi >>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>> >>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>> >>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>> appreciated)! >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > --001a114a8ef6f39f53053f473910 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
You were missing the other places, before the function - d= o_setup(..) call, where we are generating the other keys automatically.I've checked-in the code.

Fahar,
If you want to test the issue properly, please follow the belo= w steps:
- git checkout c4f1b8eb112e99d228c40a412020fa616dbe44f6<= /div>
=C2=A0 This was the commit-id, where we have the configuration sc= hema version '13'.
- Delete the existing pgadmin4.db
-= Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in config_local.p= y.
- Execute the command - 'python setup.py'.
-= Now - you've configuration file with old schema.
- 'git = checkout master' to go to the latest code.
- Make sure - you&= #39;ve latest code. 'git pull'.
- Now - rerun 'python= setup.py/pgAdmin4.py'. It = should work.


--

Thanks & Regards,

Ashesh= Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company

<= br>

<= a href=3D"http://www.linkedin.com/in/asheshvashi" target=3D"_blank">http= ://www.linkedin.com/in/asheshvashi


On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Za= buawala <murtuza.zabuawala@enterprisedb.com> wrote:
Hi,

PFA patch to fix the issue for Pyhton3.
RM#1849

--
Regards,
= Murtuz= a Zabuawala
EnterpriseDB:=C2=A0http://www.enterprisedb.com
The Enterprise PostgreSQL Company<= /span>

On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abba= s <fahar.abbas@enterprisedb.com> wrote:
Hi Dave,

I have reopened f= ollowing RM:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
https://redmine.postgresql.or= g/issues/1849

On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <dpage@pgadmin= .org> wrote:
Patch applied.

On Wed, Oct 19, = 2016 at 11:55 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com<= /a>> wrote:
Hi Fahar,

Please log the case on redmine.
P= lease find the attached patch, please apply it locally, and test it.
<= div>
And, please update the case, and this mail chain accordi= ngly.

--

Thanks & Regards,
<= br style=3D"font-style:italic">Ashesh Vas= hi
= EnterpriseDB INDIA: Enterprise PostgreSQL Company

<= br>

<= a href=3D"http://www.linkedin.com/in/asheshvashi" target=3D"_blank">http= ://www.linkedin.com/in/asheshvashi


On Wed, Oct 19, 2016 at = 3:47 PM, Fahar Abbas <fahar.abbas@enterprisedb.com> wrote:
Here is t= he output of if we copy config_local.py and execute python setup.pypgAdmin 4 - Application Initialisation
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D


The configuration database - '/home/fahar/.p= gadmin/pgadmin4.db' does not exist.
Entering initial setup mode= ...
NOTE: Configuring authentication for SERVER mode.


=C2=A0= =C2=A0=C2=A0 Enter the email address and password to use for the initial pg= Admin user=C2=A0=C2=A0=C2=A0=C2=A0 account:

Email address: fahar.abbas@ente= rprisedb.com
Password:
Retype password:
Traceback (most recen= t call last):
=C2=A0 File "setup.py", line 449, in <module&= gt;
=C2=A0=C2=A0=C2=A0 do_setup(app)
=C2=A0 File "setup.py"= , line 96, in do_setup
=C2=A0=C2=A0=C2=A0 password =3D encrypt_password(= p1)
=C2=A0 File "/home/fahar/venv/lib/python3.5/site-packages/= flask_security/utils.py", line 150, in encrypt_password
=C2=A0= =C2=A0=C2=A0 signed =3D get_hmac(password).decode('ascii')
= =C2=A0 File "/home/fahar/venv/lib/python3.5/site-packages/flask_s= ecurity/utils.py", line 108, in get_hmac
=C2=A0=C2=A0=C2=A0 &#= 39;set to "%s"' % _security.password_hash)
RuntimeError: T= he configuration value `SECURITY_PASSWORD_SALT` must not be None when the v= alue of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512"
pyt= hon setup.py
pgAdmin 4 - Application Initialisation
=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D

User can not do any setup= for web based now.


The configuration database - '= ;/home/fahar/.pgadmin/pgadmin4.db' does not exist.
Entering ini= tial setup mode...
NOTE: Configuring authentication for SERVER mode.
=

=C2=A0=C2=A0=C2=A0 Enter the email address and password to use for = the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 account:

Email addr= ess: faha= r.abbas@enterprisedb.com
Password:
Retype password:
Traceback= (most recent call last):
=C2=A0 File "setup.py", line 449, in= <module>
=C2=A0=C2=A0=C2=A0 do_setup(app)
=C2=A0 File "se= tup.py", line 96, in do_setup
=C2=A0=C2=A0=C2=A0 password =3D encry= pt_password(p1)
=C2=A0 File "/home/fahar/venv/lib/python3.5/si= te-packages/flask_security/utils.py", line 150, in encrypt_passwo= rd
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac(password).decode('asc= ii')
=C2=A0 File "/home/fahar/venv/lib/python3.5/site-pack= ages/flask_security/utils.py", line 108, in get_hmac
=C2=A0=C2= =A0=C2=A0 'set to "%s"' % _security.password_hash)
Run= timeError: The configuration value `SECURITY_PASSWORD_SALT` must not be Non= e when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512&= quot;

= On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas <fahar.abbas@ente= rprisedb.com> wrote:
Dave,

Testing Environment
= =C2=A0
Ubuntu 16.04 Linux 64:
--------------------------------

pg-AdminIV Develop= ment Environment Setup for Ubuntu=C2=A0 :


1) Install GIT

sudo apt-get install git<= /font>

2) Install pip3

sudo apt-get in= stall python3-pip

3) Install virtualenv

sudo pip3 install virtualenv

4) install below dependency as it is re= quired for psycopg2 & pycrypto module

sudo apt-get install libpq-dev =

sudo apt-get install pyth= on3-dev

5) C= reate virtual environment

virtualenv -p python3 venv

6) Create mkdir Projects

7) Clone git repo in Projects

git clone = http://g= it.postgresql.org/git/pgadmin4.git

8) activate virtual environment=

source venv= /bin/activate

9) Install modules

pip3 install -r requirements_py3.txt

10) Edit= the config.py file to config_local.py =C2=A0resides in Projects\pgAdmin4\web=C2=A0=C2=A0

11)Now run setup.py file =C2=A0(\Project= s\pgAdmin4\web)

=C2=A0 =C2=A0 python setup.py

If user does not create config_local.= py and do Python setup.py for new Development then SECURITY_PASSWORD_SALT m= essage is also displayed:

Here is the output:
--------= -----------------

python setup.py
pgAdmin 4 - Applica= tion Initialisation
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

The configuration database - '/home/fahar/.pgadmin/pgadmin4.d= b' does not exist.
Entering initial setup mode...
NOTE: Configuri= ng authentication for SERVER mode.


=C2=A0=C2=A0=C2=A0 Enter the = email address and password to use for the initial pgAdmin user=C2=A0=C2=A0= =C2=A0=C2=A0 account:

Email address: fahar.abbas@enterprisedb.com
Pass= word:
Retype password:
Traceback (most recent call last):
=C2=A0 = File "setup.py", line 449, in <module>
=C2=A0=C2=A0=C2= =A0 do_setup(app)
=C2=A0 File "setup.py", line 96, in do_setup=
=C2=A0=C2=A0=C2=A0 password =3D encrypt_password(p1)
=C2=A0 File &qu= ot;/home/fahar/venv/lib/python3.5/site-packages/flask_security/ut= ils.py", line 150, in encrypt_password
=C2=A0=C2=A0=C2=A0 signed = =3D get_hmac(password).decode('ascii')
=C2=A0 File "/h= ome/fahar/venv/lib/python3.5/site-packages/flask_security/utils.p= y", line 108, in get_hmac
=C2=A0=C2=A0=C2=A0 'set to "%s&q= uot;' % _security.password_hash)
RuntimeError: The configuration val= ue `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PA= SSWORD_HASH` is set to "pbkdf2_sha512"
(venv) fahar@fahar-virt= ual-machine:~/Projects/pgadmin4/web$


Is this expected?

On Wed, Oct 19, 2016 at 1:37 PM, Fahar= Abbas <fahar.abbas@enterprisedb.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">
Sure,

Will test this thoroughly after complete investigation.

Kind = Regards,

On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <dpage@pgadmin.org><= /span> wrote:
Patch appl= ied.

Fahar, can you please test this thoroughly in deskt= op and server modes, with both fresh and upgraded installations?
=
=
Packagers: This change means that packages are no longer for= ced to create a config_local.py file, and there is no longer any need to ex= plicitly set=C2=A0SECURITY_PASSWORD_SALT,= =C2=A0SECURITY_KEY and=C2=A0CS= RF_SESSION_KEY in the config (in fact, they should be removed for new insta= llations, if you have included them in 1.0)

Thanks.


On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Dave,

<= /div>
On Sat, Oct 15, 2016 at 8:02 AM, Dave = Page <dpage@pgadmin.org> wrote:
Hi


On Friday, October 14, 2016, Dave Pa= ge <dpage@pgadmin= .org> wrote:
Hi=

On Thursday, October 13, 2016, Ashesh Vashi <ashesh.vashi@ent= erprisedb.com> wrote:
Hi Dave,

On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <= span dir=3D"ltr"><dpage@pgadmin.org> wrote:
Hi Ashesh,

Can= you please review the attached patch, and apply if you're happy with i= t?
Overall the patch looked good to me.
Bu= t - I encounter an issue in 'web' mode, which wont happen with '= ;runtime'.

Steps for reproduction on existing = pgAdmin 4 environment with 'web' mode.
- Apply the patch<= /div>
- Start the pgAdmin4 application (stand alone application).
=
- Open pgAdmin home page.
- Log out (if already login).

And, you will see an exception.

I have figure out the issue with the patch.
We were setting th= e SECURITY_PASSWORD_SALT, after initializing the Security object.
Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT prop= erly.

Hmm.
=C2=A0

=
I had moved the Security object initialization after fetching these co= nfigurations from the database.
I have attached a addon patch for= the same.

OK, than= ks.
=C2=A0

Now - I run into another issue.
Because - the ex= isting password was hashed using the old SECURITY_PASSWORD_SALT, I am no mo= re able to login to pgAdmin 4.

I think - we need t= o think about different strategy for upgrading the configuration file in th= e 'web' mode.
I was thinking - we can store the existing = security configurations in the database during upgrade process in 'web&= #39; mode.

My conce= rn with that is that we'll likely be storing the default config values = in many cases, thus for those users, perpetuating the problem.
I guess what we need to do is re-encrypt the password during t= he upgrade - however, that makes me think; we then have both the key and th= e encrypted passwords in the same database which is clearly not a good idea= . Sigh... Needs more thought.=C2=A0

=
OK, so I've been thinking about this and experimenting for a= couple of hours, as well as annoying the crap out of Magnus by thinking ou= t loud in his general direction, and it looks like this isn't a major p= roblem as from what I can see, =C2=A0SECURITY_PASSWORD_SALT is (aside from = really being a key not a salt) not the only salting that's done.=C2=A0<= /div>

It looks like it's used system-wide as the key= to generate an HMAC of the users password, which is then passed to passlib= which salts and hashes it. I did some testing, and found that two users wi= th the same password end up with different hashes in the database, so clear= ly there is also per-user salting happening. I also created two users, then= dropped the database and created the same user accounts with the same pass= words again, and found that the resulting hashes were different in both dat= abases - thus there is something else ensuring the hashes are unique across= different installations/databases.

So, I believe = we can do as you suggest and migrate existing values for SECURITY_PASSWORD_= SALT, given that there's clearly some other per user and per installati= on/database salting going on anyway. New installations can have the random = value for SECURITY_PASSWORD_SALT.
We do = not need to generate the random SECURITY_PASSWORD_SALT during upgrade mode,= which was wrong added in my addon patch.

Please f= ind the updated patch.

Otherwise - looks good to m= e.
Please commit the new patch (if you're ok with the change)= .


--=

= Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA:=C2=A0Enterprise PostgreSQL Company



I don't believe SECURITY_KEY and=C2= =A0CSRF_SESSION_KEY are issues either, as they're used for purposes tha= t are essentially ephemeral, and thus can be changed during an upgrade.

Adding Magnus as I'd appreciate any thoughts he m= ay have.

Patch attached - please review (Ashesh, b= ut others too would be appreciated)!

Thanks.
=


--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
=
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enter= prise PostgreSQL Company



--
Syed Fahar Abbas
Quality Manag= ement Group

EnterpriseDB Corporation
Phone Office: +92-51-8= 35-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707<= br>Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com


--
Syed Fahar Abbas
Quality Management Group

EnterpriseDB Corporation
Phone Of= fice: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-33= 3-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com



--
Syed Fahar Abbas
Quality Management G= roup

EnterpriseDB Corporation
Phone Office: +92-51-835-8874=
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skyp= e ID: syed.fahar.abbas
Website: www.enterprisedb.com




--
Dave Page
Blog: http://pgsnake.blogspot.com
= Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise = PostgreSQL Company


--
Syed Fahar Abbas
<= /div>
Quality Management Group

EnterpriseDB CorporationPhone Office: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: += 92-333-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com
<= /div>


--001a114a8ef6f39f53053f473910--