Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bx6Pp-0007lA-Ph for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 06:01:22 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1bx6Pp-000412-CL for pgadmin-hackers@arkaria.postgresql.org; Thu, 20 Oct 2016 06:01:21 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1bx6Pn-00040w-P5 for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 06:01:20 +0000 Received: from mail-qk0-x232.google.com ([2607:f8b0:400d:c09::232]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1bx6Pk-0001po-5G for pgadmin-hackers@postgresql.org; Thu, 20 Oct 2016 06:01:18 +0000 Received: by mail-qk0-x232.google.com with SMTP id z190so73153889qkc.2 for ; Wed, 19 Oct 2016 23:01:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=H5XTt/7i4ei6RNVgr4mZPdclNjeGWuN1C+vaQASy3H4=; b=EyPPFAqH4lLDPvTrxhta1FRxFflbyvVobNbjSVzu3bNGhIJMW06Nicq/N83rYKq55i y+NGWCCEIAxUAvkoj407v26cIoavP/Cvp00KJoyu7xH7vpu+89DyWilF4SDVbLKxbLVB ph3GO/EgbK9MAxeEZDsbogyCWr6yvolIJZCtgaVKBkAZA9DoUIDvICoZ0WjsDiPeDiEU BJzZupx8GYawWqVvNJmaSeWLTH0r+lKgzhBK8jnGtu6hxkYphZVvOfpd63ShUIqcW2Ec +gziGbejl2S6lEb1FJP5lmyuio+nfjL07A7eyKiVdB8dNNTKZ6n2m38fHF/OSF1koAnT QNTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=H5XTt/7i4ei6RNVgr4mZPdclNjeGWuN1C+vaQASy3H4=; b=XbfA4BbuiO7cc5tUVWlmAK1tnSrmN1PvlnDhxMRjRNYPsk3M9PDUNQj+Sc3BDdHmFP 3PQq0YI0tUnLnTc7oq/EKjqyt814h7Ql/UvN6TeWbeVzqF8HNUgrIl+Ln8PKIT8rLLwU GnJDm7Jn/OfXRV0nZLQ3i4fbXGG9TsI9fgEkte4KsU8FbGGylvp+xmIRCcWjOb7olq/L ItCAoE1thcQT861DhOTkxoYe+MIfSIdYe2eHxw/lHOX6N9wrI+UohZD4b2jk3jneDU0A 5/A8wdcVF+PUbBvmbU5Cp5AkVJ3X1TtK9md3vjtIFC71uhOjf26IhDcLvo1u1/6mBEzI P3nA== X-Gm-Message-State: AA6/9RlLdTryBvvd9Nq29A9O7dXiYzoOzM2gA7OAyHNox1tMipENVG2/XOzAcYwgCoisESpzdIUR0JQjZOWNI6on X-Received: by 10.55.212.221 with SMTP id s90mr10800290qks.169.1476943275186; Wed, 19 Oct 2016 23:01:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.130.133 with HTTP; Wed, 19 Oct 2016 23:00:54 -0700 (PDT) In-Reply-To: References: From: Murtuza Zabuawala Date: Thu, 20 Oct 2016 11:30:54 +0530 Message-ID: Subject: Re: RM1849: Auto-generating security keys To: Fahar Abbas Cc: Dave Page , Ashesh Vashi , pgadmin-hackers , Josh Berkus , =?UTF-8?B?RGV2cmltIEfDnE5Ew5xa?= , Magnus Hagander , Sandeep Thakkar , Hamid Quddus Akhtar Content-Type: multipart/alternative; boundary=001a1149ae0c456f93053f45a7f1 X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgadmin-hackers Precedence: bulk Sender: pgadmin-hackers-owner@postgresql.org --001a1149ae0c456f93053f45a7f1 Content-Type: text/plain; charset=UTF-8 Could you delete 'keys' table from pgadmin4.db file & try again? -- Regards, Murtuza Zabuawala EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Thu, Oct 20, 2016 at 11:26 AM, Fahar Abbas wrote: > Murtaza, > > I have applied this patch and there is no success on new pgAdmin4 setup as > well as existing pgAdmin4 setup. > > On Thu, Oct 20, 2016 at 10:45 AM, Murtuza Zabuawala enterprisedb.com> wrote: > >> Hi, >> >> PFA patch to fix the issue for Pyhton3. >> RM#1849 >> >> -- >> Regards, >> Murtuza Zabuawala >> EnterpriseDB: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >> fahar.abbas@enterprisedb.com> wrote: >> >>> Hi Dave, >>> >>> I have reopened following RM: >>> ================================ >>> https://redmine.postgresql.org/issues/1849 >>> >>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page wrote: >>> >>>> Patch applied. >>>> >>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>> ashesh.vashi@enterprisedb.com> wrote: >>>> >>>>> Hi Fahar, >>>>> >>>>> Please log the case on redmine. >>>>> Please find the attached patch, please apply it locally, and test it. >>>>> >>>>> And, please update the case, and this mail chain accordingly. >>>>> >>>>> -- >>>>> >>>>> Thanks & Regards, >>>>> >>>>> Ashesh Vashi >>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>> >>>>> >>>>> >>>>> *http://www.linkedin.com/in/asheshvashi* >>>>> >>>>> >>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>> fahar.abbas@enterprisedb.com> wrote: >>>>> >>>>>> Here is the output of if we copy config_local.py and execute python >>>>>> setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> User can not do any setup for web based now. >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> >>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>> >>>>>>> Dave, >>>>>>> >>>>>>> Testing Environment >>>>>>> >>>>>>> Ubuntu 16.04 Linux 64: >>>>>>> -------------------------------- >>>>>>> >>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>> >>>>>>> >>>>>>> 1) Install GIT >>>>>>> >>>>>>> sudo apt-get install git >>>>>>> >>>>>>> 2) Install pip3 >>>>>>> >>>>>>> sudo apt-get install python3-pip >>>>>>> >>>>>>> 3) Install virtualenv >>>>>>> >>>>>>> sudo pip3 install virtualenv >>>>>>> >>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>> pycrypto module >>>>>>> >>>>>>> sudo apt-get install libpq-dev >>>>>>> >>>>>>> sudo apt-get install python3-dev >>>>>>> >>>>>>> 5) Create virtual environment >>>>>>> >>>>>>> virtualenv -p python3 venv >>>>>>> >>>>>>> 6) Create mkdir Projects >>>>>>> >>>>>>> 7) Clone git repo in Projects >>>>>>> >>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>> >>>>>>> 8) activate virtual environment >>>>>>> >>>>>>> source venv/bin/activate >>>>>>> >>>>>>> 9) Install modules >>>>>>> >>>>>>> pip3 install -r requirements_py3.txt >>>>>>> >>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>> Projects\pgAdmin4\web * >>>>>>> >>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>> python setup.py >>>>>>> >>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>> >>>>>>> Here is the output: >>>>>>> ------------------------- >>>>>>> >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: fahar.abbas@enterprisedb.com >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>> >>>>>>> >>>>>>> Is this expected? >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>> fahar.abbas@enterprisedb.com> wrote: >>>>>>> >>>>>>>> Sure, >>>>>>>> >>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>> >>>>>>>> Kind Regards, >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Patch applied. >>>>>>>>> >>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>> >>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>> >>>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Dave, >>>>>>>>>> >>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Friday, October 14, 2016, Dave Page >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>> ashesh.vashi@enterprisedb.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>>> happy with it? >>>>>>>>>>>>>> >>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>> >>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>>> 'web' mode. >>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>> >>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>> >>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>>> the Security object. >>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hmm. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>> >>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>> problem. >>>>>>>>>>>> >>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>> >>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>> installations/databases. >>>>>>>>>>> >>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>> >>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>> >>>>>>>>>> Please find the updated patch. >>>>>>>>>> >>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks & Regards, >>>>>>>>>> >>>>>>>>>> Ashesh Vashi >>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>> >>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>> >>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>>> appreciated)! >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dave Page >>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>> Twitter: @pgsnake >>>>>>>>> >>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > --001a1149ae0c456f93053f45a7f1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Could you delete 'keys' table from pgadmin4.db fil= e & try again?=C2=A0

=
--
Regards,
= Murtuza Zabuawala
EnterpriseDB:=C2=A0http://www.enterprisedb.com
The Enterprise PostgreSQL Compa= ny

On Thu, Oct 20, 2016 at 11:26 AM, Fahar Abba= s <fahar.abbas@enterprisedb.com> wrote:
Murtaza,

I have= applied this patch and there is no success on new pgAdmin4 setup as well a= s existing pgAdmin4 setup.

On Thu, Oct 20, 2016 at 10:45 AM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote= :
Hi,

PFA patch to fix the issue for Pyhton3.
RM#1849

--
Regards,
Murtuza Zabuawala
EnterpriseDB:=C2=A0http://www.enterprisedb.com
The Enterprise PostgreSQ= L Company

On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abba= s <fahar.abbas@enterprisedb.com> wrote:
Hi Dave,

I have reopened f= ollowing RM:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
https://redmine.postgresql.or= g/issues/1849

On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <dpage@pgadmin= .org> wrote:
Patch applied.

On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Fahar,

Please log the case on redmine.
Please find the attached patch, please= apply it locally, and test it.

And, please update= the case, and this mail chain accordingly.

=

--=

Thanks &= ; Regards,

Ashesh Vashi
EnterpriseDB INDIA= : Enterpri= se PostgreSQL Company

<= br>

<= a href=3D"http://www.linkedin.com/in/asheshvashi" target=3D"_blank">http= ://www.linkedin.com/in/asheshvashi


On We= d, Oct 19, 2016 at 3:47 PM, Fahar Abbas <fahar.abbas@enterprise= db.com> wrote:
Here is the output of if we copy config_local.py and execute pyth= on setup.py
pgAdmin 4 - Application Initialisation
=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D


The configuration database - &#= 39;/home/fahar/.pgadmin/pgadmin4.db' does not exist.
Entering i= nitial setup mode...
NOTE: Configuring authentication for SERVER mode.

=C2=A0=C2=A0=C2=A0 Enter the email address and password to use fo= r the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 account:

Email ad= dress: fa= har.abbas@enterprisedb.com
Password:
Retype password:
Traceba= ck (most recent call last):
=C2=A0 File "setup.py", line 449, = in <module>
=C2=A0=C2=A0=C2=A0 do_setup(app)
=C2=A0 File "= setup.py", line 96, in do_setup
=C2=A0=C2=A0=C2=A0 password =3D enc= rypt_password(p1)
=C2=A0 File "/home/fahar/venv/lib/python3.5/= site-packages/flask_security/utils.py", line 150, in encrypt_pass= word
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac(password).decode('ascii')
=C2=A0 File "/home/fahar/venv/lib/python3.5/site-pa= ckages/flask_security/utils.py", line 108, in get_hmac
=C2=A0= =C2=A0=C2=A0 'set to "%s"' % _security.password_hash)
= RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be = None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha5= 12"
python setup.py
pgAdmin 4 - Application Initialisation
= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

User can n= ot do any setup for web based now.


The configuration = database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist.<= br>Entering initial setup mode...
NOTE: Configuring authentication for S= ERVER mode.


=C2=A0=C2=A0=C2=A0 Enter the email address and passw= ord to use for the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 account:
Email address: fahar.abbas@enterprisedb.com
Password:
Retype passwo= rd:
Traceback (most recent call last):
=C2=A0 File "setup.py&quo= t;, line 449, in <module>
=C2=A0=C2=A0=C2=A0 do_setup(app)
=C2= =A0 File "setup.py", line 96, in do_setup
=C2=A0=C2=A0=C2=A0 p= assword =3D encrypt_password(p1)
=C2=A0 File "/home/fahar/venv/lib/= python3.5/site-packages/flask_security/utils.py", line 150, = in encrypt_password
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac(password).dec= ode('ascii')
=C2=A0 File "/home/fahar/venv/lib/python3= .5/site-packages/flask_security/utils.py", line 108, in get_= hmac
=C2=A0=C2=A0=C2=A0 'set to "%s"' % _security.pass= word_hash)
RuntimeError: The configuration value `SECURITY_PASSWORD_SALT= ` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to &qu= ot;pbkdf2_sha512"

On Wed, Oct 19, 2016= at 3:03 PM, Fahar Abbas <fahar.abbas@enterprisedb.com><= /span> wrote:
= Dave,

Testing Environment
=C2=A0
Ubuntu= 16.04 Linux 64:
--------------------------------

pg-AdminIV Development Environment Setup for Ubuntu= =C2=A0 :


1) Install GIT

sudo apt-get i= nstall git

2= ) Install pip3

sudo apt-get install python3-pip

3) Install virtualenv

sudo pip3 install virtualenv

4) install below depen= dency as it is required for psycopg2 & pycrypto module

sudo apt-get install libpq-d= ev

sudo apt= -get install python3-dev

5) Create virtual environment

virtualenv -p python3 venv

=

6) Create mkdir Projects

7) Clone git repo in P= rojects

= git clone http://git.postgresql.org/git/pgadmin= 4.git

8) activate virtual environment

source venv/bin/activate

9) Install modules

pip3 install -r requirements= _py3.txt

10) Edit the config.py file to config_local.py= =C2=A0resides in Projects\pgAdmin4\web=C2=A0=C2=A0

11)Now ru= n setup.py file =C2=A0(\Project= s\pgAdmin4\web)

=C2=A0 = =C2=A0 python setup.py
If user does not create config_local.py and do Python setup.py for new Dev= elopment then SECURITY_PASSWORD_SALT message is also displayed:

Here is the output:
-------------------------

p= ython setup.py
pgAdmin 4 - Application Initialisation
=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D


The configuration database - &#= 39;/home/fahar/.pgadmin/pgadmin4.db' does not exist.
Entering i= nitial setup mode...
NOTE: Configuring authentication for SERVER mode.

=C2=A0=C2=A0=C2=A0 Enter the email address and password to use fo= r the initial pgAdmin user=C2=A0=C2=A0=C2=A0=C2=A0 account:

Email ad= dress: fa= har.abbas@enterprisedb.com
Password:
Retype password:
Traceba= ck (most recent call last):
=C2=A0 File "setup.py", line 449, = in <module>
=C2=A0=C2=A0=C2=A0 do_setup(app)
=C2=A0 File "= setup.py", line 96, in do_setup
=C2=A0=C2=A0=C2=A0 password =3D enc= rypt_password(p1)
=C2=A0 File "/home/fahar/venv/lib/python3.5/= site-packages/flask_security/utils.py", line 150, in encrypt_pass= word
=C2=A0=C2=A0=C2=A0 signed =3D get_hmac(password).decode('ascii')
=C2=A0 File "/home/fahar/venv/lib/python3.5/site-pa= ckages/flask_security/utils.py", line 108, in get_hmac
=C2=A0= =C2=A0=C2=A0 'set to "%s"' % _security.password_hash)
= RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be = None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha5= 12"
(venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web= $


Is this expected?

On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas <= fahar.abbas@enterprisedb.com> wrote:
Sure,

Will test this thor= oughly after complete investigation.

Kind Regards,

On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <dpage@pgadmin.org> wrote:
Patch appli= ed.

Fahar, can you please test this thoroughly in deskto= p and server modes, with both fresh and upgraded installations?
<= br>
<= br>
Packagers: This change means that packages are no longer forc= ed to create a config_local.py file, and there is no longer any need to exp= licitly set=C2=A0SECURITY_PASSWORD_SALT,= =C2=A0SECURITY_KEY and=C2=A0CS= RF_SESSION_KEY in the config (in fact, they should be removed for new insta= llations, if you have included them in 1.0)

Thanks.


On Wed, Oct 19, 2016 at 6:46 AM, Ash= esh Vashi <ashesh.vashi@enterprisedb.com> w= rote:
Hi Dave,

O= n Sat, Oct 15, 2016 at 8:02 AM, Dave Page <dpage@pgadmin.org> wrote:
Hi


On Friday, October 14, 2016, Dave Page <dpage@pgadmin.org> wrote:=
Hi

On Thursday= , October 13, 2016, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:
Hi Dave,

On Tue, Oct 11, 2016 at 9:10 PM, Dave Page &l= t;dpage@pgadmin.org> wrote:
Hi Ashesh,

Can you please review = the attached patch, and apply if you're happy with it?
Overall the patch looked good to me.
But - I encounter an = issue in 'web' mode, which wont happen with 'runtime'.

Steps for reproduction on existing pgAdmin 4 environme= nt with 'web' mode.
- Apply the patch
- Start t= he pgAdmin4 application (stand alone application).
- Open pgAdmin= home page.
- Log out (if already login).

And, you will see an exception.

I have figure ou= t the issue with the patch.
We were setting the SECURITY_PASSWORD= _SALT, after initializing the Security object.
Hence - it could n= ot set the SECURITY_KEY, and SECURITY_PASSWORD_SALT properly.

Hmm.
=C2=A0

I had moved th= e Security object initialization after fetching these configurations from t= he database.
I have attached a addon patch for the same.

OK, thanks.
=C2= =A0

= Now - I run into another issue.
Because - the existing password w= as hashed using the old SECURITY_PASSWORD_SALT, I am no more able to login = to pgAdmin 4.

I think - we need to think about dif= ferent strategy for upgrading the configuration file in the 'web' m= ode.
I was thinking - we can store the existing security configur= ations in the database during upgrade process in 'web' mode.
<= /div>

My concern with that is t= hat we'll likely be storing the default config values in many cases, th= us for those users, perpetuating the problem.

I gu= ess what we need to do is re-encrypt the password during the upgrade - howe= ver, that makes me think; we then have both the key and the encrypted passw= ords in the same database which is clearly not a good idea. Sigh... Needs m= ore thought.=C2=A0

OK, so= I've been thinking about this and experimenting for a couple of hours,= as well as annoying the crap out of Magnus by thinking out loud in his gen= eral direction, and it looks like this isn't a major problem as from wh= at I can see, =C2=A0SECURITY_PASSWORD_SALT is (aside from really being a ke= y not a salt) not the only salting that's done.=C2=A0

It looks like it's used system-wide as the key to generate an H= MAC of the users password, which is then passed to passlib which salts and = hashes it. I did some testing, and found that two users with the same passw= ord end up with different hashes in the database, so clearly there is also = per-user salting happening. I also created two users, then dropped the data= base and created the same user accounts with the same passwords again, and = found that the resulting hashes were different in both databases - thus the= re is something else ensuring the hashes are unique across different instal= lations/databases.

So, I believe we can do as you = suggest and migrate existing values for SECURITY_PASSWORD_SALT, given that = there's clearly some other per user and per installation/database salti= ng going on anyway. New installations can have the random value for SECURIT= Y_PASSWORD_SALT.
We do not need to gener= ate the random SECURITY_PASSWORD_SALT during upgrade mode, which was wrong = added in my addon patch.

Please find the updated p= atch.

Otherwise - looks good to me.
Plea= se commit the new patch (if you're ok with the change).


Thanks & Regar= ds,

<= span style=3D"font-style:italic">Ashesh Vashi
EnterpriseDB INDIA:=C2=A0= Enterpris= e PostgreSQL Company



<= div>I don't believe SECURITY_KEY and=C2=A0CSRF_SESSION_KEY are issues e= ither, as they're used for purposes that are essentially ephemeral, and= thus can be changed during an upgrade.

Adding Mag= nus as I'd appreciate any thoughts he may have.

Patch attached - please review (Ashesh, but others too would be appreciat= ed)!

Thanks.


--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsna= ke

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Co= mpany





--
Dave PageBlog: http://pgs= nake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.= com
The Enterprise PostgreSQL Company



--
Syed F= ahar Abbas
Quality Management Group

EnterpriseDB= Corporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-846680= 3
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Webs= ite: www.enterpri= sedb.com



--
Syed Fahar = Abbas
Quality Management Group

EnterpriseDB Corp= oration
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803=
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: = www.enterprisedb.= com



--
Syed Fahar Abbas
Q= uality Management Group

EnterpriseDB Corporation
Phone Offi= ce: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-33= 3-5409707
Skype ID: syed.fahar.abbas
Website: www.enterprisedb.com




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB= UK: http://www.e= nterprisedb.com
The Enterprise PostgreSQL Company


--
Syed Faha= r Abbas
Quality Management Group

EnterpriseDB Co= rporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-8466803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website= : www.enterprised= b.com



--
Syed= Fahar Abbas
Quality Management Group

Enterprise= DB Corporation
Phone Office: +92-51-835-8874
Phone Direct: +92-51-846= 6803
Mobile: +92-333-5409707
Skype ID: syed.fahar.abbas
Website: <= a href=3D"http://www.enterprisedb.com" target=3D"_blank">www.enterprisedb.c= om
--001a1149ae0c456f93053f45a7f1--