public inbox for [email protected]help / color / mirror / Atom feed
RM1849: Auto-generating security keys 33+ messages / 6 participants [nested] [flat]
* RM1849: Auto-generating security keys @ 2016-10-11 15:40 Dave Page <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Dave Page @ 2016-10-11 15:40 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]> Hi Ashesh, Can you please review the attached patch, and apply if you're happy with it? The purpose is to auto-generate the various security keys that are currently in the configuration file, and store them in the SQLite database. This allows us to remove the checks for config_local.py and the hard-coded default keys which are causing some problems with packaging: - Hard coded defaults are fine for Desktop mode, and packages generally aim to make that work primarily. - Hard coded defaults are a security risk for Server mode, hence we currently require the user to manually setup keys, which is currently being overridden by packagers for Desktop mode. This change ensures that we have unique security keys for every installation, whether running in desktop or server mode (generated from os.urandom). Thanks! -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [text/x-diff] auto_generate_security_keys.diff (8.6K, 3-auto_generate_security_keys.diff) download | inline diff: diff --git a/web/config.py b/web/config.py index 20714f9..8411d79 100644 --- a/web/config.py +++ b/web/config.py @@ -140,21 +140,13 @@ DEFAULT_SERVER_PORT = 5050 # Enable CSRF protection? CSRF_ENABLED = True -# Secret key for signing CSRF data. Override this in config_local.py if -# running on a web server -CSRF_SESSION_KEY = 'SuperSecret1' - -# Secret key for signing cookies. Override this in config_local.py if -# running on a web server -SECRET_KEY = 'SuperSecret2' - -# Salt used when hashing passwords. Override this in config_local.py if -# running on a web server -SECURITY_PASSWORD_SALT = 'SuperSecret3' - # Hashing algorithm used for password storage SECURITY_PASSWORD_HASH = 'pbkdf2_sha512' +# NOTE: CSRF_SESSION_KEY, SECRET_KEY and SECURITY_PASSWORD_SALT are no +# longer part of the main configuration, but are stored in the +# configuration databases 'keys' table and are auto-generated. + # Should HTML be minified on the fly when not in debug mode? # Note: This is disabled by default as it will error when processing the # docs. If the serving of docs is handled by an Apache HTTPD diff --git a/web/pgAdmin4.py b/web/pgAdmin4.py index 1fb34f9..f894f8b 100644 --- a/web/pgAdmin4.py +++ b/web/pgAdmin4.py @@ -32,18 +32,6 @@ config.SETTINGS_SCHEMA_VERSION = SCHEMA_VERSION # Sanity checks ########################################################################## -# Check for local settings if running in server mode -if config.SERVER_MODE is True: - local_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), - 'config_local.py') - if not os.path.isfile(local_config): - print("The configuration file %s does not exist.\n" % local_config) - print("Before running this application, ensure that config_local.py has been created") - print("and sets values for SECRET_KEY, SECURITY_PASSWORD_SALT and CSRF_SESSION_KEY") - print("at bare minimum. See config.py for more information and a complete list of") - print("settings. Exiting...") - sys.exit(1) - # Check if the database exists. If it does not, create it. if not os.path.isfile(config.SQLITE_PATH): setupfile = os.path.join(os.path.dirname(os.path.realpath(__file__)), diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py index d988172..21fe636 100644 --- a/web/pgadmin/__init__.py +++ b/web/pgadmin/__init__.py @@ -26,7 +26,7 @@ from pgadmin.utils.session import create_session_interface from werkzeug.local import LocalProxy from werkzeug.utils import find_modules -from pgadmin.model import db, Role, Server, ServerGroup, User, Version +from pgadmin.model import db, Role, Server, ServerGroup, User, Version, Keys # Configuration settings import config @@ -127,11 +127,6 @@ def create_app(app_name=config.APP_NAME): app.config.update(dict(PROPAGATE_EXCEPTIONS=True)) ########################################################################## - # Setup session management - ########################################################################## - app.session_interface = create_session_interface(app) - - ########################################################################## # Setup logging and log the application startup ########################################################################## @@ -222,7 +217,19 @@ def create_app(app_name=config.APP_NAME): from setup import do_upgrade do_upgrade(app, user_datastore, security, version) + ########################################################################## + # Setup security + ########################################################################## + with app.app_context(): + config.CSRF_SESSION_KEY = Keys.query.filter_by(name = 'CSRF_SESSION_KEY').first().value + config.SECRET_KEY = Keys.query.filter_by(name = 'SECRET_KEY').first().value + config.SECURITY_PASSWORD_SALT = Keys.query.filter_by(name = 'SECURITY_PASSWORD_SALT').first().value + + app.session_interface = create_session_interface(app) + + ########################################################################## # Load all available server drivers + ########################################################################## driver.init_app(app) ########################################################################## diff --git a/web/pgadmin/model/__init__.py b/web/pgadmin/model/__init__.py index 019e9b1..9727d2b 100644 --- a/web/pgadmin/model/__init__.py +++ b/web/pgadmin/model/__init__.py @@ -29,7 +29,7 @@ from flask_sqlalchemy import SQLAlchemy # ########################################################################## -SCHEMA_VERSION = 13 +SCHEMA_VERSION = 14 ########################################################################## # @@ -207,3 +207,10 @@ class Process(db.Model): end_time = db.Column(db.String(), nullable=True) exit_code = db.Column(db.Integer(), nullable=True) acknowledge = db.Column(db.String(), nullable=True) + + +class Keys(db.Model): + """Define the keys table.""" + __tablename__ = 'keys' + name = db.Column(db.String(), nullable=False, primary_key=True) + value = db.Column(db.String(), nullable=False) \ No newline at end of file diff --git a/web/setup.py b/web/setup.py index 64273fb..90697fa 100755 --- a/web/setup.py +++ b/web/setup.py @@ -10,6 +10,7 @@ """Perform the initial setup of the application, by creating the auth and settings database.""" +import base64 import getpass import os import random @@ -22,7 +23,7 @@ from flask_security import Security, SQLAlchemyUserDatastore from flask_security.utils import encrypt_password from pgadmin.model import db, Role, User, Server, \ - ServerGroup, Version + ServerGroup, Version, Keys # Configuration settings import config @@ -40,6 +41,16 @@ if hasattr(__builtins__, 'raw_input'): def do_setup(app): """Create a new settings database from scratch""" + + # Get some defaults for the various keys + with app.app_context(): + config.CSRF_SESSION_KEY = base64.urlsafe_b64encode(os.urandom(32)) + app.jinja_env.globals['config']['CSRF_SESSION_KEY'] = config.CSRF_SESSION_KEY + config.SECRET_KEY = base64.urlsafe_b64encode(os.urandom(32)) + app.jinja_env.globals['config']['SECRET_KEY'] = config.SECRET_KEY + config.SECURITY_PASSWORD_SALT = base64.urlsafe_b64encode(os.urandom(32)) + app.jinja_env.globals['config']['SECURITY_PASSWORD_SALT'] = config.SECURITY_PASSWORD_SALT + if config.SERVER_MODE is False: print("NOTE: Configuring authentication for DESKTOP mode.") email = config.DESKTOP_USER @@ -116,6 +127,17 @@ def do_setup(app): name='ConfigDB', value=config.SETTINGS_SCHEMA_VERSION ) db.session.merge(version) + db.session.commit() + + # Create the keys + key = Keys(name='CSRF_SESSION_KEY', value=config.CSRF_SESSION_KEY) + db.session.merge(key) + + key = Keys(name='SECRET_KEY', value=config.SECRET_KEY) + db.session.merge(key) + + key = Keys(name='SECURITY_PASSWORD_SALT', value=config.SECURITY_PASSWORD_SALT) + db.session.merge(key) db.session.commit() @@ -329,6 +351,23 @@ ALTER TABLE SERVER ADD COLUMN discovery_id TEXT """) + if int(version.value) < 14: + db.engine.execute(""" +CREATE TABLE keys ( + name TEST NOT NULL, + value TEXT NOT NULL, + PRIMARY KEY (name)) + """) + + sql = "INSERT INTO keys (name, value) VALUES ('CSRF_SESSION_KEY', '%s')" % base64.urlsafe_b64encode(os.urandom(32)) + db.engine.execute(sql) + + sql = "INSERT INTO keys (name, value) VALUES ('SECRET_KEY', '%s')" % base64.urlsafe_b64encode(os.urandom(32)) + db.engine.execute(sql) + + sql = "INSERT INTO keys (name, value) VALUES ('SECURITY_PASSWORD_SALT', '%s')" % base64.urlsafe_b64encode(os.urandom(32)) + db.engine.execute(sql) + # Finally, update the schema version version.value = config.SETTINGS_SCHEMA_VERSION db.session.merge(version) @@ -364,15 +403,6 @@ if __name__ == '__main__': 'config_local.py' ) - if not os.path.isfile(local_config): - print(""" - The configuration file - {0} does not exist. - Before running this application, ensure that config_local.py has been created - and sets values for SECRET_KEY, SECURITY_PASSWORD_SALT and CSRF_SESSION_KEY - at bare minimum. See config.py for more information and a complete list of - settings. Exiting...""".format(local_config)) - sys.exit(1) - # Check if the database exists. If it does, tell the user and exit. if os.path.isfile(config.SQLITE_PATH): print(""" ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-13 14:16 Ashesh Vashi <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Ashesh Vashi @ 2016-10-13 14:16 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]> Hi Dave, On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> wrote: > Hi Ashesh, > > Can you please review the attached patch, and apply if you're happy with > it? > Overall the patch looked good to me. But - I encounter an issue in 'web' mode, which wont happen with 'runtime'. Steps for reproduction on existing pgAdmin 4 environment with 'web' mode. - Apply the patch - Start the pgAdmin4 application (stand alone application). - Open pgAdmin home page. - Log out (if already login). And, you will see an exception. I have figure out the issue with the patch. We were setting the SECURITY_PASSWORD_SALT, after initializing the Security object. Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT properly. I had moved the Security object initialization after fetching these configurations from the database. I have attached a addon patch for the same. Now - I run into another issue. Because - the existing password was hashed using the old SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. I think - we need to think about different strategy for upgrading the configuration file in the 'web' mode. I was thinking - we can store the existing security configurations in the database during upgrade process in 'web' mode. I was not able to spend much time on it due to some other priorities. -- Thanks & Regards, Ashesh Vashi > The purpose is to auto-generate the various security keys that are > currently in the configuration file, and store them in the SQLite database. > This allows us to remove the checks for config_local.py and the hard-coded > default keys which are causing some problems with packaging: > > - Hard coded defaults are fine for Desktop mode, and packages generally > aim to make that work primarily. > - Hard coded defaults are a security risk for Server mode, hence we > currently require the user to manually setup keys, which is currently being > overridden by packagers for Desktop mode. > > This change ensures that we have unique security keys for every > installation, whether running in desktop or server mode (generated from > os.urandom). > > Thanks! > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/octet-stream] add_auto_generate_security_keys.patch (3.7K, 3-add_auto_generate_security_keys.patch) download | inline diff: diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py index 21fe636..b9d1cb4 100644 --- a/web/pgadmin/__init__.py +++ b/web/pgadmin/__init__.py @@ -201,7 +201,7 @@ def create_app(app_name=config.APP_NAME): # Setup Flask-Security user_datastore = SQLAlchemyUserDatastore(db, User, Role) - security = Security(app, user_datastore) + security = Security(None, user_datastore) # Upgrade the schema (if required) with app.app_context(): @@ -225,6 +225,14 @@ def create_app(app_name=config.APP_NAME): config.SECRET_KEY = Keys.query.filter_by(name = 'SECRET_KEY').first().value config.SECURITY_PASSWORD_SALT = Keys.query.filter_by(name = 'SECURITY_PASSWORD_SALT').first().value + # Update the app.config with proper security keyes for signing CSRF data, + # signing cookies, and the SALT for hashing the passwords. + app.config.update(dict(CSRF_SESSION_KEY=config.CSRF_SESSION_KEY)) + app.config.update(dict(SECRET_KEY=config.SECRET_KEY)) + app.config.update(dict(SECURITY_PASSWORD_SALT=config.SECURITY_PASSWORD_SALT)) + + security.init_app(app) + app.session_interface = create_session_interface(app) ########################################################################## diff --git a/web/setup.py b/web/setup.py index 90697fa..eaa3235 100755 --- a/web/setup.py +++ b/web/setup.py @@ -42,15 +42,6 @@ if hasattr(__builtins__, 'raw_input'): def do_setup(app): """Create a new settings database from scratch""" - # Get some defaults for the various keys - with app.app_context(): - config.CSRF_SESSION_KEY = base64.urlsafe_b64encode(os.urandom(32)) - app.jinja_env.globals['config']['CSRF_SESSION_KEY'] = config.CSRF_SESSION_KEY - config.SECRET_KEY = base64.urlsafe_b64encode(os.urandom(32)) - app.jinja_env.globals['config']['SECRET_KEY'] = config.SECRET_KEY - config.SECURITY_PASSWORD_SALT = base64.urlsafe_b64encode(os.urandom(32)) - app.jinja_env.globals['config']['SECURITY_PASSWORD_SALT'] = config.SECURITY_PASSWORD_SALT - if config.SERVER_MODE is False: print("NOTE: Configuring authentication for DESKTOP mode.") email = config.DESKTOP_USER @@ -150,7 +141,7 @@ def do_setup(app): ) -def do_upgrade(app, datastore, security, version): +def do_upgrade(app, datastore, version): """Upgrade an existing settings database""" ####################################################################### # Run whatever is required to update the database schema to the current @@ -386,6 +377,12 @@ CREATE TABLE keys ( ############################################################################### if __name__ == '__main__': app = Flask(__name__) + + # Get some defaults for the various keys + config.CSRF_SESSION_KEY = base64.urlsafe_b64encode(os.urandom(32)) + config.SECRET_KEY = base64.urlsafe_b64encode(os.urandom(32)) + config.SECURITY_PASSWORD_SALT = base64.urlsafe_b64encode(os.urandom(32)) + app.config.from_object(config) if config.TESTING_MODE: @@ -411,7 +408,6 @@ Entering upgrade mode...""" % config.SQLITE_PATH) # Setup Flask-Security user_datastore = SQLAlchemyUserDatastore(db, User, Role) - security = Security(app, user_datastore) # Always use "< REQUIRED_VERSION" as the test for readability with app.app_context(): @@ -433,7 +429,7 @@ Exiting...""" % (version.value)) print("NOTE: Upgrading database schema from version %d to %d." % ( version.value, config.SETTINGS_SCHEMA_VERSION )) - do_upgrade(app, user_datastore, security, version) + do_upgrade(app, user_datastore, version) else: directory = os.path.dirname(config.SQLITE_PATH) if not os.path.exists(directory): ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-14 17:27 Dave Page <[email protected]> parent: Ashesh Vashi <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Dave Page @ 2016-10-14 17:27 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]> Hi On Thursday, October 13, 2016, Ashesh Vashi <[email protected]> wrote: > Hi Dave, > > On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> Hi Ashesh, >> >> Can you please review the attached patch, and apply if you're happy with >> it? >> > Overall the patch looked good to me. > But - I encounter an issue in 'web' mode, which wont happen with 'runtime'. > > Steps for reproduction on existing pgAdmin 4 environment with 'web' mode. > - Apply the patch > - Start the pgAdmin4 application (stand alone application). > - Open pgAdmin home page. > - Log out (if already login). > > And, you will see an exception. > > I have figure out the issue with the patch. > We were setting the SECURITY_PASSWORD_SALT, after initializing the > Security object. > Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT > properly. > Hmm. > > I had moved the Security object initialization after fetching these > configurations from the database. > I have attached a addon patch for the same. > OK, thanks. > > Now - I run into another issue. > Because - the existing password was hashed using the old > SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. > > I think - we need to think about different strategy for upgrading the > configuration file in the 'web' mode. > I was thinking - we can store the existing security configurations in the > database during upgrade process in 'web' mode. > My concern with that is that we'll likely be storing the default config values in many cases, thus for those users, perpetuating the problem. I guess what we need to do is re-encrypt the password during the upgrade - however, that makes me think; we then have both the key and the encrypted passwords in the same database which is clearly not a good idea. Sigh... Needs more thought. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-15 02:32 Dave Page <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Dave Page @ 2016-10-15 02:32 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]> Hi On Friday, October 14, 2016, Dave Page <[email protected]> wrote: > Hi > > On Thursday, October 13, 2016, Ashesh Vashi <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> Hi Dave, >> >> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> wrote: >> >>> Hi Ashesh, >>> >>> Can you please review the attached patch, and apply if you're happy with >>> it? >>> >> Overall the patch looked good to me. >> But - I encounter an issue in 'web' mode, which wont happen with >> 'runtime'. >> >> Steps for reproduction on existing pgAdmin 4 environment with 'web' mode. >> - Apply the patch >> - Start the pgAdmin4 application (stand alone application). >> - Open pgAdmin home page. >> - Log out (if already login). >> >> And, you will see an exception. >> >> I have figure out the issue with the patch. >> We were setting the SECURITY_PASSWORD_SALT, after initializing the >> Security object. >> Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT >> properly. >> > > Hmm. > > >> >> I had moved the Security object initialization after fetching these >> configurations from the database. >> I have attached a addon patch for the same. >> > > OK, thanks. > > >> >> Now - I run into another issue. >> Because - the existing password was hashed using the old >> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >> >> I think - we need to think about different strategy for upgrading the >> configuration file in the 'web' mode. >> I was thinking - we can store the existing security configurations in the >> database during upgrade process in 'web' mode. >> > > My concern with that is that we'll likely be storing the default config > values in many cases, thus for those users, perpetuating the problem. > > I guess what we need to do is re-encrypt the password during the upgrade - > however, that makes me think; we then have both the key and the encrypted > passwords in the same database which is clearly not a good idea. Sigh... > Needs more thought. > OK, so I've been thinking about this and experimenting for a couple of hours, as well as annoying the crap out of Magnus by thinking out loud in his general direction, and it looks like this isn't a major problem as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key not a salt) not the only salting that's done. It looks like it's used system-wide as the key to generate an HMAC of the users password, which is then passed to passlib which salts and hashes it. I did some testing, and found that two users with the same password end up with different hashes in the database, so clearly there is also per-user salting happening. I also created two users, then dropped the database and created the same user accounts with the same passwords again, and found that the resulting hashes were different in both databases - thus there is something else ensuring the hashes are unique across different installations/databases. So, I believe we can do as you suggest and migrate existing values for SECURITY_PASSWORD_SALT, given that there's clearly some other per user and per installation/database salting going on anyway. New installations can have the random value for SECURITY_PASSWORD_SALT. I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, as they're used for purposes that are essentially ephemeral, and thus can be changed during an upgrade. Adding Magnus as I'd appreciate any thoughts he may have. Patch attached - please review (Ashesh, but others too would be appreciated)! Thanks. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [text/x-diff] auto_generate_security_keys_v2.patch (10.9K, 3-auto_generate_security_keys_v2.patch) download | inline diff: diff --git a/web/config.py b/web/config.py index 20714f9..8411d79 100644 --- a/web/config.py +++ b/web/config.py @@ -140,21 +140,13 @@ DEFAULT_SERVER_PORT = 5050 # Enable CSRF protection? CSRF_ENABLED = True -# Secret key for signing CSRF data. Override this in config_local.py if -# running on a web server -CSRF_SESSION_KEY = 'SuperSecret1' - -# Secret key for signing cookies. Override this in config_local.py if -# running on a web server -SECRET_KEY = 'SuperSecret2' - -# Salt used when hashing passwords. Override this in config_local.py if -# running on a web server -SECURITY_PASSWORD_SALT = 'SuperSecret3' - # Hashing algorithm used for password storage SECURITY_PASSWORD_HASH = 'pbkdf2_sha512' +# NOTE: CSRF_SESSION_KEY, SECRET_KEY and SECURITY_PASSWORD_SALT are no +# longer part of the main configuration, but are stored in the +# configuration databases 'keys' table and are auto-generated. + # Should HTML be minified on the fly when not in debug mode? # Note: This is disabled by default as it will error when processing the # docs. If the serving of docs is handled by an Apache HTTPD diff --git a/web/pgAdmin4.py b/web/pgAdmin4.py index 1fb34f9..f894f8b 100644 --- a/web/pgAdmin4.py +++ b/web/pgAdmin4.py @@ -32,18 +32,6 @@ config.SETTINGS_SCHEMA_VERSION = SCHEMA_VERSION # Sanity checks ########################################################################## -# Check for local settings if running in server mode -if config.SERVER_MODE is True: - local_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), - 'config_local.py') - if not os.path.isfile(local_config): - print("The configuration file %s does not exist.\n" % local_config) - print("Before running this application, ensure that config_local.py has been created") - print("and sets values for SECRET_KEY, SECURITY_PASSWORD_SALT and CSRF_SESSION_KEY") - print("at bare minimum. See config.py for more information and a complete list of") - print("settings. Exiting...") - sys.exit(1) - # Check if the database exists. If it does not, create it. if not os.path.isfile(config.SQLITE_PATH): setupfile = os.path.join(os.path.dirname(os.path.realpath(__file__)), diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py index d988172..79fa1c6 100644 --- a/web/pgadmin/__init__.py +++ b/web/pgadmin/__init__.py @@ -26,7 +26,7 @@ from pgadmin.utils.session import create_session_interface from werkzeug.local import LocalProxy from werkzeug.utils import find_modules -from pgadmin.model import db, Role, Server, ServerGroup, User, Version +from pgadmin.model import db, Role, Server, ServerGroup, User, Version, Keys # Configuration settings import config @@ -127,11 +127,6 @@ def create_app(app_name=config.APP_NAME): app.config.update(dict(PROPAGATE_EXCEPTIONS=True)) ########################################################################## - # Setup session management - ########################################################################## - app.session_interface = create_session_interface(app) - - ########################################################################## # Setup logging and log the application startup ########################################################################## @@ -206,7 +201,7 @@ def create_app(app_name=config.APP_NAME): # Setup Flask-Security user_datastore = SQLAlchemyUserDatastore(db, User, Role) - security = Security(app, user_datastore) + security = Security(None, user_datastore) # Upgrade the schema (if required) with app.app_context(): @@ -220,9 +215,29 @@ def create_app(app_name=config.APP_NAME): ) ) from setup import do_upgrade - do_upgrade(app, user_datastore, security, version) + do_upgrade(app, user_datastore, version) + + ########################################################################## + # Setup security + ########################################################################## + with app.app_context(): + config.CSRF_SESSION_KEY = Keys.query.filter_by(name = 'CSRF_SESSION_KEY').first().value + config.SECRET_KEY = Keys.query.filter_by(name = 'SECRET_KEY').first().value + config.SECURITY_PASSWORD_SALT = Keys.query.filter_by(name = 'SECURITY_PASSWORD_SALT').first().value + + # Update the app.config with proper security keyes for signing CSRF data, + # signing cookies, and the SALT for hashing the passwords. + app.config.update(dict(CSRF_SESSION_KEY=config.CSRF_SESSION_KEY)) + app.config.update(dict(SECRET_KEY=config.SECRET_KEY)) + app.config.update(dict(SECURITY_PASSWORD_SALT=config.SECURITY_PASSWORD_SALT)) + security.init_app(app) + + app.session_interface = create_session_interface(app) + + ########################################################################## # Load all available server drivers + ########################################################################## driver.init_app(app) ########################################################################## diff --git a/web/pgadmin/model/__init__.py b/web/pgadmin/model/__init__.py index 019e9b1..9727d2b 100644 --- a/web/pgadmin/model/__init__.py +++ b/web/pgadmin/model/__init__.py @@ -29,7 +29,7 @@ from flask_sqlalchemy import SQLAlchemy # ########################################################################## -SCHEMA_VERSION = 13 +SCHEMA_VERSION = 14 ########################################################################## # @@ -207,3 +207,10 @@ class Process(db.Model): end_time = db.Column(db.String(), nullable=True) exit_code = db.Column(db.Integer(), nullable=True) acknowledge = db.Column(db.String(), nullable=True) + + +class Keys(db.Model): + """Define the keys table.""" + __tablename__ = 'keys' + name = db.Column(db.String(), nullable=False, primary_key=True) + value = db.Column(db.String(), nullable=False) \ No newline at end of file diff --git a/web/setup.py b/web/setup.py index 64273fb..441a0f3 100755 --- a/web/setup.py +++ b/web/setup.py @@ -10,6 +10,7 @@ """Perform the initial setup of the application, by creating the auth and settings database.""" +import base64 import getpass import os import random @@ -22,7 +23,7 @@ from flask_security import Security, SQLAlchemyUserDatastore from flask_security.utils import encrypt_password from pgadmin.model import db, Role, User, Server, \ - ServerGroup, Version + ServerGroup, Version, Keys # Configuration settings import config @@ -40,6 +41,7 @@ if hasattr(__builtins__, 'raw_input'): def do_setup(app): """Create a new settings database from scratch""" + if config.SERVER_MODE is False: print("NOTE: Configuring authentication for DESKTOP mode.") email = config.DESKTOP_USER @@ -116,6 +118,17 @@ def do_setup(app): name='ConfigDB', value=config.SETTINGS_SCHEMA_VERSION ) db.session.merge(version) + db.session.commit() + + # Create the keys + key = Keys(name='CSRF_SESSION_KEY', value=config.CSRF_SESSION_KEY) + db.session.merge(key) + + key = Keys(name='SECRET_KEY', value=config.SECRET_KEY) + db.session.merge(key) + + key = Keys(name='SECURITY_PASSWORD_SALT', value=config.SECURITY_PASSWORD_SALT) + db.session.merge(key) db.session.commit() @@ -128,7 +141,7 @@ def do_setup(app): ) -def do_upgrade(app, datastore, security, version): +def do_upgrade(app, datastore, version): """Upgrade an existing settings database""" ####################################################################### # Run whatever is required to update the database schema to the current @@ -329,6 +342,29 @@ ALTER TABLE SERVER ADD COLUMN discovery_id TEXT """) + if int(version.value) < 14: + db.engine.execute(""" +CREATE TABLE keys ( + name TEST NOT NULL, + value TEXT NOT NULL, + PRIMARY KEY (name)) + """) + + sql = "INSERT INTO keys (name, value) VALUES ('CSRF_SESSION_KEY', '%s')" % base64.urlsafe_b64encode(os.urandom(32)) + db.engine.execute(sql) + + sql = "INSERT INTO keys (name, value) VALUES ('SECRET_KEY', '%s')" % base64.urlsafe_b64encode(os.urandom(32)) + db.engine.execute(sql) + + # If SECURITY_PASSWORD_SALT is not in the config, but we're upgrading, then it must (unless the + # user edited the main config - which they shouldn't have done) have been at it's default + # value, so we'll use that. Otherwise, use whatever we can find in the config. + if hasattr(config, 'SECURITY_PASSWORD_SALT'): + sql = "INSERT INTO keys (name, value) VALUES ('SECURITY_PASSWORD_SALT', '%s')" % config.SECURITY_PASSWORD_SALT + else: + sql = "INSERT INTO keys (name, value) VALUES ('SECURITY_PASSWORD_SALT', 'SuperSecret3')" + db.engine.execute(sql) + # Finally, update the schema version version.value = config.SETTINGS_SCHEMA_VERSION db.session.merge(version) @@ -347,6 +383,12 @@ ALTER TABLE SERVER ############################################################################### if __name__ == '__main__': app = Flask(__name__) + + # Get some defaults for the various keys + config.CSRF_SESSION_KEY = base64.urlsafe_b64encode(os.urandom(32)) + config.SECRET_KEY = base64.urlsafe_b64encode(os.urandom(32)) + config.SECURITY_PASSWORD_SALT = base64.urlsafe_b64encode(os.urandom(32)) + app.config.from_object(config) if config.TESTING_MODE: @@ -364,15 +406,6 @@ if __name__ == '__main__': 'config_local.py' ) - if not os.path.isfile(local_config): - print(""" - The configuration file - {0} does not exist. - Before running this application, ensure that config_local.py has been created - and sets values for SECRET_KEY, SECURITY_PASSWORD_SALT and CSRF_SESSION_KEY - at bare minimum. See config.py for more information and a complete list of - settings. Exiting...""".format(local_config)) - sys.exit(1) - # Check if the database exists. If it does, tell the user and exit. if os.path.isfile(config.SQLITE_PATH): print(""" @@ -381,7 +414,6 @@ Entering upgrade mode...""" % config.SQLITE_PATH) # Setup Flask-Security user_datastore = SQLAlchemyUserDatastore(db, User, Role) - security = Security(app, user_datastore) # Always use "< REQUIRED_VERSION" as the test for readability with app.app_context(): @@ -403,7 +435,7 @@ Exiting...""" % (version.value)) print("NOTE: Upgrading database schema from version %d to %d." % ( version.value, config.SETTINGS_SCHEMA_VERSION )) - do_upgrade(app, user_datastore, security, version) + do_upgrade(app, user_datastore, version) else: directory = os.path.dirname(config.SQLITE_PATH) if not os.path.exists(directory): ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 05:46 Ashesh Vashi <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Ashesh Vashi @ 2016-10-19 05:46 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]> Hi Dave, On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: > Hi > > > On Friday, October 14, 2016, Dave Page <[email protected]> wrote: > >> Hi >> >> On Thursday, October 13, 2016, Ashesh Vashi < >> [email protected]> wrote: >> >>> Hi Dave, >>> >>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> wrote: >>> >>>> Hi Ashesh, >>>> >>>> Can you please review the attached patch, and apply if you're happy >>>> with it? >>>> >>> Overall the patch looked good to me. >>> But - I encounter an issue in 'web' mode, which wont happen with >>> 'runtime'. >>> >>> Steps for reproduction on existing pgAdmin 4 environment with 'web' mode. >>> - Apply the patch >>> - Start the pgAdmin4 application (stand alone application). >>> - Open pgAdmin home page. >>> - Log out (if already login). >>> >>> And, you will see an exception. >>> >>> I have figure out the issue with the patch. >>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>> Security object. >>> Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT >>> properly. >>> >> >> Hmm. >> >> >>> >>> I had moved the Security object initialization after fetching these >>> configurations from the database. >>> I have attached a addon patch for the same. >>> >> >> OK, thanks. >> >> >>> >>> Now - I run into another issue. >>> Because - the existing password was hashed using the old >>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>> >>> I think - we need to think about different strategy for upgrading the >>> configuration file in the 'web' mode. >>> I was thinking - we can store the existing security configurations in >>> the database during upgrade process in 'web' mode. >>> >> >> My concern with that is that we'll likely be storing the default config >> values in many cases, thus for those users, perpetuating the problem. >> >> I guess what we need to do is re-encrypt the password during the upgrade >> - however, that makes me think; we then have both the key and the encrypted >> passwords in the same database which is clearly not a good idea. Sigh... >> Needs more thought. >> > > OK, so I've been thinking about this and experimenting for a couple of > hours, as well as annoying the crap out of Magnus by thinking out loud in > his general direction, and it looks like this isn't a major problem as from > what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key > not a salt) not the only salting that's done. > > It looks like it's used system-wide as the key to generate an HMAC of the > users password, which is then passed to passlib which salts and hashes it. > I did some testing, and found that two users with the same password end up > with different hashes in the database, so clearly there is also per-user > salting happening. I also created two users, then dropped the database and > created the same user accounts with the same passwords again, and found > that the resulting hashes were different in both databases - thus there is > something else ensuring the hashes are unique across different > installations/databases. > > So, I believe we can do as you suggest and migrate existing values for > SECURITY_PASSWORD_SALT, given that there's clearly some other per user and > per installation/database salting going on anyway. New installations can > have the random value for SECURITY_PASSWORD_SALT. > We do not need to generate the random SECURITY_PASSWORD_SALT during upgrade mode, which was wrong added in my addon patch. Please find the updated patch. Otherwise - looks good to me. Please commit the new patch (if you're ok with the change). -- Thanks & Regards, Ashesh Vashi EnterpriseDB INDIA: Enterprise PostgreSQL Company <http://www.enterprisedb.com/; *http://www.linkedin.com/in/asheshvashi* <http://www.linkedin.com/in/asheshvashi; > > I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, as > they're used for purposes that are essentially ephemeral, and thus can be > changed during an upgrade. > > Adding Magnus as I'd appreciate any thoughts he may have. > > Patch attached - please review (Ashesh, but others too would be > appreciated)! > > Thanks. > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/octet-stream] auto_generate_security_keys_v3.patch (11.0K, 3-auto_generate_security_keys_v3.patch) download | inline diff: diff --git a/web/config.py b/web/config.py index 20714f9..8411d79 100644 --- a/web/config.py +++ b/web/config.py @@ -140,21 +140,13 @@ DEFAULT_SERVER_PORT = 5050 # Enable CSRF protection? CSRF_ENABLED = True -# Secret key for signing CSRF data. Override this in config_local.py if -# running on a web server -CSRF_SESSION_KEY = 'SuperSecret1' - -# Secret key for signing cookies. Override this in config_local.py if -# running on a web server -SECRET_KEY = 'SuperSecret2' - -# Salt used when hashing passwords. Override this in config_local.py if -# running on a web server -SECURITY_PASSWORD_SALT = 'SuperSecret3' - # Hashing algorithm used for password storage SECURITY_PASSWORD_HASH = 'pbkdf2_sha512' +# NOTE: CSRF_SESSION_KEY, SECRET_KEY and SECURITY_PASSWORD_SALT are no +# longer part of the main configuration, but are stored in the +# configuration databases 'keys' table and are auto-generated. + # Should HTML be minified on the fly when not in debug mode? # Note: This is disabled by default as it will error when processing the # docs. If the serving of docs is handled by an Apache HTTPD diff --git a/web/pgAdmin4.py b/web/pgAdmin4.py index 1fb34f9..f894f8b 100644 --- a/web/pgAdmin4.py +++ b/web/pgAdmin4.py @@ -32,18 +32,6 @@ config.SETTINGS_SCHEMA_VERSION = SCHEMA_VERSION # Sanity checks ########################################################################## -# Check for local settings if running in server mode -if config.SERVER_MODE is True: - local_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), - 'config_local.py') - if not os.path.isfile(local_config): - print("The configuration file %s does not exist.\n" % local_config) - print("Before running this application, ensure that config_local.py has been created") - print("and sets values for SECRET_KEY, SECURITY_PASSWORD_SALT and CSRF_SESSION_KEY") - print("at bare minimum. See config.py for more information and a complete list of") - print("settings. Exiting...") - sys.exit(1) - # Check if the database exists. If it does not, create it. if not os.path.isfile(config.SQLITE_PATH): setupfile = os.path.join(os.path.dirname(os.path.realpath(__file__)), diff --git a/web/pgadmin/__init__.py b/web/pgadmin/__init__.py index d988172..79fa1c6 100644 --- a/web/pgadmin/__init__.py +++ b/web/pgadmin/__init__.py @@ -26,7 +26,7 @@ from pgadmin.utils.session import create_session_interface from werkzeug.local import LocalProxy from werkzeug.utils import find_modules -from pgadmin.model import db, Role, Server, ServerGroup, User, Version +from pgadmin.model import db, Role, Server, ServerGroup, User, Version, Keys # Configuration settings import config @@ -127,11 +127,6 @@ def create_app(app_name=config.APP_NAME): app.config.update(dict(PROPAGATE_EXCEPTIONS=True)) ########################################################################## - # Setup session management - ########################################################################## - app.session_interface = create_session_interface(app) - - ########################################################################## # Setup logging and log the application startup ########################################################################## @@ -206,7 +201,7 @@ def create_app(app_name=config.APP_NAME): # Setup Flask-Security user_datastore = SQLAlchemyUserDatastore(db, User, Role) - security = Security(app, user_datastore) + security = Security(None, user_datastore) # Upgrade the schema (if required) with app.app_context(): @@ -220,9 +215,29 @@ def create_app(app_name=config.APP_NAME): ) ) from setup import do_upgrade - do_upgrade(app, user_datastore, security, version) + do_upgrade(app, user_datastore, version) + + ########################################################################## + # Setup security + ########################################################################## + with app.app_context(): + config.CSRF_SESSION_KEY = Keys.query.filter_by(name = 'CSRF_SESSION_KEY').first().value + config.SECRET_KEY = Keys.query.filter_by(name = 'SECRET_KEY').first().value + config.SECURITY_PASSWORD_SALT = Keys.query.filter_by(name = 'SECURITY_PASSWORD_SALT').first().value + + # Update the app.config with proper security keyes for signing CSRF data, + # signing cookies, and the SALT for hashing the passwords. + app.config.update(dict(CSRF_SESSION_KEY=config.CSRF_SESSION_KEY)) + app.config.update(dict(SECRET_KEY=config.SECRET_KEY)) + app.config.update(dict(SECURITY_PASSWORD_SALT=config.SECURITY_PASSWORD_SALT)) + security.init_app(app) + + app.session_interface = create_session_interface(app) + + ########################################################################## # Load all available server drivers + ########################################################################## driver.init_app(app) ########################################################################## diff --git a/web/pgadmin/model/__init__.py b/web/pgadmin/model/__init__.py index 019e9b1..9727d2b 100644 --- a/web/pgadmin/model/__init__.py +++ b/web/pgadmin/model/__init__.py @@ -29,7 +29,7 @@ from flask_sqlalchemy import SQLAlchemy # ########################################################################## -SCHEMA_VERSION = 13 +SCHEMA_VERSION = 14 ########################################################################## # @@ -207,3 +207,10 @@ class Process(db.Model): end_time = db.Column(db.String(), nullable=True) exit_code = db.Column(db.Integer(), nullable=True) acknowledge = db.Column(db.String(), nullable=True) + + +class Keys(db.Model): + """Define the keys table.""" + __tablename__ = 'keys' + name = db.Column(db.String(), nullable=False, primary_key=True) + value = db.Column(db.String(), nullable=False) \ No newline at end of file diff --git a/web/setup.py b/web/setup.py index 64273fb..12465f3 100755 --- a/web/setup.py +++ b/web/setup.py @@ -10,6 +10,7 @@ """Perform the initial setup of the application, by creating the auth and settings database.""" +import base64 import getpass import os import random @@ -22,7 +23,7 @@ from flask_security import Security, SQLAlchemyUserDatastore from flask_security.utils import encrypt_password from pgadmin.model import db, Role, User, Server, \ - ServerGroup, Version + ServerGroup, Version, Keys # Configuration settings import config @@ -40,6 +41,7 @@ if hasattr(__builtins__, 'raw_input'): def do_setup(app): """Create a new settings database from scratch""" + if config.SERVER_MODE is False: print("NOTE: Configuring authentication for DESKTOP mode.") email = config.DESKTOP_USER @@ -116,6 +118,17 @@ def do_setup(app): name='ConfigDB', value=config.SETTINGS_SCHEMA_VERSION ) db.session.merge(version) + db.session.commit() + + # Create the keys + key = Keys(name='CSRF_SESSION_KEY', value=config.CSRF_SESSION_KEY) + db.session.merge(key) + + key = Keys(name='SECRET_KEY', value=config.SECRET_KEY) + db.session.merge(key) + + key = Keys(name='SECURITY_PASSWORD_SALT', value=config.SECURITY_PASSWORD_SALT) + db.session.merge(key) db.session.commit() @@ -128,7 +141,7 @@ def do_setup(app): ) -def do_upgrade(app, datastore, security, version): +def do_upgrade(app, datastore, version): """Upgrade an existing settings database""" ####################################################################### # Run whatever is required to update the database schema to the current @@ -329,6 +342,29 @@ ALTER TABLE SERVER ADD COLUMN discovery_id TEXT """) + if int(version.value) < 14: + db.engine.execute(""" +CREATE TABLE keys ( + name TEST NOT NULL, + value TEXT NOT NULL, + PRIMARY KEY (name)) + """) + + sql = "INSERT INTO keys (name, value) VALUES ('CSRF_SESSION_KEY', '%s')" % base64.urlsafe_b64encode(os.urandom(32)) + db.engine.execute(sql) + + sql = "INSERT INTO keys (name, value) VALUES ('SECRET_KEY', '%s')" % base64.urlsafe_b64encode(os.urandom(32)) + db.engine.execute(sql) + + # If SECURITY_PASSWORD_SALT is not in the config, but we're upgrading, then it must (unless the + # user edited the main config - which they shouldn't have done) have been at it's default + # value, so we'll use that. Otherwise, use whatever we can find in the config. + if hasattr(config, 'SECURITY_PASSWORD_SALT'): + sql = "INSERT INTO keys (name, value) VALUES ('SECURITY_PASSWORD_SALT', '%s')" % config.SECURITY_PASSWORD_SALT + else: + sql = "INSERT INTO keys (name, value) VALUES ('SECURITY_PASSWORD_SALT', 'SuperSecret3')" + db.engine.execute(sql) + # Finally, update the schema version version.value = config.SETTINGS_SCHEMA_VERSION db.session.merge(version) @@ -347,6 +383,7 @@ ALTER TABLE SERVER ############################################################################### if __name__ == '__main__': app = Flask(__name__) + app.config.from_object(config) if config.TESTING_MODE: @@ -364,15 +401,6 @@ if __name__ == '__main__': 'config_local.py' ) - if not os.path.isfile(local_config): - print(""" - The configuration file - {0} does not exist. - Before running this application, ensure that config_local.py has been created - and sets values for SECRET_KEY, SECURITY_PASSWORD_SALT and CSRF_SESSION_KEY - at bare minimum. See config.py for more information and a complete list of - settings. Exiting...""".format(local_config)) - sys.exit(1) - # Check if the database exists. If it does, tell the user and exit. if os.path.isfile(config.SQLITE_PATH): print(""" @@ -381,7 +409,6 @@ Entering upgrade mode...""" % config.SQLITE_PATH) # Setup Flask-Security user_datastore = SQLAlchemyUserDatastore(db, User, Role) - security = Security(app, user_datastore) # Always use "< REQUIRED_VERSION" as the test for readability with app.app_context(): @@ -403,8 +430,13 @@ Exiting...""" % (version.value)) print("NOTE: Upgrading database schema from version %d to %d." % ( version.value, config.SETTINGS_SCHEMA_VERSION )) - do_upgrade(app, user_datastore, security, version) + do_upgrade(app, user_datastore, version) else: + # Get some defaults for the various keys + config.CSRF_SESSION_KEY = base64.urlsafe_b64encode(os.urandom(32)) + config.SECRET_KEY = base64.urlsafe_b64encode(os.urandom(32)) + config.SECURITY_PASSWORD_SALT = base64.urlsafe_b64encode(os.urandom(32)) + directory = os.path.dirname(config.SQLITE_PATH) if not os.path.exists(directory): os.makedirs(directory, int('700', 8)) ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 08:27 Dave Page <[email protected]> parent: Ashesh Vashi <[email protected]> 0 siblings, 2 replies; 33+ messages in thread From: Dave Page @ 2016-10-19 08:27 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Syed Fahar Abbas <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Patch applied. Fahar, can you please test this thoroughly in desktop and server modes, with both fresh and upgraded installations? https://redmine.postgresql.org/issues/1849 Packagers: This change means that packages are no longer forced to create a config_local.py file, and there is no longer any need to explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY and CSRF_SESSION_KEY in the config (in fact, they should be removed for new installations, if you have included them in 1.0) Thanks. On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi <[email protected] > wrote: > Hi Dave, > > On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: > >> Hi >> >> >> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >> >>> Hi >>> >>> On Thursday, October 13, 2016, Ashesh Vashi < >>> [email protected]> wrote: >>> >>>> Hi Dave, >>>> >>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> wrote: >>>> >>>>> Hi Ashesh, >>>>> >>>>> Can you please review the attached patch, and apply if you're happy >>>>> with it? >>>>> >>>> Overall the patch looked good to me. >>>> But - I encounter an issue in 'web' mode, which wont happen with >>>> 'runtime'. >>>> >>>> Steps for reproduction on existing pgAdmin 4 environment with 'web' >>>> mode. >>>> - Apply the patch >>>> - Start the pgAdmin4 application (stand alone application). >>>> - Open pgAdmin home page. >>>> - Log out (if already login). >>>> >>>> And, you will see an exception. >>>> >>>> I have figure out the issue with the patch. >>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>> Security object. >>>> Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT >>>> properly. >>>> >>> >>> Hmm. >>> >>> >>>> >>>> I had moved the Security object initialization after fetching these >>>> configurations from the database. >>>> I have attached a addon patch for the same. >>>> >>> >>> OK, thanks. >>> >>> >>>> >>>> Now - I run into another issue. >>>> Because - the existing password was hashed using the old >>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>> >>>> I think - we need to think about different strategy for upgrading the >>>> configuration file in the 'web' mode. >>>> I was thinking - we can store the existing security configurations in >>>> the database during upgrade process in 'web' mode. >>>> >>> >>> My concern with that is that we'll likely be storing the default config >>> values in many cases, thus for those users, perpetuating the problem. >>> >>> I guess what we need to do is re-encrypt the password during the upgrade >>> - however, that makes me think; we then have both the key and the encrypted >>> passwords in the same database which is clearly not a good idea. Sigh... >>> Needs more thought. >>> >> >> OK, so I've been thinking about this and experimenting for a couple of >> hours, as well as annoying the crap out of Magnus by thinking out loud in >> his general direction, and it looks like this isn't a major problem as from >> what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key >> not a salt) not the only salting that's done. >> >> It looks like it's used system-wide as the key to generate an HMAC of the >> users password, which is then passed to passlib which salts and hashes it. >> I did some testing, and found that two users with the same password end up >> with different hashes in the database, so clearly there is also per-user >> salting happening. I also created two users, then dropped the database and >> created the same user accounts with the same passwords again, and found >> that the resulting hashes were different in both databases - thus there is >> something else ensuring the hashes are unique across different >> installations/databases. >> >> So, I believe we can do as you suggest and migrate existing values for >> SECURITY_PASSWORD_SALT, given that there's clearly some other per user and >> per installation/database salting going on anyway. New installations can >> have the random value for SECURITY_PASSWORD_SALT. >> > We do not need to generate the random SECURITY_PASSWORD_SALT during > upgrade mode, which was wrong added in my addon patch. > > Please find the updated patch. > > Otherwise - looks good to me. > Please commit the new patch (if you're ok with the change). > > > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > <http://www.enterprisedb.com/; > > > *http://www.linkedin.com/in/asheshvashi* > <http://www.linkedin.com/in/asheshvashi; > >> >> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, as >> they're used for purposes that are essentially ephemeral, and thus can be >> changed during an upgrade. >> >> Adding Magnus as I'd appreciate any thoughts he may have. >> >> Patch attached - please review (Ashesh, but others too would be >> appreciated)! >> >> Thanks. >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 08:37 Fahar Abbas <[email protected]> parent: Dave Page <[email protected]> 1 sibling, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-19 08:37 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Sure, Will test this thoroughly after complete investigation. Kind Regards, On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: > Patch applied. > > Fahar, can you please test this thoroughly in desktop and server modes, > with both fresh and upgraded installations? > > https://redmine.postgresql.org/issues/1849 > > Packagers: This change means that packages are no longer forced to create > a config_local.py file, and there is no longer any need to explicitly set > SECURITY_PASSWORD_SALT, SECURITY_KEY and CSRF_SESSION_KEY in the config > (in fact, they should be removed for new installations, if you have > included them in 1.0) > > Thanks. > > > On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < > [email protected]> wrote: > >> Hi Dave, >> >> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >> >>> Hi >>> >>> >>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>> >>>> Hi >>>> >>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>> [email protected]> wrote: >>>> >>>>> Hi Dave, >>>>> >>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> wrote: >>>>> >>>>>> Hi Ashesh, >>>>>> >>>>>> Can you please review the attached patch, and apply if you're happy >>>>>> with it? >>>>>> >>>>> Overall the patch looked good to me. >>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>> 'runtime'. >>>>> >>>>> Steps for reproduction on existing pgAdmin 4 environment with 'web' >>>>> mode. >>>>> - Apply the patch >>>>> - Start the pgAdmin4 application (stand alone application). >>>>> - Open pgAdmin home page. >>>>> - Log out (if already login). >>>>> >>>>> And, you will see an exception. >>>>> >>>>> I have figure out the issue with the patch. >>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>>> Security object. >>>>> Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT >>>>> properly. >>>>> >>>> >>>> Hmm. >>>> >>>> >>>>> >>>>> I had moved the Security object initialization after fetching these >>>>> configurations from the database. >>>>> I have attached a addon patch for the same. >>>>> >>>> >>>> OK, thanks. >>>> >>>> >>>>> >>>>> Now - I run into another issue. >>>>> Because - the existing password was hashed using the old >>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>> >>>>> I think - we need to think about different strategy for upgrading the >>>>> configuration file in the 'web' mode. >>>>> I was thinking - we can store the existing security configurations in >>>>> the database during upgrade process in 'web' mode. >>>>> >>>> >>>> My concern with that is that we'll likely be storing the default config >>>> values in many cases, thus for those users, perpetuating the problem. >>>> >>>> I guess what we need to do is re-encrypt the password during the >>>> upgrade - however, that makes me think; we then have both the key and the >>>> encrypted passwords in the same database which is clearly not a good idea. >>>> Sigh... Needs more thought. >>>> >>> >>> OK, so I've been thinking about this and experimenting for a couple of >>> hours, as well as annoying the crap out of Magnus by thinking out loud in >>> his general direction, and it looks like this isn't a major problem as from >>> what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key >>> not a salt) not the only salting that's done. >>> >>> It looks like it's used system-wide as the key to generate an HMAC of >>> the users password, which is then passed to passlib which salts and hashes >>> it. I did some testing, and found that two users with the same password end >>> up with different hashes in the database, so clearly there is also per-user >>> salting happening. I also created two users, then dropped the database and >>> created the same user accounts with the same passwords again, and found >>> that the resulting hashes were different in both databases - thus there is >>> something else ensuring the hashes are unique across different >>> installations/databases. >>> >>> So, I believe we can do as you suggest and migrate existing values for >>> SECURITY_PASSWORD_SALT, given that there's clearly some other per user and >>> per installation/database salting going on anyway. New installations can >>> have the random value for SECURITY_PASSWORD_SALT. >>> >> We do not need to generate the random SECURITY_PASSWORD_SALT during >> upgrade mode, which was wrong added in my addon patch. >> >> Please find the updated patch. >> >> Otherwise - looks good to me. >> Please commit the new patch (if you're ok with the change). >> >> >> -- >> >> Thanks & Regards, >> >> Ashesh Vashi >> EnterpriseDB INDIA: Enterprise PostgreSQL Company >> <http://www.enterprisedb.com/; >> >> >> *http://www.linkedin.com/in/asheshvashi* >> <http://www.linkedin.com/in/asheshvashi; >> >>> >>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, as >>> they're used for purposes that are essentially ephemeral, and thus can be >>> changed during an upgrade. >>> >>> Adding Magnus as I'd appreciate any thoughts he may have. >>> >>> Patch attached - please review (Ashesh, but others too would be >>> appreciated)! >>> >>> Thanks. >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> >> > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 10:03 Fahar Abbas <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-19 10:03 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Dave, Testing Environment Ubuntu 16.04 Linux 64: -------------------------------- pg-AdminIV Development Environment Setup for Ubuntu : 1) Install GIT sudo apt-get install git 2) Install pip3 sudo apt-get install python3-pip 3) Install virtualenv sudo pip3 install virtualenv 4) install below dependency as it is required for psycopg2 & pycrypto module sudo apt-get install libpq-dev sudo apt-get install python3-dev 5) Create virtual environment virtualenv -p python3 venv 6) Create mkdir Projects 7) Clone git repo in Projects git clone http://git.postgresql.org/git/pgadmin4.git 8) activate virtual environment source venv/bin/activate 9) Install modules pip3 install -r requirements_py3.txt *10) Edit the config.py file to config_local.py resides in Projects\pgAdmin4\web * 11)Now run setup.py file (\Projects\pgAdmin4\web) python setup.py If user does not create config_local.py and do Python setup.py for new Development then SECURITY_PASSWORD_SALT message is also displayed: Here is the output: ------------------------- python setup.py pgAdmin 4 - Application Initialisation ====================================== The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist. Entering initial setup mode... NOTE: Configuring authentication for SERVER mode. Enter the email address and password to use for the initial pgAdmin user account: Email address: [email protected] Password: Retype password: Traceback (most recent call last): File "setup.py", line 449, in <module> do_setup(app) File "setup.py", line 96, in do_setup password = encrypt_password(p1) File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 150, in encrypt_password signed = get_hmac(password).decode('ascii') File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac 'set to "%s"' % _security.password_hash) RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ Is this expected? On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas <[email protected]> wrote: > Sure, > > Will test this thoroughly after complete investigation. > > Kind Regards, > > On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: > >> Patch applied. >> >> Fahar, can you please test this thoroughly in desktop and server modes, >> with both fresh and upgraded installations? >> >> https://redmine.postgresql.org/issues/1849 >> >> Packagers: This change means that packages are no longer forced to create >> a config_local.py file, and there is no longer any need to explicitly set >> SECURITY_PASSWORD_SALT, SECURITY_KEY and CSRF_SESSION_KEY in the config >> (in fact, they should be removed for new installations, if you have >> included them in 1.0) >> >> Thanks. >> >> >> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >> [email protected]> wrote: >> >>> Hi Dave, >>> >>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >>> >>>> Hi >>>> >>>> >>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>> >>>>> Hi >>>>> >>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Dave, >>>>>> >>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> wrote: >>>>>> >>>>>>> Hi Ashesh, >>>>>>> >>>>>>> Can you please review the attached patch, and apply if you're happy >>>>>>> with it? >>>>>>> >>>>>> Overall the patch looked good to me. >>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>> 'runtime'. >>>>>> >>>>>> Steps for reproduction on existing pgAdmin 4 environment with 'web' >>>>>> mode. >>>>>> - Apply the patch >>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>> - Open pgAdmin home page. >>>>>> - Log out (if already login). >>>>>> >>>>>> And, you will see an exception. >>>>>> >>>>>> I have figure out the issue with the patch. >>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>>>> Security object. >>>>>> Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT >>>>>> properly. >>>>>> >>>>> >>>>> Hmm. >>>>> >>>>> >>>>>> >>>>>> I had moved the Security object initialization after fetching these >>>>>> configurations from the database. >>>>>> I have attached a addon patch for the same. >>>>>> >>>>> >>>>> OK, thanks. >>>>> >>>>> >>>>>> >>>>>> Now - I run into another issue. >>>>>> Because - the existing password was hashed using the old >>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>> >>>>>> I think - we need to think about different strategy for upgrading the >>>>>> configuration file in the 'web' mode. >>>>>> I was thinking - we can store the existing security configurations in >>>>>> the database during upgrade process in 'web' mode. >>>>>> >>>>> >>>>> My concern with that is that we'll likely be storing the default >>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>> >>>>> I guess what we need to do is re-encrypt the password during the >>>>> upgrade - however, that makes me think; we then have both the key and the >>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>> Sigh... Needs more thought. >>>>> >>>> >>>> OK, so I've been thinking about this and experimenting for a couple of >>>> hours, as well as annoying the crap out of Magnus by thinking out loud in >>>> his general direction, and it looks like this isn't a major problem as from >>>> what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key >>>> not a salt) not the only salting that's done. >>>> >>>> It looks like it's used system-wide as the key to generate an HMAC of >>>> the users password, which is then passed to passlib which salts and hashes >>>> it. I did some testing, and found that two users with the same password end >>>> up with different hashes in the database, so clearly there is also per-user >>>> salting happening. I also created two users, then dropped the database and >>>> created the same user accounts with the same passwords again, and found >>>> that the resulting hashes were different in both databases - thus there is >>>> something else ensuring the hashes are unique across different >>>> installations/databases. >>>> >>>> So, I believe we can do as you suggest and migrate existing values for >>>> SECURITY_PASSWORD_SALT, given that there's clearly some other per user and >>>> per installation/database salting going on anyway. New installations can >>>> have the random value for SECURITY_PASSWORD_SALT. >>>> >>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>> upgrade mode, which was wrong added in my addon patch. >>> >>> Please find the updated patch. >>> >>> Otherwise - looks good to me. >>> Please commit the new patch (if you're ok with the change). >>> >>> >>> -- >>> >>> Thanks & Regards, >>> >>> Ashesh Vashi >>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>> <http://www.enterprisedb.com/; >>> >>> >>> *http://www.linkedin.com/in/asheshvashi* >>> <http://www.linkedin.com/in/asheshvashi; >>> >>>> >>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, as >>>> they're used for purposes that are essentially ephemeral, and thus can be >>>> changed during an upgrade. >>>> >>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>> >>>> Patch attached - please review (Ashesh, but others too would be >>>> appreciated)! >>>> >>>> Thanks. >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>>> >>> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 10:17 Fahar Abbas <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-19 10:17 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Here is the output of if we copy config_local.py and execute python setup.py pgAdmin 4 - Application Initialisation ====================================== The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist. Entering initial setup mode... NOTE: Configuring authentication for SERVER mode. Enter the email address and password to use for the initial pgAdmin user account: Email address: [email protected] Password: Retype password: Traceback (most recent call last): File "setup.py", line 449, in <module> do_setup(app) File "setup.py", line 96, in do_setup password = encrypt_password(p1) File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 150, in encrypt_password signed = get_hmac(password).decode('ascii') File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac 'set to "%s"' % _security.password_hash) RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" python setup.py pgAdmin 4 - Application Initialisation ====================================== User can not do any setup for web based now. The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist. Entering initial setup mode... NOTE: Configuring authentication for SERVER mode. Enter the email address and password to use for the initial pgAdmin user account: Email address: [email protected] Password: Retype password: Traceback (most recent call last): File "setup.py", line 449, in <module> do_setup(app) File "setup.py", line 96, in do_setup password = encrypt_password(p1) File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 150, in encrypt_password signed = get_hmac(password).decode('ascii') File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac 'set to "%s"' % _security.password_hash) RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas <[email protected]> wrote: > Dave, > > Testing Environment > > Ubuntu 16.04 Linux 64: > -------------------------------- > > pg-AdminIV Development Environment Setup for Ubuntu : > > > 1) Install GIT > > sudo apt-get install git > > 2) Install pip3 > > sudo apt-get install python3-pip > > 3) Install virtualenv > > sudo pip3 install virtualenv > > 4) install below dependency as it is required for psycopg2 & pycrypto > module > > sudo apt-get install libpq-dev > > sudo apt-get install python3-dev > > 5) Create virtual environment > > virtualenv -p python3 venv > > 6) Create mkdir Projects > > 7) Clone git repo in Projects > > git clone http://git.postgresql.org/git/pgadmin4.git > > 8) activate virtual environment > > source venv/bin/activate > > 9) Install modules > > pip3 install -r requirements_py3.txt > > *10) Edit the config.py file to config_local.py resides in > Projects\pgAdmin4\web * > > 11)Now run setup.py file (\Projects\pgAdmin4\web) > python setup.py > > If user does not create config_local.py and do Python setup.py for new > Development then SECURITY_PASSWORD_SALT message is also displayed: > > Here is the output: > ------------------------- > > python setup.py > pgAdmin 4 - Application Initialisation > ====================================== > > > The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not > exist. > Entering initial setup mode... > NOTE: Configuring authentication for SERVER mode. > > > Enter the email address and password to use for the initial pgAdmin > user account: > > Email address: [email protected] > Password: > Retype password: > Traceback (most recent call last): > File "setup.py", line 449, in <module> > do_setup(app) > File "setup.py", line 96, in do_setup > password = encrypt_password(p1) > File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", > line 150, in encrypt_password > signed = get_hmac(password).decode('ascii') > File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", > line 108, in get_hmac > 'set to "%s"' % _security.password_hash) > RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be > None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" > (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ > > > Is this expected? > > On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas <[email protected] > > wrote: > >> Sure, >> >> Will test this thoroughly after complete investigation. >> >> Kind Regards, >> >> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >> >>> Patch applied. >>> >>> Fahar, can you please test this thoroughly in desktop and server modes, >>> with both fresh and upgraded installations? >>> >>> https://redmine.postgresql.org/issues/1849 >>> >>> Packagers: This change means that packages are no longer forced to >>> create a config_local.py file, and there is no longer any need to >>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>> installations, if you have included them in 1.0) >>> >>> Thanks. >>> >>> >>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>> [email protected]> wrote: >>> >>>> Hi Dave, >>>> >>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >>>> >>>>> Hi >>>>> >>>>> >>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Dave, >>>>>>> >>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Ashesh, >>>>>>>> >>>>>>>> Can you please review the attached patch, and apply if you're happy >>>>>>>> with it? >>>>>>>> >>>>>>> Overall the patch looked good to me. >>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>> 'runtime'. >>>>>>> >>>>>>> Steps for reproduction on existing pgAdmin 4 environment with 'web' >>>>>>> mode. >>>>>>> - Apply the patch >>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>> - Open pgAdmin home page. >>>>>>> - Log out (if already login). >>>>>>> >>>>>>> And, you will see an exception. >>>>>>> >>>>>>> I have figure out the issue with the patch. >>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>>>>> Security object. >>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>> >>>>>> >>>>>> Hmm. >>>>>> >>>>>> >>>>>>> >>>>>>> I had moved the Security object initialization after fetching these >>>>>>> configurations from the database. >>>>>>> I have attached a addon patch for the same. >>>>>>> >>>>>> >>>>>> OK, thanks. >>>>>> >>>>>> >>>>>>> >>>>>>> Now - I run into another issue. >>>>>>> Because - the existing password was hashed using the old >>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>> >>>>>>> I think - we need to think about different strategy for upgrading >>>>>>> the configuration file in the 'web' mode. >>>>>>> I was thinking - we can store the existing security configurations >>>>>>> in the database during upgrade process in 'web' mode. >>>>>>> >>>>>> >>>>>> My concern with that is that we'll likely be storing the default >>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>> >>>>>> I guess what we need to do is re-encrypt the password during the >>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>> Sigh... Needs more thought. >>>>>> >>>>> >>>>> OK, so I've been thinking about this and experimenting for a couple of >>>>> hours, as well as annoying the crap out of Magnus by thinking out loud in >>>>> his general direction, and it looks like this isn't a major problem as from >>>>> what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key >>>>> not a salt) not the only salting that's done. >>>>> >>>>> It looks like it's used system-wide as the key to generate an HMAC of >>>>> the users password, which is then passed to passlib which salts and hashes >>>>> it. I did some testing, and found that two users with the same password end >>>>> up with different hashes in the database, so clearly there is also per-user >>>>> salting happening. I also created two users, then dropped the database and >>>>> created the same user accounts with the same passwords again, and found >>>>> that the resulting hashes were different in both databases - thus there is >>>>> something else ensuring the hashes are unique across different >>>>> installations/databases. >>>>> >>>>> So, I believe we can do as you suggest and migrate existing values for >>>>> SECURITY_PASSWORD_SALT, given that there's clearly some other per user and >>>>> per installation/database salting going on anyway. New installations can >>>>> have the random value for SECURITY_PASSWORD_SALT. >>>>> >>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>> upgrade mode, which was wrong added in my addon patch. >>>> >>>> Please find the updated patch. >>>> >>>> Otherwise - looks good to me. >>>> Please commit the new patch (if you're ok with the change). >>>> >>>> >>>> -- >>>> >>>> Thanks & Regards, >>>> >>>> Ashesh Vashi >>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>> <http://www.enterprisedb.com/; >>>> >>>> >>>> *http://www.linkedin.com/in/asheshvashi* >>>> <http://www.linkedin.com/in/asheshvashi; >>>> >>>>> >>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, >>>>> as they're used for purposes that are essentially ephemeral, and thus can >>>>> be changed during an upgrade. >>>>> >>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>> >>>>> Patch attached - please review (Ashesh, but others too would be >>>>> appreciated)! >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>>> >>>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 10:55 Ashesh Vashi <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 2 replies; 33+ messages in thread From: Ashesh Vashi @ 2016-10-19 10:55 UTC (permalink / raw) To: Fahar Abbas <[email protected]>; +Cc: Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Hi Fahar, Please log the case on redmine. Please find the attached patch, please apply it locally, and test it. And, please update the case, and this mail chain accordingly. -- Thanks & Regards, Ashesh Vashi EnterpriseDB INDIA: Enterprise PostgreSQL Company <http://www.enterprisedb.com; *http://www.linkedin.com/in/asheshvashi* <http://www.linkedin.com/in/asheshvashi; On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas <[email protected]> wrote: > Here is the output of if we copy config_local.py and execute python > setup.py > pgAdmin 4 - Application Initialisation > ====================================== > > > The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not > exist. > Entering initial setup mode... > NOTE: Configuring authentication for SERVER mode. > > > Enter the email address and password to use for the initial pgAdmin > user account: > > Email address: [email protected] > Password: > Retype password: > Traceback (most recent call last): > File "setup.py", line 449, in <module> > do_setup(app) > File "setup.py", line 96, in do_setup > password = encrypt_password(p1) > File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", > line 150, in encrypt_password > signed = get_hmac(password).decode('ascii') > File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", > line 108, in get_hmac > 'set to "%s"' % _security.password_hash) > RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be > None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" > python setup.py > pgAdmin 4 - Application Initialisation > ====================================== > > User can not do any setup for web based now. > > > The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not > exist. > Entering initial setup mode... > NOTE: Configuring authentication for SERVER mode. > > > Enter the email address and password to use for the initial pgAdmin > user account: > > Email address: [email protected] > Password: > Retype password: > Traceback (most recent call last): > File "setup.py", line 449, in <module> > do_setup(app) > File "setup.py", line 96, in do_setup > password = encrypt_password(p1) > File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", > line 150, in encrypt_password > signed = get_hmac(password).decode('ascii') > File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", > line 108, in get_hmac > 'set to "%s"' % _security.password_hash) > RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be > None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" > > On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas <[email protected] > > wrote: > >> Dave, >> >> Testing Environment >> >> Ubuntu 16.04 Linux 64: >> -------------------------------- >> >> pg-AdminIV Development Environment Setup for Ubuntu : >> >> >> 1) Install GIT >> >> sudo apt-get install git >> >> 2) Install pip3 >> >> sudo apt-get install python3-pip >> >> 3) Install virtualenv >> >> sudo pip3 install virtualenv >> >> 4) install below dependency as it is required for psycopg2 & pycrypto >> module >> >> sudo apt-get install libpq-dev >> >> sudo apt-get install python3-dev >> >> 5) Create virtual environment >> >> virtualenv -p python3 venv >> >> 6) Create mkdir Projects >> >> 7) Clone git repo in Projects >> >> git clone http://git.postgresql.org/git/pgadmin4.git >> >> 8) activate virtual environment >> >> source venv/bin/activate >> >> 9) Install modules >> >> pip3 install -r requirements_py3.txt >> >> *10) Edit the config.py file to config_local.py resides in >> Projects\pgAdmin4\web * >> >> 11)Now run setup.py file (\Projects\pgAdmin4\web) >> python setup.py >> >> If user does not create config_local.py and do Python setup.py for new >> Development then SECURITY_PASSWORD_SALT message is also displayed: >> >> Here is the output: >> ------------------------- >> >> python setup.py >> pgAdmin 4 - Application Initialisation >> ====================================== >> >> >> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not >> exist. >> Entering initial setup mode... >> NOTE: Configuring authentication for SERVER mode. >> >> >> Enter the email address and password to use for the initial pgAdmin >> user account: >> >> Email address: [email protected] >> Password: >> Retype password: >> Traceback (most recent call last): >> File "setup.py", line 449, in <module> >> do_setup(app) >> File "setup.py", line 96, in do_setup >> password = encrypt_password(p1) >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 150, in encrypt_password >> signed = get_hmac(password).decode('ascii') >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 108, in get_hmac >> 'set to "%s"' % _security.password_hash) >> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >> >> >> Is this expected? >> >> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >> [email protected]> wrote: >> >>> Sure, >>> >>> Will test this thoroughly after complete investigation. >>> >>> Kind Regards, >>> >>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >>> >>>> Patch applied. >>>> >>>> Fahar, can you please test this thoroughly in desktop and server modes, >>>> with both fresh and upgraded installations? >>>> >>>> https://redmine.postgresql.org/issues/1849 >>>> >>>> Packagers: This change means that packages are no longer forced to >>>> create a config_local.py file, and there is no longer any need to >>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>> installations, if you have included them in 1.0) >>>> >>>> Thanks. >>>> >>>> >>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>> [email protected]> wrote: >>>> >>>>> Hi Dave, >>>>> >>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> >>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Dave, >>>>>>>> >>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Ashesh, >>>>>>>>> >>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>> happy with it? >>>>>>>>> >>>>>>>> Overall the patch looked good to me. >>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>>> 'runtime'. >>>>>>>> >>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with 'web' >>>>>>>> mode. >>>>>>>> - Apply the patch >>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>> - Open pgAdmin home page. >>>>>>>> - Log out (if already login). >>>>>>>> >>>>>>>> And, you will see an exception. >>>>>>>> >>>>>>>> I have figure out the issue with the patch. >>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>>>>>> Security object. >>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>> >>>>>>> >>>>>>> Hmm. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> I had moved the Security object initialization after fetching these >>>>>>>> configurations from the database. >>>>>>>> I have attached a addon patch for the same. >>>>>>>> >>>>>>> >>>>>>> OK, thanks. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Now - I run into another issue. >>>>>>>> Because - the existing password was hashed using the old >>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>> >>>>>>>> I think - we need to think about different strategy for upgrading >>>>>>>> the configuration file in the 'web' mode. >>>>>>>> I was thinking - we can store the existing security configurations >>>>>>>> in the database during upgrade process in 'web' mode. >>>>>>>> >>>>>>> >>>>>>> My concern with that is that we'll likely be storing the default >>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>> >>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>> Sigh... Needs more thought. >>>>>>> >>>>>> >>>>>> OK, so I've been thinking about this and experimenting for a couple >>>>>> of hours, as well as annoying the crap out of Magnus by thinking out loud >>>>>> in his general direction, and it looks like this isn't a major problem as >>>>>> from what I can see, SECURITY_PASSWORD_SALT is (aside from really being a >>>>>> key not a salt) not the only salting that's done. >>>>>> >>>>>> It looks like it's used system-wide as the key to generate an HMAC of >>>>>> the users password, which is then passed to passlib which salts and hashes >>>>>> it. I did some testing, and found that two users with the same password end >>>>>> up with different hashes in the database, so clearly there is also per-user >>>>>> salting happening. I also created two users, then dropped the database and >>>>>> created the same user accounts with the same passwords again, and found >>>>>> that the resulting hashes were different in both databases - thus there is >>>>>> something else ensuring the hashes are unique across different >>>>>> installations/databases. >>>>>> >>>>>> So, I believe we can do as you suggest and migrate existing values >>>>>> for SECURITY_PASSWORD_SALT, given that there's clearly some other per user >>>>>> and per installation/database salting going on anyway. New installations >>>>>> can have the random value for SECURITY_PASSWORD_SALT. >>>>>> >>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>>> upgrade mode, which was wrong added in my addon patch. >>>>> >>>>> Please find the updated patch. >>>>> >>>>> Otherwise - looks good to me. >>>>> Please commit the new patch (if you're ok with the change). >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & Regards, >>>>> >>>>> Ashesh Vashi >>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>> <http://www.enterprisedb.com/; >>>>> >>>>> >>>>> *http://www.linkedin.com/in/asheshvashi* >>>>> <http://www.linkedin.com/in/asheshvashi; >>>>> >>>>>> >>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, >>>>>> as they're used for purposes that are essentially ephemeral, and thus can >>>>>> be changed during an upgrade. >>>>>> >>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>> >>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>> appreciated)! >>>>>> >>>>>> Thanks. >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/octet-stream] reload_config_setup.patch (509B, 3-reload_config_setup.patch) download | inline diff: diff --git a/web/setup.py b/web/setup.py index 12465f3..c3d3b36 100755 --- a/web/setup.py +++ b/web/setup.py @@ -437,6 +437,8 @@ Exiting...""" % (version.value)) config.SECRET_KEY = base64.urlsafe_b64encode(os.urandom(32)) config.SECURITY_PASSWORD_SALT = base64.urlsafe_b64encode(os.urandom(32)) + app.config.from_object(config) + directory = os.path.dirname(config.SQLITE_PATH) if not os.path.exists(directory): os.makedirs(directory, int('700', 8)) ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 11:03 Fahar Abbas <[email protected]> parent: Ashesh Vashi <[email protected]> 1 sibling, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-19 11:03 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> On Wed, Oct 19, 2016 at 3:55 PM, Ashesh Vashi <[email protected] > wrote: > Hi Fahar, > > Please log the case on redmine. > https://redmine.postgresql.org/issues/1871 > Please find the attached patch, please apply it locally, and test it. > > And, please update the case, and this mail chain accordingly. > > Sure Will test the patch and update the status accordingly. > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > <http://www.enterprisedb.com; > > > *http://www.linkedin.com/in/asheshvashi* > <http://www.linkedin.com/in/asheshvashi; > > On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas <[email protected] > > wrote: > >> Here is the output of if we copy config_local.py and execute python >> setup.py >> pgAdmin 4 - Application Initialisation >> ====================================== >> >> >> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not >> exist. >> Entering initial setup mode... >> NOTE: Configuring authentication for SERVER mode. >> >> >> Enter the email address and password to use for the initial pgAdmin >> user account: >> >> Email address: [email protected] >> Password: >> Retype password: >> Traceback (most recent call last): >> File "setup.py", line 449, in <module> >> do_setup(app) >> File "setup.py", line 96, in do_setup >> password = encrypt_password(p1) >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 150, in encrypt_password >> signed = get_hmac(password).decode('ascii') >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 108, in get_hmac >> 'set to "%s"' % _security.password_hash) >> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >> python setup.py >> pgAdmin 4 - Application Initialisation >> ====================================== >> >> User can not do any setup for web based now. >> >> >> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not >> exist. >> Entering initial setup mode... >> NOTE: Configuring authentication for SERVER mode. >> >> >> Enter the email address and password to use for the initial pgAdmin >> user account: >> >> Email address: [email protected] >> Password: >> Retype password: >> Traceback (most recent call last): >> File "setup.py", line 449, in <module> >> do_setup(app) >> File "setup.py", line 96, in do_setup >> password = encrypt_password(p1) >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 150, in encrypt_password >> signed = get_hmac(password).decode('ascii') >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 108, in get_hmac >> 'set to "%s"' % _security.password_hash) >> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >> >> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >> [email protected]> wrote: >> >>> Dave, >>> >>> Testing Environment >>> >>> Ubuntu 16.04 Linux 64: >>> -------------------------------- >>> >>> pg-AdminIV Development Environment Setup for Ubuntu : >>> >>> >>> 1) Install GIT >>> >>> sudo apt-get install git >>> >>> 2) Install pip3 >>> >>> sudo apt-get install python3-pip >>> >>> 3) Install virtualenv >>> >>> sudo pip3 install virtualenv >>> >>> 4) install below dependency as it is required for psycopg2 & pycrypto >>> module >>> >>> sudo apt-get install libpq-dev >>> >>> sudo apt-get install python3-dev >>> >>> 5) Create virtual environment >>> >>> virtualenv -p python3 venv >>> >>> 6) Create mkdir Projects >>> >>> 7) Clone git repo in Projects >>> >>> git clone http://git.postgresql.org/git/pgadmin4.git >>> >>> 8) activate virtual environment >>> >>> source venv/bin/activate >>> >>> 9) Install modules >>> >>> pip3 install -r requirements_py3.txt >>> >>> *10) Edit the config.py file to config_local.py resides in >>> Projects\pgAdmin4\web * >>> >>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>> python setup.py >>> >>> If user does not create config_local.py and do Python setup.py for new >>> Development then SECURITY_PASSWORD_SALT message is also displayed: >>> >>> Here is the output: >>> ------------------------- >>> >>> python setup.py >>> pgAdmin 4 - Application Initialisation >>> ====================================== >>> >>> >>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>> not exist. >>> Entering initial setup mode... >>> NOTE: Configuring authentication for SERVER mode. >>> >>> >>> Enter the email address and password to use for the initial pgAdmin >>> user account: >>> >>> Email address: [email protected] >>> Password: >>> Retype password: >>> Traceback (most recent call last): >>> File "setup.py", line 449, in <module> >>> do_setup(app) >>> File "setup.py", line 96, in do_setup >>> password = encrypt_password(p1) >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 150, in encrypt_password >>> signed = get_hmac(password).decode('ascii') >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 108, in get_hmac >>> 'set to "%s"' % _security.password_hash) >>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>> >>> >>> Is this expected? >>> >>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Sure, >>>> >>>> Will test this thoroughly after complete investigation. >>>> >>>> Kind Regards, >>>> >>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >>>> >>>>> Patch applied. >>>>> >>>>> Fahar, can you please test this thoroughly in desktop and server >>>>> modes, with both fresh and upgraded installations? >>>>> >>>>> https://redmine.postgresql.org/issues/1849 >>>>> >>>>> Packagers: This change means that packages are no longer forced to >>>>> create a config_local.py file, and there is no longer any need to >>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>> installations, if you have included them in 1.0) >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Dave, >>>>>> >>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> >>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Dave, >>>>>>>>> >>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Ashesh, >>>>>>>>>> >>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>> happy with it? >>>>>>>>>> >>>>>>>>> Overall the patch looked good to me. >>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>>>> 'runtime'. >>>>>>>>> >>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>> 'web' mode. >>>>>>>>> - Apply the patch >>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>> - Open pgAdmin home page. >>>>>>>>> - Log out (if already login). >>>>>>>>> >>>>>>>>> And, you will see an exception. >>>>>>>>> >>>>>>>>> I have figure out the issue with the patch. >>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>>>>>>> Security object. >>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>> >>>>>>>> >>>>>>>> Hmm. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>> these configurations from the database. >>>>>>>>> I have attached a addon patch for the same. >>>>>>>>> >>>>>>>> >>>>>>>> OK, thanks. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Now - I run into another issue. >>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>> >>>>>>>>> I think - we need to think about different strategy for upgrading >>>>>>>>> the configuration file in the 'web' mode. >>>>>>>>> I was thinking - we can store the existing security configurations >>>>>>>>> in the database during upgrade process in 'web' mode. >>>>>>>>> >>>>>>>> >>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>> >>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>> Sigh... Needs more thought. >>>>>>>> >>>>>>> >>>>>>> OK, so I've been thinking about this and experimenting for a couple >>>>>>> of hours, as well as annoying the crap out of Magnus by thinking out loud >>>>>>> in his general direction, and it looks like this isn't a major problem as >>>>>>> from what I can see, SECURITY_PASSWORD_SALT is (aside from really being a >>>>>>> key not a salt) not the only salting that's done. >>>>>>> >>>>>>> It looks like it's used system-wide as the key to generate an HMAC >>>>>>> of the users password, which is then passed to passlib which salts and >>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>> database and created the same user accounts with the same passwords again, >>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>> there is something else ensuring the hashes are unique across different >>>>>>> installations/databases. >>>>>>> >>>>>>> So, I believe we can do as you suggest and migrate existing values >>>>>>> for SECURITY_PASSWORD_SALT, given that there's clearly some other per user >>>>>>> and per installation/database salting going on anyway. New installations >>>>>>> can have the random value for SECURITY_PASSWORD_SALT. >>>>>>> >>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>>>> upgrade mode, which was wrong added in my addon patch. >>>>>> >>>>>> Please find the updated patch. >>>>>> >>>>>> Otherwise - looks good to me. >>>>>> Please commit the new patch (if you're ok with the change). >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & Regards, >>>>>> >>>>>> Ashesh Vashi >>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>> <http://www.enterprisedb.com/; >>>>>> >>>>>> >>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>> >>>>>>> >>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, >>>>>>> as they're used for purposes that are essentially ephemeral, and thus can >>>>>>> be changed during an upgrade. >>>>>>> >>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>> >>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>> appreciated)! >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Dave Page >>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>> Twitter: @pgsnake >>>>>>> >>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> The Enterprise PostgreSQL Company >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 11:05 Sandeep Thakkar <[email protected]> parent: Dave Page <[email protected]> 1 sibling, 1 reply; 33+ messages in thread From: Sandeep Thakkar @ 2016-10-19 11:05 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Syed Fahar Abbas <[email protected]>; Hamid Quddus Akhtar <[email protected]> Hi Dave, On Wed, Oct 19, 2016 at 1:57 PM, Dave Page <[email protected]> wrote: > Patch applied. > > Fahar, can you please test this thoroughly in desktop and server modes, > with both fresh and upgraded installations? > > https://redmine.postgresql.org/issues/1849 > > Packagers: This change means that packages are no longer forced to create > a config_local.py file, and there is no longer any need to explicitly set > SECURITY_PASSWORD_SALT, SECURITY_KEY and CSRF_SESSION_KEY in the config > (in fact, they should be removed for new installations, if you have > included them in 1.0) > > OK. Will remove config_local.py from the packaging. We do not set the mentioned directives in the config. > Thanks. > > On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < > [email protected]> wrote: > >> Hi Dave, >> >> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >> >>> Hi >>> >>> >>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>> >>>> Hi >>>> >>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>> [email protected]> wrote: >>>> >>>>> Hi Dave, >>>>> >>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> wrote: >>>>> >>>>>> Hi Ashesh, >>>>>> >>>>>> Can you please review the attached patch, and apply if you're happy >>>>>> with it? >>>>>> >>>>> Overall the patch looked good to me. >>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>> 'runtime'. >>>>> >>>>> Steps for reproduction on existing pgAdmin 4 environment with 'web' >>>>> mode. >>>>> - Apply the patch >>>>> - Start the pgAdmin4 application (stand alone application). >>>>> - Open pgAdmin home page. >>>>> - Log out (if already login). >>>>> >>>>> And, you will see an exception. >>>>> >>>>> I have figure out the issue with the patch. >>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>>> Security object. >>>>> Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT >>>>> properly. >>>>> >>>> >>>> Hmm. >>>> >>>> >>>>> >>>>> I had moved the Security object initialization after fetching these >>>>> configurations from the database. >>>>> I have attached a addon patch for the same. >>>>> >>>> >>>> OK, thanks. >>>> >>>> >>>>> >>>>> Now - I run into another issue. >>>>> Because - the existing password was hashed using the old >>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>> >>>>> I think - we need to think about different strategy for upgrading the >>>>> configuration file in the 'web' mode. >>>>> I was thinking - we can store the existing security configurations in >>>>> the database during upgrade process in 'web' mode. >>>>> >>>> >>>> My concern with that is that we'll likely be storing the default config >>>> values in many cases, thus for those users, perpetuating the problem. >>>> >>>> I guess what we need to do is re-encrypt the password during the >>>> upgrade - however, that makes me think; we then have both the key and the >>>> encrypted passwords in the same database which is clearly not a good idea. >>>> Sigh... Needs more thought. >>>> >>> >>> OK, so I've been thinking about this and experimenting for a couple of >>> hours, as well as annoying the crap out of Magnus by thinking out loud in >>> his general direction, and it looks like this isn't a major problem as from >>> what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key >>> not a salt) not the only salting that's done. >>> >>> It looks like it's used system-wide as the key to generate an HMAC of >>> the users password, which is then passed to passlib which salts and hashes >>> it. I did some testing, and found that two users with the same password end >>> up with different hashes in the database, so clearly there is also per-user >>> salting happening. I also created two users, then dropped the database and >>> created the same user accounts with the same passwords again, and found >>> that the resulting hashes were different in both databases - thus there is >>> something else ensuring the hashes are unique across different >>> installations/databases. >>> >>> So, I believe we can do as you suggest and migrate existing values for >>> SECURITY_PASSWORD_SALT, given that there's clearly some other per user and >>> per installation/database salting going on anyway. New installations can >>> have the random value for SECURITY_PASSWORD_SALT. >>> >> We do not need to generate the random SECURITY_PASSWORD_SALT during >> upgrade mode, which was wrong added in my addon patch. >> >> Please find the updated patch. >> >> Otherwise - looks good to me. >> Please commit the new patch (if you're ok with the change). >> >> >> -- >> >> Thanks & Regards, >> >> Ashesh Vashi >> EnterpriseDB INDIA: Enterprise PostgreSQL Company >> <http://www.enterprisedb.com/; >> >> >> *http://www.linkedin.com/in/asheshvashi* >> <http://www.linkedin.com/in/asheshvashi; >> >>> >>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, as >>> they're used for purposes that are essentially ephemeral, and thus can be >>> changed during an upgrade. >>> >>> Adding Magnus as I'd appreciate any thoughts he may have. >>> >>> Patch attached - please review (Ashesh, but others too would be >>> appreciated)! >>> >>> Thanks. >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> >> > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Sandeep Thakkar ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 11:42 Fahar Abbas <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 2 replies; 33+ messages in thread From: Fahar Abbas @ 2016-10-19 11:42 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> On Wed, Oct 19, 2016 at 4:03 PM, Fahar Abbas <[email protected]> wrote: > > > On Wed, Oct 19, 2016 at 3:55 PM, Ashesh Vashi < > [email protected]> wrote: > >> Hi Fahar, >> >> Please log the case on redmine. >> > https://redmine.postgresql.org/issues/1871 > >> Please find the attached patch, please apply it locally, and test it. >> >> And, please update the case, and this mail chain accordingly. >> > This is resolved now and no error message displayed when we apply the patch that is already shared. > >> Sure Will test the patch and update the status accordingly. > >> -- >> >> Thanks & Regards, >> >> Ashesh Vashi >> EnterpriseDB INDIA: Enterprise PostgreSQL Company >> <http://www.enterprisedb.com; >> >> >> *http://www.linkedin.com/in/asheshvashi* >> <http://www.linkedin.com/in/asheshvashi; >> >> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >> [email protected]> wrote: >> >>> Here is the output of if we copy config_local.py and execute python >>> setup.py >>> pgAdmin 4 - Application Initialisation >>> ====================================== >>> >>> >>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>> not exist. >>> Entering initial setup mode... >>> NOTE: Configuring authentication for SERVER mode. >>> >>> >>> Enter the email address and password to use for the initial pgAdmin >>> user account: >>> >>> Email address: [email protected] >>> Password: >>> Retype password: >>> Traceback (most recent call last): >>> File "setup.py", line 449, in <module> >>> do_setup(app) >>> File "setup.py", line 96, in do_setup >>> password = encrypt_password(p1) >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 150, in encrypt_password >>> signed = get_hmac(password).decode('ascii') >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 108, in get_hmac >>> 'set to "%s"' % _security.password_hash) >>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>> python setup.py >>> pgAdmin 4 - Application Initialisation >>> ====================================== >>> >>> User can not do any setup for web based now. >>> >>> >>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>> not exist. >>> Entering initial setup mode... >>> NOTE: Configuring authentication for SERVER mode. >>> >>> >>> Enter the email address and password to use for the initial pgAdmin >>> user account: >>> >>> Email address: [email protected] >>> Password: >>> Retype password: >>> Traceback (most recent call last): >>> File "setup.py", line 449, in <module> >>> do_setup(app) >>> File "setup.py", line 96, in do_setup >>> password = encrypt_password(p1) >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 150, in encrypt_password >>> signed = get_hmac(password).decode('ascii') >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 108, in get_hmac >>> 'set to "%s"' % _security.password_hash) >>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>> >>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Dave, >>>> >>>> Testing Environment >>>> >>>> Ubuntu 16.04 Linux 64: >>>> -------------------------------- >>>> >>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>> >>>> >>>> 1) Install GIT >>>> >>>> sudo apt-get install git >>>> >>>> 2) Install pip3 >>>> >>>> sudo apt-get install python3-pip >>>> >>>> 3) Install virtualenv >>>> >>>> sudo pip3 install virtualenv >>>> >>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>> module >>>> >>>> sudo apt-get install libpq-dev >>>> >>>> sudo apt-get install python3-dev >>>> >>>> 5) Create virtual environment >>>> >>>> virtualenv -p python3 venv >>>> >>>> 6) Create mkdir Projects >>>> >>>> 7) Clone git repo in Projects >>>> >>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>> >>>> 8) activate virtual environment >>>> >>>> source venv/bin/activate >>>> >>>> 9) Install modules >>>> >>>> pip3 install -r requirements_py3.txt >>>> >>>> *10) Edit the config.py file to config_local.py resides in >>>> Projects\pgAdmin4\web * >>>> >>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>> python setup.py >>>> >>>> If user does not create config_local.py and do Python setup.py for new >>>> Development then SECURITY_PASSWORD_SALT message is also displayed: >>>> >>>> Here is the output: >>>> ------------------------- >>>> >>>> python setup.py >>>> pgAdmin 4 - Application Initialisation >>>> ====================================== >>>> >>>> >>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>> not exist. >>>> Entering initial setup mode... >>>> NOTE: Configuring authentication for SERVER mode. >>>> >>>> >>>> Enter the email address and password to use for the initial pgAdmin >>>> user account: >>>> >>>> Email address: [email protected] >>>> Password: >>>> Retype password: >>>> Traceback (most recent call last): >>>> File "setup.py", line 449, in <module> >>>> do_setup(app) >>>> File "setup.py", line 96, in do_setup >>>> password = encrypt_password(p1) >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 150, in encrypt_password >>>> signed = get_hmac(password).decode('ascii') >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 108, in get_hmac >>>> 'set to "%s"' % _security.password_hash) >>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>> >>>> >>>> Is this expected? >>>> >>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Sure, >>>>> >>>>> Will test this thoroughly after complete investigation. >>>>> >>>>> Kind Regards, >>>>> >>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >>>>> >>>>>> Patch applied. >>>>>> >>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>> modes, with both fresh and upgraded installations? >>>>>> >>>>>> https://redmine.postgresql.org/issues/1849 >>>>>> >>>>>> Packagers: This change means that packages are no longer forced to >>>>>> create a config_local.py file, and there is no longer any need to >>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>> installations, if you have included them in 1.0) >>>>>> >>>>>> Thanks. >>>>>> >>>>>> >>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Dave, >>>>>>> >>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> >>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Dave, >>>>>>>>>> >>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Ashesh, >>>>>>>>>>> >>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>> happy with it? >>>>>>>>>>> >>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>>>>> 'runtime'. >>>>>>>>>> >>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>> 'web' mode. >>>>>>>>>> - Apply the patch >>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>> - Log out (if already login). >>>>>>>>>> >>>>>>>>>> And, you will see an exception. >>>>>>>>>> >>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>> the Security object. >>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hmm. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>> these configurations from the database. >>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK, thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Now - I run into another issue. >>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>> >>>>>>>>>> I think - we need to think about different strategy for upgrading >>>>>>>>>> the configuration file in the 'web' mode. >>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>> >>>>>>>>> >>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>> >>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>> Sigh... Needs more thought. >>>>>>>>> >>>>>>>> >>>>>>>> OK, so I've been thinking about this and experimenting for a couple >>>>>>>> of hours, as well as annoying the crap out of Magnus by thinking out loud >>>>>>>> in his general direction, and it looks like this isn't a major problem as >>>>>>>> from what I can see, SECURITY_PASSWORD_SALT is (aside from really being a >>>>>>>> key not a salt) not the only salting that's done. >>>>>>>> >>>>>>>> It looks like it's used system-wide as the key to generate an HMAC >>>>>>>> of the users password, which is then passed to passlib which salts and >>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>> installations/databases. >>>>>>>> >>>>>>>> So, I believe we can do as you suggest and migrate existing values >>>>>>>> for SECURITY_PASSWORD_SALT, given that there's clearly some other per user >>>>>>>> and per installation/database salting going on anyway. New installations >>>>>>>> can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>> >>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>>>>> upgrade mode, which was wrong added in my addon patch. >>>>>>> >>>>>>> Please find the updated patch. >>>>>>> >>>>>>> Otherwise - looks good to me. >>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> >>>>>>> Ashesh Vashi >>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>> <http://www.enterprisedb.com/; >>>>>>> >>>>>>> >>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>> >>>>>>>> >>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>> thus can be changed during an upgrade. >>>>>>>> >>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>> >>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>> appreciated)! >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 12:05 Dave Page <[email protected]> parent: Fahar Abbas <[email protected]> 1 sibling, 0 replies; 33+ messages in thread From: Dave Page @ 2016-10-19 12:05 UTC (permalink / raw) To: Fahar Abbas <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Great, thanks! On Wed, Oct 19, 2016 at 12:42 PM, Fahar Abbas <[email protected]> wrote: > > > On Wed, Oct 19, 2016 at 4:03 PM, Fahar Abbas <[email protected] > > wrote: > >> >> >> On Wed, Oct 19, 2016 at 3:55 PM, Ashesh Vashi < >> [email protected]> wrote: >> >>> Hi Fahar, >>> >>> Please log the case on redmine. >>> >> https://redmine.postgresql.org/issues/1871 >> >>> Please find the attached patch, please apply it locally, and test it. >>> >>> And, please update the case, and this mail chain accordingly. >>> >> This is resolved now and no error message displayed when we apply the > patch that is already shared. > >> >>> Sure Will test the patch and update the status accordingly. >> >>> -- >>> >>> Thanks & Regards, >>> >>> Ashesh Vashi >>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>> <http://www.enterprisedb.com; >>> >>> >>> *http://www.linkedin.com/in/asheshvashi* >>> <http://www.linkedin.com/in/asheshvashi; >>> >>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Here is the output of if we copy config_local.py and execute python >>>> setup.py >>>> pgAdmin 4 - Application Initialisation >>>> ====================================== >>>> >>>> >>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>> not exist. >>>> Entering initial setup mode... >>>> NOTE: Configuring authentication for SERVER mode. >>>> >>>> >>>> Enter the email address and password to use for the initial pgAdmin >>>> user account: >>>> >>>> Email address: [email protected] >>>> Password: >>>> Retype password: >>>> Traceback (most recent call last): >>>> File "setup.py", line 449, in <module> >>>> do_setup(app) >>>> File "setup.py", line 96, in do_setup >>>> password = encrypt_password(p1) >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 150, in encrypt_password >>>> signed = get_hmac(password).decode('ascii') >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 108, in get_hmac >>>> 'set to "%s"' % _security.password_hash) >>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>> python setup.py >>>> pgAdmin 4 - Application Initialisation >>>> ====================================== >>>> >>>> User can not do any setup for web based now. >>>> >>>> >>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>> not exist. >>>> Entering initial setup mode... >>>> NOTE: Configuring authentication for SERVER mode. >>>> >>>> >>>> Enter the email address and password to use for the initial pgAdmin >>>> user account: >>>> >>>> Email address: [email protected] >>>> Password: >>>> Retype password: >>>> Traceback (most recent call last): >>>> File "setup.py", line 449, in <module> >>>> do_setup(app) >>>> File "setup.py", line 96, in do_setup >>>> password = encrypt_password(p1) >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 150, in encrypt_password >>>> signed = get_hmac(password).decode('ascii') >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 108, in get_hmac >>>> 'set to "%s"' % _security.password_hash) >>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>> >>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Dave, >>>>> >>>>> Testing Environment >>>>> >>>>> Ubuntu 16.04 Linux 64: >>>>> -------------------------------- >>>>> >>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>> >>>>> >>>>> 1) Install GIT >>>>> >>>>> sudo apt-get install git >>>>> >>>>> 2) Install pip3 >>>>> >>>>> sudo apt-get install python3-pip >>>>> >>>>> 3) Install virtualenv >>>>> >>>>> sudo pip3 install virtualenv >>>>> >>>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>>> module >>>>> >>>>> sudo apt-get install libpq-dev >>>>> >>>>> sudo apt-get install python3-dev >>>>> >>>>> 5) Create virtual environment >>>>> >>>>> virtualenv -p python3 venv >>>>> >>>>> 6) Create mkdir Projects >>>>> >>>>> 7) Clone git repo in Projects >>>>> >>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>> >>>>> 8) activate virtual environment >>>>> >>>>> source venv/bin/activate >>>>> >>>>> 9) Install modules >>>>> >>>>> pip3 install -r requirements_py3.txt >>>>> >>>>> *10) Edit the config.py file to config_local.py resides in >>>>> Projects\pgAdmin4\web * >>>>> >>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>> python setup.py >>>>> >>>>> If user does not create config_local.py and do Python setup.py for new >>>>> Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>> >>>>> Here is the output: >>>>> ------------------------- >>>>> >>>>> python setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>> >>>>> >>>>> Is this expected? >>>>> >>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Sure, >>>>>> >>>>>> Will test this thoroughly after complete investigation. >>>>>> >>>>>> Kind Regards, >>>>>> >>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >>>>>> >>>>>>> Patch applied. >>>>>>> >>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>> modes, with both fresh and upgraded installations? >>>>>>> >>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>> >>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>> installations, if you have included them in 1.0) >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Dave, >>>>>>>> >>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> >>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Dave, >>>>>>>>>>> >>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>> >>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>> happy with it? >>>>>>>>>>>> >>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>>>>>> 'runtime'. >>>>>>>>>>> >>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>> 'web' mode. >>>>>>>>>>> - Apply the patch >>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>> - Log out (if already login). >>>>>>>>>>> >>>>>>>>>>> And, you will see an exception. >>>>>>>>>>> >>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>> the Security object. >>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hmm. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>> these configurations from the database. >>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>> >>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>>> >>>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>>> Sigh... Needs more thought. >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>> >>>>>>>>> It looks like it's used system-wide as the key to generate an HMAC >>>>>>>>> of the users password, which is then passed to passlib which salts and >>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>> installations/databases. >>>>>>>>> >>>>>>>>> So, I believe we can do as you suggest and migrate existing values >>>>>>>>> for SECURITY_PASSWORD_SALT, given that there's clearly some other per user >>>>>>>>> and per installation/database salting going on anyway. New installations >>>>>>>>> can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>> >>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>>>>>> upgrade mode, which was wrong added in my addon patch. >>>>>>>> >>>>>>>> Please find the updated patch. >>>>>>>> >>>>>>>> Otherwise - looks good to me. >>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks & Regards, >>>>>>>> >>>>>>>> Ashesh Vashi >>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>> <http://www.enterprisedb.com/; >>>>>>>> >>>>>>>> >>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>> >>>>>>>>> >>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>> thus can be changed during an upgrade. >>>>>>>>> >>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>> >>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>> appreciated)! >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dave Page >>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>> Twitter: @pgsnake >>>>>>>>> >>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Dave Page >>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>> Twitter: @pgsnake >>>>>>> >>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> The Enterprise PostgreSQL Company >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 12:11 Neel Patel <[email protected]> parent: Fahar Abbas <[email protected]> 1 sibling, 2 replies; 33+ messages in thread From: Neel Patel @ 2016-10-19 12:11 UTC (permalink / raw) To: Fahar Abbas <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Hi, Just to update for Python 3. It gives below error while running "pgAdmin4.py". ##### Traceback (most recent call last): File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner self.run() File "/usr/lib/python3.4/threading.py", line 868, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.4/socketserver.py", line 620, in process_request_thread self.handle_error(request, client_address) File "/usr/lib/python3.4/socketserver.py", line 617, in process_request_thread self.finish_request(request, client_address) File "/usr/lib/python3.4/socketserver.py", line 344, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python3.4/socketserver.py", line 673, in __init__ self.handle() File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/werkzeug/serving.py", line 200, in handle rv = BaseHTTPRequestHandler.handle(self) File "/usr/lib/python3.4/http/server.py", line 398, in handle self.handle_one_request() File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/werkzeug/serving.py", line 235, in handle_one_request return self.run_wsgi() File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/werkzeug/serving.py", line 177, in run_wsgi execute(self.server.app) File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/werkzeug/serving.py", line 165, in execute application_iter = app(environ, start_response) File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", line 2000, in __call__ return self.wsgi_app(environ, start_response) File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", line 1991, in wsgi_app response = self.make_response(self.handle_exception(e)) File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", line 1567, in handle_exception reraise(exc_type, exc_value, tb) File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/_compat.py", line 33, in reraise raise value File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", line 1988, in wsgi_app response = self.full_dispatch_request() File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", line 1643, in full_dispatch_request response = self.process_response(response) File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", line 1864, in process_response self.save_session(ctx.session, response) File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", line 926, in save_session return self.session_interface.save_session(self, session, response) File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", line 267, in save_session self.manager.put(session) File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", line 144, in put self.parent.put(session) File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", line 214, in put session.sign(self.secret) File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", line 71, in sign self.hmac_digest = _calc_hmac('%s:%s' % (self.sid, self.randval), secret) File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", line 44, in _calc_hmac secret.encode(), body.encode(), hashlib.sha1 AttributeError: 'bytes' object has no attribute 'encode' ####### Thanks, Neel Patel On Wed, Oct 19, 2016 at 5:12 PM, Fahar Abbas <[email protected]> wrote: > > > On Wed, Oct 19, 2016 at 4:03 PM, Fahar Abbas <[email protected] > > wrote: > >> >> >> On Wed, Oct 19, 2016 at 3:55 PM, Ashesh Vashi < >> [email protected]> wrote: >> >>> Hi Fahar, >>> >>> Please log the case on redmine. >>> >> https://redmine.postgresql.org/issues/1871 >> >>> Please find the attached patch, please apply it locally, and test it. >>> >>> And, please update the case, and this mail chain accordingly. >>> >> This is resolved now and no error message displayed when we apply the > patch that is already shared. > >> >>> Sure Will test the patch and update the status accordingly. >> >>> -- >>> >>> Thanks & Regards, >>> >>> Ashesh Vashi >>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>> <http://www.enterprisedb.com; >>> >>> >>> *http://www.linkedin.com/in/asheshvashi* >>> <http://www.linkedin.com/in/asheshvashi; >>> >>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Here is the output of if we copy config_local.py and execute python >>>> setup.py >>>> pgAdmin 4 - Application Initialisation >>>> ====================================== >>>> >>>> >>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>> not exist. >>>> Entering initial setup mode... >>>> NOTE: Configuring authentication for SERVER mode. >>>> >>>> >>>> Enter the email address and password to use for the initial pgAdmin >>>> user account: >>>> >>>> Email address: [email protected] >>>> Password: >>>> Retype password: >>>> Traceback (most recent call last): >>>> File "setup.py", line 449, in <module> >>>> do_setup(app) >>>> File "setup.py", line 96, in do_setup >>>> password = encrypt_password(p1) >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 150, in encrypt_password >>>> signed = get_hmac(password).decode('ascii') >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 108, in get_hmac >>>> 'set to "%s"' % _security.password_hash) >>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>> python setup.py >>>> pgAdmin 4 - Application Initialisation >>>> ====================================== >>>> >>>> User can not do any setup for web based now. >>>> >>>> >>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>> not exist. >>>> Entering initial setup mode... >>>> NOTE: Configuring authentication for SERVER mode. >>>> >>>> >>>> Enter the email address and password to use for the initial pgAdmin >>>> user account: >>>> >>>> Email address: [email protected] >>>> Password: >>>> Retype password: >>>> Traceback (most recent call last): >>>> File "setup.py", line 449, in <module> >>>> do_setup(app) >>>> File "setup.py", line 96, in do_setup >>>> password = encrypt_password(p1) >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 150, in encrypt_password >>>> signed = get_hmac(password).decode('ascii') >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 108, in get_hmac >>>> 'set to "%s"' % _security.password_hash) >>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>> >>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Dave, >>>>> >>>>> Testing Environment >>>>> >>>>> Ubuntu 16.04 Linux 64: >>>>> -------------------------------- >>>>> >>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>> >>>>> >>>>> 1) Install GIT >>>>> >>>>> sudo apt-get install git >>>>> >>>>> 2) Install pip3 >>>>> >>>>> sudo apt-get install python3-pip >>>>> >>>>> 3) Install virtualenv >>>>> >>>>> sudo pip3 install virtualenv >>>>> >>>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>>> module >>>>> >>>>> sudo apt-get install libpq-dev >>>>> >>>>> sudo apt-get install python3-dev >>>>> >>>>> 5) Create virtual environment >>>>> >>>>> virtualenv -p python3 venv >>>>> >>>>> 6) Create mkdir Projects >>>>> >>>>> 7) Clone git repo in Projects >>>>> >>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>> >>>>> 8) activate virtual environment >>>>> >>>>> source venv/bin/activate >>>>> >>>>> 9) Install modules >>>>> >>>>> pip3 install -r requirements_py3.txt >>>>> >>>>> *10) Edit the config.py file to config_local.py resides in >>>>> Projects\pgAdmin4\web * >>>>> >>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>> python setup.py >>>>> >>>>> If user does not create config_local.py and do Python setup.py for new >>>>> Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>> >>>>> Here is the output: >>>>> ------------------------- >>>>> >>>>> python setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>> >>>>> >>>>> Is this expected? >>>>> >>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Sure, >>>>>> >>>>>> Will test this thoroughly after complete investigation. >>>>>> >>>>>> Kind Regards, >>>>>> >>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >>>>>> >>>>>>> Patch applied. >>>>>>> >>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>> modes, with both fresh and upgraded installations? >>>>>>> >>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>> >>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>> installations, if you have included them in 1.0) >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Dave, >>>>>>>> >>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> >>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Dave, >>>>>>>>>>> >>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>> >>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>> happy with it? >>>>>>>>>>>> >>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>>>>>> 'runtime'. >>>>>>>>>>> >>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>> 'web' mode. >>>>>>>>>>> - Apply the patch >>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>> - Log out (if already login). >>>>>>>>>>> >>>>>>>>>>> And, you will see an exception. >>>>>>>>>>> >>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>> the Security object. >>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hmm. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>> these configurations from the database. >>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>> >>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>>> >>>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>>> Sigh... Needs more thought. >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>> >>>>>>>>> It looks like it's used system-wide as the key to generate an HMAC >>>>>>>>> of the users password, which is then passed to passlib which salts and >>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>> installations/databases. >>>>>>>>> >>>>>>>>> So, I believe we can do as you suggest and migrate existing values >>>>>>>>> for SECURITY_PASSWORD_SALT, given that there's clearly some other per user >>>>>>>>> and per installation/database salting going on anyway. New installations >>>>>>>>> can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>> >>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>>>>>> upgrade mode, which was wrong added in my addon patch. >>>>>>>> >>>>>>>> Please find the updated patch. >>>>>>>> >>>>>>>> Otherwise - looks good to me. >>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks & Regards, >>>>>>>> >>>>>>>> Ashesh Vashi >>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>> <http://www.enterprisedb.com/; >>>>>>>> >>>>>>>> >>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>> >>>>>>>>> >>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>> thus can be changed during an upgrade. >>>>>>>>> >>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>> >>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>> appreciated)! >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dave Page >>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>> Twitter: @pgsnake >>>>>>>>> >>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Dave Page >>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>> Twitter: @pgsnake >>>>>>> >>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> The Enterprise PostgreSQL Company >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 12:19 Fahar Abbas <[email protected]> parent: Neel Patel <[email protected]> 1 sibling, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-19 12:19 UTC (permalink / raw) To: Neel Patel <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Yes Neel is Right. This issue is also reproducible with Python 3.5 when user Launch python with pgAdmin4.py python pgAdmin4.py Starting pgAdmin 4. Please navigate to http://localhost:5050 in your browser. Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/usr/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.5/socketserver.py", line 628, in process_request_thread self.handle_error(request, client_address) File "/usr/lib/python3.5/socketserver.py", line 625, in process_request_thread self.finish_request(request, client_address) File "/usr/lib/python3.5/socketserver.py", line 354, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python3.5/socketserver.py", line 681, in __init__ self.handle() File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", line 200, in handle rv = BaseHTTPRequestHandler.handle(self) File "/usr/lib/python3.5/http/server.py", line 422, in handle self.handle_one_request() File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", line 235, in handle_one_request return self.run_wsgi() File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", line 177, in run_wsgi execute(self.server.app) File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", line 165, in execute application_iter = app(environ, start_response) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 2000, in __call__ return self.wsgi_app(environ, start_response) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1991, in wsgi_app response = self.make_response(self.handle_exception(e)) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1567, in handle_exception reraise(exc_type, exc_value, tb) File "/home/fahar/venv/lib/python3.5/site-packages/flask/_compat.py", line 33, in reraise raise value File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1988, in wsgi_app response = self.full_dispatch_request() File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1643, in full_dispatch_request response = self.process_response(response) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1864, in process_response self.save_session(ctx.session, response) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 926, in save_session return self.session_interface.save_session(self, session, response) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 267, in save_session self.manager.put(session) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 144, in put self.parent.put(session) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 214, in put session.sign(self.secret) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 71, in sign self.hmac_digest = _calc_hmac('%s:%s' % (self.sid, self.randval), secret) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 44, in _calc_hmac secret.encode(), body.encode(), hashlib.sha1 AttributeError: 'bytes' object has no attribute 'encode' On Wed, Oct 19, 2016 at 5:11 PM, Neel Patel <[email protected]> wrote: > Hi, > > Just to update for Python 3. > It gives below error while running "pgAdmin4.py". > > ##### > > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "/usr/lib/python3.4/threading.py", line 868, in run > self._target(*self._args, **self._kwargs) > File "/usr/lib/python3.4/socketserver.py", line 620, in > process_request_thread > self.handle_error(request, client_address) > File "/usr/lib/python3.4/socketserver.py", line 617, in > process_request_thread > self.finish_request(request, client_address) > File "/usr/lib/python3.4/socketserver.py", line 344, in finish_request > self.RequestHandlerClass(request, client_address, self) > File "/usr/lib/python3.4/socketserver.py", line 673, in __init__ > self.handle() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 200, in handle > rv = BaseHTTPRequestHandler.handle(self) > File "/usr/lib/python3.4/http/server.py", line 398, in handle > self.handle_one_request() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 235, in handle_one_request > return self.run_wsgi() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 177, in run_wsgi > execute(self.server.app) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 165, in execute > application_iter = app(environ, start_response) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 2000, in __call__ > return self.wsgi_app(environ, start_response) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1991, in wsgi_app > response = self.make_response(self.handle_exception(e)) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1567, in handle_exception > reraise(exc_type, exc_value, tb) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/flask/_compat.py", line 33, in reraise > raise value > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1988, in wsgi_app > response = self.full_dispatch_request() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1643, in full_dispatch_request > response = self.process_response(response) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1864, in process_response > self.save_session(ctx.session, response) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 926, in save_session > return self.session_interface.save_session(self, session, response) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 267, in save_session > self.manager.put(session) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 144, in put > self.parent.put(session) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 214, in put > session.sign(self.secret) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 71, in sign > self.hmac_digest = _calc_hmac('%s:%s' % (self.sid, self.randval), > secret) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 44, in _calc_hmac > secret.encode(), body.encode(), hashlib.sha1 > AttributeError: 'bytes' object has no attribute 'encode' > ####### > > Thanks, > Neel Patel > > On Wed, Oct 19, 2016 at 5:12 PM, Fahar Abbas <[email protected] > > wrote: > >> >> >> On Wed, Oct 19, 2016 at 4:03 PM, Fahar Abbas < >> [email protected]> wrote: >> >>> >>> >>> On Wed, Oct 19, 2016 at 3:55 PM, Ashesh Vashi < >>> [email protected]> wrote: >>> >>>> Hi Fahar, >>>> >>>> Please log the case on redmine. >>>> >>> https://redmine.postgresql.org/issues/1871 >>> >>>> Please find the attached patch, please apply it locally, and test it. >>>> >>>> And, please update the case, and this mail chain accordingly. >>>> >>> This is resolved now and no error message displayed when we apply the >> patch that is already shared. >> >>> >>>> Sure Will test the patch and update the status accordingly. >>> >>>> -- >>>> >>>> Thanks & Regards, >>>> >>>> Ashesh Vashi >>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>> <http://www.enterprisedb.com; >>>> >>>> >>>> *http://www.linkedin.com/in/asheshvashi* >>>> <http://www.linkedin.com/in/asheshvashi; >>>> >>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Here is the output of if we copy config_local.py and execute python >>>>> setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> python setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> User can not do any setup for web based now. >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> >>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Dave, >>>>>> >>>>>> Testing Environment >>>>>> >>>>>> Ubuntu 16.04 Linux 64: >>>>>> -------------------------------- >>>>>> >>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>> >>>>>> >>>>>> 1) Install GIT >>>>>> >>>>>> sudo apt-get install git >>>>>> >>>>>> 2) Install pip3 >>>>>> >>>>>> sudo apt-get install python3-pip >>>>>> >>>>>> 3) Install virtualenv >>>>>> >>>>>> sudo pip3 install virtualenv >>>>>> >>>>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>>>> module >>>>>> >>>>>> sudo apt-get install libpq-dev >>>>>> >>>>>> sudo apt-get install python3-dev >>>>>> >>>>>> 5) Create virtual environment >>>>>> >>>>>> virtualenv -p python3 venv >>>>>> >>>>>> 6) Create mkdir Projects >>>>>> >>>>>> 7) Clone git repo in Projects >>>>>> >>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>> >>>>>> 8) activate virtual environment >>>>>> >>>>>> source venv/bin/activate >>>>>> >>>>>> 9) Install modules >>>>>> >>>>>> pip3 install -r requirements_py3.txt >>>>>> >>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>> Projects\pgAdmin4\web * >>>>>> >>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>> python setup.py >>>>>> >>>>>> If user does not create config_local.py and do Python setup.py for >>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>> >>>>>> Here is the output: >>>>>> ------------------------- >>>>>> >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>> >>>>>> >>>>>> Is this expected? >>>>>> >>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Sure, >>>>>>> >>>>>>> Will test this thoroughly after complete investigation. >>>>>>> >>>>>>> Kind Regards, >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Patch applied. >>>>>>>> >>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>> >>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>> >>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>> installations, if you have included them in 1.0) >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Dave, >>>>>>>>> >>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave, >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>> >>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>> happy with it? >>>>>>>>>>>>> >>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>> >>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>> 'web' mode. >>>>>>>>>>>> - Apply the patch >>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>> >>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>> >>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>> the Security object. >>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hmm. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>> >>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>>>> >>>>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>>>> Sigh... Needs more thought. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>> >>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>> installations/databases. >>>>>>>>>> >>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>> >>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>> >>>>>>>>> Please find the updated patch. >>>>>>>>> >>>>>>>>> Otherwise - looks good to me. >>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks & Regards, >>>>>>>>> >>>>>>>>> Ashesh Vashi >>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>> >>>>>>>>> >>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>> >>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>> >>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>> appreciated)! >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 12:39 Sandeep Thakkar <[email protected]> parent: Sandeep Thakkar <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Sandeep Thakkar @ 2016-10-19 12:39 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Syed Fahar Abbas <[email protected]>; Hamid Quddus Akhtar <[email protected]> Here is the patch where we remove the config_local.py being created during packaging. The mac build script missed creating config_distro.py earlier and it has been take care of now. Please review the attached patch. I'll also make the changes in the EDB packaging scripts where we bundle pgAdmin in PG server and EPAS Meta. On Wed, Oct 19, 2016 at 4:35 PM, Sandeep Thakkar < [email protected]> wrote: > Hi Dave, > > On Wed, Oct 19, 2016 at 1:57 PM, Dave Page <[email protected]> wrote: > >> Patch applied. >> >> Fahar, can you please test this thoroughly in desktop and server modes, >> with both fresh and upgraded installations? >> >> https://redmine.postgresql.org/issues/1849 >> >> Packagers: This change means that packages are no longer forced to create >> a config_local.py file, and there is no longer any need to explicitly set >> SECURITY_PASSWORD_SALT, SECURITY_KEY and CSRF_SESSION_KEY in the config >> (in fact, they should be removed for new installations, if you have >> included them in 1.0) >> >> OK. Will remove config_local.py from the packaging. We do not set the > mentioned directives in the config. > > >> Thanks. >> >> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >> [email protected]> wrote: >> >>> Hi Dave, >>> >>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >>> >>>> Hi >>>> >>>> >>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>> >>>>> Hi >>>>> >>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Dave, >>>>>> >>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> wrote: >>>>>> >>>>>>> Hi Ashesh, >>>>>>> >>>>>>> Can you please review the attached patch, and apply if you're happy >>>>>>> with it? >>>>>>> >>>>>> Overall the patch looked good to me. >>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>> 'runtime'. >>>>>> >>>>>> Steps for reproduction on existing pgAdmin 4 environment with 'web' >>>>>> mode. >>>>>> - Apply the patch >>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>> - Open pgAdmin home page. >>>>>> - Log out (if already login). >>>>>> >>>>>> And, you will see an exception. >>>>>> >>>>>> I have figure out the issue with the patch. >>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>>>> Security object. >>>>>> Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT >>>>>> properly. >>>>>> >>>>> >>>>> Hmm. >>>>> >>>>> >>>>>> >>>>>> I had moved the Security object initialization after fetching these >>>>>> configurations from the database. >>>>>> I have attached a addon patch for the same. >>>>>> >>>>> >>>>> OK, thanks. >>>>> >>>>> >>>>>> >>>>>> Now - I run into another issue. >>>>>> Because - the existing password was hashed using the old >>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>> >>>>>> I think - we need to think about different strategy for upgrading the >>>>>> configuration file in the 'web' mode. >>>>>> I was thinking - we can store the existing security configurations in >>>>>> the database during upgrade process in 'web' mode. >>>>>> >>>>> >>>>> My concern with that is that we'll likely be storing the default >>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>> >>>>> I guess what we need to do is re-encrypt the password during the >>>>> upgrade - however, that makes me think; we then have both the key and the >>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>> Sigh... Needs more thought. >>>>> >>>> >>>> OK, so I've been thinking about this and experimenting for a couple of >>>> hours, as well as annoying the crap out of Magnus by thinking out loud in >>>> his general direction, and it looks like this isn't a major problem as from >>>> what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key >>>> not a salt) not the only salting that's done. >>>> >>>> It looks like it's used system-wide as the key to generate an HMAC of >>>> the users password, which is then passed to passlib which salts and hashes >>>> it. I did some testing, and found that two users with the same password end >>>> up with different hashes in the database, so clearly there is also per-user >>>> salting happening. I also created two users, then dropped the database and >>>> created the same user accounts with the same passwords again, and found >>>> that the resulting hashes were different in both databases - thus there is >>>> something else ensuring the hashes are unique across different >>>> installations/databases. >>>> >>>> So, I believe we can do as you suggest and migrate existing values for >>>> SECURITY_PASSWORD_SALT, given that there's clearly some other per user and >>>> per installation/database salting going on anyway. New installations can >>>> have the random value for SECURITY_PASSWORD_SALT. >>>> >>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>> upgrade mode, which was wrong added in my addon patch. >>> >>> Please find the updated patch. >>> >>> Otherwise - looks good to me. >>> Please commit the new patch (if you're ok with the change). >>> >>> >>> -- >>> >>> Thanks & Regards, >>> >>> Ashesh Vashi >>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>> <http://www.enterprisedb.com/; >>> >>> >>> *http://www.linkedin.com/in/asheshvashi* >>> <http://www.linkedin.com/in/asheshvashi; >>> >>>> >>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, as >>>> they're used for purposes that are essentially ephemeral, and thus can be >>>> changed during an upgrade. >>>> >>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>> >>>> Patch attached - please review (Ashesh, but others too would be >>>> appreciated)! >>>> >>>> Thanks. >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>>> >>> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > > > -- > Sandeep Thakkar > > > > -- Sandeep Thakkar -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/octet-stream] donot-create-local.patch (1.4K, 3-donot-create-local.patch) download | inline diff: diff --git a/Make.bat b/Make.bat index 9ab7773..bf1707c 100644 --- a/Make.bat +++ b/Make.bat @@ -247,9 +247,6 @@ GOTO:EOF ECHO HELP_PATH = '../../../docs/en_US/html/' >> "%PGBUILDPATH%\web\config_distro.py" ECHO MINIFY_HTML = False >> "%PGBUILDPATH%\web\config_distro.py" - ECHO Creating config_local.py - ECHO # Add any configuration changes to this file. > "%PGBUILDPATH%\web\config_local.py" - ECHO Building docs... MKDIR "%PGBUILDPATH%\docs\en_US\html" IF %ERRORLEVEL% NEQ 0 EXIT /B %ERRORLEVEL% diff --git a/pkg/mac/build.sh b/pkg/mac/build.sh index 1faee08..3fdb34f 100755 --- a/pkg/mac/build.sh +++ b/pkg/mac/build.sh @@ -156,9 +156,9 @@ _complete_bundle() { cp -r $SOURCEDIR/web "$BUILDROOT/$APP_BUNDLE_NAME/Contents/Resources/" || exit 1 cd "$BUILDROOT/$APP_BUNDLE_NAME/Contents/Resources/web" rm -f pgadmin4.db config_local.* - echo "SERVER_MODE = False" > config_local.py - echo "MINIFY_HTML = False" >> config_local.py - echo "HELP_PATH = '../../../docs/en_US/html/'" >> config_local.py + echo "SERVER_MODE = False" > config_distro.py + echo "MINIFY_HTML = False" >> config_distro.py + echo "HELP_PATH = '../../../docs/en_US/html/'" >> config_distro.py # Remove the .pyc files if any cd "$BUILDROOT/$APP_BUNDLE_NAME" @@ -211,4 +211,4 @@ _build_doc _complete_bundle _codesign_bundle _create_dmg -_codesign_dmg \ No newline at end of file +_codesign_dmg ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 12:51 Fahar Abbas <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 0 replies; 33+ messages in thread From: Fahar Abbas @ 2016-10-19 12:51 UTC (permalink / raw) To: Neel Patel <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> After applying this patch on Python 2.7.11 on Windows 2012 64bit, User can launch pgAdmin4 with python pgAdmin4.py without any error. Will also retest once the 1871 will be resolved On Wed, Oct 19, 2016 at 5:19 PM, Fahar Abbas <[email protected]> wrote: > Yes Neel is Right. > > This issue is also reproducible with Python 3.5 when user Launch python > with pgAdmin4.py > > python pgAdmin4.py > Starting pgAdmin 4. Please navigate to http://localhost:5050 in your > browser. > Exception in thread Thread-1: > Traceback (most recent call last): > File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner > self.run() > File "/usr/lib/python3.5/threading.py", line 862, in run > self._target(*self._args, **self._kwargs) > File "/usr/lib/python3.5/socketserver.py", line 628, in > process_request_thread > self.handle_error(request, client_address) > File "/usr/lib/python3.5/socketserver.py", line 625, in > process_request_thread > self.finish_request(request, client_address) > File "/usr/lib/python3.5/socketserver.py", line 354, in finish_request > self.RequestHandlerClass(request, client_address, self) > File "/usr/lib/python3.5/socketserver.py", line 681, in __init__ > self.handle() > File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", > line 200, in handle > rv = BaseHTTPRequestHandler.handle(self) > File "/usr/lib/python3.5/http/server.py", line 422, in handle > self.handle_one_request() > File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", > line 235, in handle_one_request > return self.run_wsgi() > File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", > line 177, in run_wsgi > execute(self.server.app) > File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", > line 165, in execute > application_iter = app(environ, start_response) > File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line > 2000, in __call__ > return self.wsgi_app(environ, start_response) > File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line > 1991, in wsgi_app > response = self.make_response(self.handle_exception(e)) > File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line > 1567, in handle_exception > reraise(exc_type, exc_value, tb) > File "/home/fahar/venv/lib/python3.5/site-packages/flask/_compat.py", > line 33, in reraise > raise value > File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line > 1988, in wsgi_app > response = self.full_dispatch_request() > File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line > 1643, in full_dispatch_request > response = self.process_response(response) > File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line > 1864, in process_response > self.save_session(ctx.session, response) > File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line > 926, in save_session > return self.session_interface.save_session(self, session, response) > File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line > 267, in save_session > self.manager.put(session) > File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line > 144, in put > self.parent.put(session) > File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line > 214, in put > session.sign(self.secret) > File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line > 71, in sign > self.hmac_digest = _calc_hmac('%s:%s' % (self.sid, self.randval), > secret) > File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line > 44, in _calc_hmac > secret.encode(), body.encode(), hashlib.sha1 > AttributeError: 'bytes' object has no attribute 'encode' > > > On Wed, Oct 19, 2016 at 5:11 PM, Neel Patel <[email protected]> > wrote: > >> Hi, >> >> Just to update for Python 3. >> It gives below error while running "pgAdmin4.py". >> >> ##### >> >> Traceback (most recent call last): >> File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner >> self.run() >> File "/usr/lib/python3.4/threading.py", line 868, in run >> self._target(*self._args, **self._kwargs) >> File "/usr/lib/python3.4/socketserver.py", line 620, in >> process_request_thread >> self.handle_error(request, client_address) >> File "/usr/lib/python3.4/socketserver.py", line 617, in >> process_request_thread >> self.finish_request(request, client_address) >> File "/usr/lib/python3.4/socketserver.py", line 344, in finish_request >> self.RequestHandlerClass(request, client_address, self) >> File "/usr/lib/python3.4/socketserver.py", line 673, in __init__ >> self.handle() >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/werkzeug/serving.py", >> line 200, in handle >> rv = BaseHTTPRequestHandler.handle(self) >> File "/usr/lib/python3.4/http/server.py", line 398, in handle >> self.handle_one_request() >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/werkzeug/serving.py", >> line 235, in handle_one_request >> return self.run_wsgi() >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/werkzeug/serving.py", >> line 177, in run_wsgi >> execute(self.server.app) >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/werkzeug/serving.py", >> line 165, in execute >> application_iter = app(environ, start_response) >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", >> line 2000, in __call__ >> return self.wsgi_app(environ, start_response) >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", >> line 1991, in wsgi_app >> response = self.make_response(self.handle_exception(e)) >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", >> line 1567, in handle_exception >> reraise(exc_type, exc_value, tb) >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/_compat.py", >> line 33, in reraise >> raise value >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", >> line 1988, in wsgi_app >> response = self.full_dispatch_request() >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", >> line 1643, in full_dispatch_request >> response = self.process_response(response) >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", >> line 1864, in process_response >> self.save_session(ctx.session, response) >> File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", >> line 926, in save_session >> return self.session_interface.save_session(self, session, response) >> File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", >> line 267, in save_session >> self.manager.put(session) >> File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", >> line 144, in put >> self.parent.put(session) >> File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", >> line 214, in put >> session.sign(self.secret) >> File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", >> line 71, in sign >> self.hmac_digest = _calc_hmac('%s:%s' % (self.sid, self.randval), >> secret) >> File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", >> line 44, in _calc_hmac >> secret.encode(), body.encode(), hashlib.sha1 >> AttributeError: 'bytes' object has no attribute 'encode' >> ####### >> >> Thanks, >> Neel Patel >> >> On Wed, Oct 19, 2016 at 5:12 PM, Fahar Abbas < >> [email protected]> wrote: >> >>> >>> >>> On Wed, Oct 19, 2016 at 4:03 PM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> >>>> >>>> On Wed, Oct 19, 2016 at 3:55 PM, Ashesh Vashi < >>>> [email protected]> wrote: >>>> >>>>> Hi Fahar, >>>>> >>>>> Please log the case on redmine. >>>>> >>>> https://redmine.postgresql.org/issues/1871 >>>> >>>>> Please find the attached patch, please apply it locally, and test it. >>>>> >>>>> And, please update the case, and this mail chain accordingly. >>>>> >>>> This is resolved now and no error message displayed when we apply the >>> patch that is already shared. >>> >>>> >>>>> Sure Will test the patch and update the status accordingly. >>>> >>>>> -- >>>>> >>>>> Thanks & Regards, >>>>> >>>>> Ashesh Vashi >>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>> <http://www.enterprisedb.com; >>>>> >>>>> >>>>> *http://www.linkedin.com/in/asheshvashi* >>>>> <http://www.linkedin.com/in/asheshvashi; >>>>> >>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Here is the output of if we copy config_local.py and execute python >>>>>> setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> User can not do any setup for web based now. >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> >>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Dave, >>>>>>> >>>>>>> Testing Environment >>>>>>> >>>>>>> Ubuntu 16.04 Linux 64: >>>>>>> -------------------------------- >>>>>>> >>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>> >>>>>>> >>>>>>> 1) Install GIT >>>>>>> >>>>>>> sudo apt-get install git >>>>>>> >>>>>>> 2) Install pip3 >>>>>>> >>>>>>> sudo apt-get install python3-pip >>>>>>> >>>>>>> 3) Install virtualenv >>>>>>> >>>>>>> sudo pip3 install virtualenv >>>>>>> >>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>> pycrypto module >>>>>>> >>>>>>> sudo apt-get install libpq-dev >>>>>>> >>>>>>> sudo apt-get install python3-dev >>>>>>> >>>>>>> 5) Create virtual environment >>>>>>> >>>>>>> virtualenv -p python3 venv >>>>>>> >>>>>>> 6) Create mkdir Projects >>>>>>> >>>>>>> 7) Clone git repo in Projects >>>>>>> >>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>> >>>>>>> 8) activate virtual environment >>>>>>> >>>>>>> source venv/bin/activate >>>>>>> >>>>>>> 9) Install modules >>>>>>> >>>>>>> pip3 install -r requirements_py3.txt >>>>>>> >>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>> Projects\pgAdmin4\web * >>>>>>> >>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>> python setup.py >>>>>>> >>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>> >>>>>>> Here is the output: >>>>>>> ------------------------- >>>>>>> >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>> >>>>>>> >>>>>>> Is this expected? >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Sure, >>>>>>>> >>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>> >>>>>>>> Kind Regards, >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Patch applied. >>>>>>>>> >>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>> >>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>> >>>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Dave, >>>>>>>>>> >>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>>> happy with it? >>>>>>>>>>>>>> >>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>> >>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>>> 'web' mode. >>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>> >>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>> >>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>>> the Security object. >>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hmm. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>> >>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>> problem. >>>>>>>>>>>> >>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>> >>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>> installations/databases. >>>>>>>>>>> >>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>> >>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>> >>>>>>>>>> Please find the updated patch. >>>>>>>>>> >>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks & Regards, >>>>>>>>>> >>>>>>>>>> Ashesh Vashi >>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>> >>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>> >>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>>> appreciated)! >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dave Page >>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>> Twitter: @pgsnake >>>>>>>>> >>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 12:56 Dave Page <[email protected]> parent: Neel Patel <[email protected]> 1 sibling, 0 replies; 33+ messages in thread From: Dave Page @ 2016-10-19 12:56 UTC (permalink / raw) To: Neel Patel <[email protected]>; +Cc: Fahar Abbas <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> I assume that's an existing issue with Python 3.5? That file wasn't changed by this patch. On Wed, Oct 19, 2016 at 1:11 PM, Neel Patel <[email protected]> wrote: > Hi, > > Just to update for Python 3. > It gives below error while running "pgAdmin4.py". > > ##### > > Traceback (most recent call last): > File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner > self.run() > File "/usr/lib/python3.4/threading.py", line 868, in run > self._target(*self._args, **self._kwargs) > File "/usr/lib/python3.4/socketserver.py", line 620, in > process_request_thread > self.handle_error(request, client_address) > File "/usr/lib/python3.4/socketserver.py", line 617, in > process_request_thread > self.finish_request(request, client_address) > File "/usr/lib/python3.4/socketserver.py", line 344, in finish_request > self.RequestHandlerClass(request, client_address, self) > File "/usr/lib/python3.4/socketserver.py", line 673, in __init__ > self.handle() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 200, in handle > rv = BaseHTTPRequestHandler.handle(self) > File "/usr/lib/python3.4/http/server.py", line 398, in handle > self.handle_one_request() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 235, in handle_one_request > return self.run_wsgi() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 177, in run_wsgi > execute(self.server.app) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/werkzeug/serving.py", line 165, in execute > application_iter = app(environ, start_response) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 2000, in __call__ > return self.wsgi_app(environ, start_response) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1991, in wsgi_app > response = self.make_response(self.handle_exception(e)) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1567, in handle_exception > reraise(exc_type, exc_value, tb) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/ > site-packages/flask/_compat.py", line 33, in reraise > raise value > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1988, in wsgi_app > response = self.full_dispatch_request() > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1643, in full_dispatch_request > response = self.process_response(response) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 1864, in process_response > self.save_session(ctx.session, response) > File "/home/neel/workspace/pgAdmin4_3_4/lib/python3.4/site-packages/flask/app.py", > line 926, in save_session > return self.session_interface.save_session(self, session, response) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 267, in save_session > self.manager.put(session) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 144, in put > self.parent.put(session) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 214, in put > session.sign(self.secret) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 71, in sign > self.hmac_digest = _calc_hmac('%s:%s' % (self.sid, self.randval), > secret) > File "/home/neel/Projects/pgAdmin4/pgadmin4_patch/pgadmin4/web/pgadmin/utils/session.py", > line 44, in _calc_hmac > secret.encode(), body.encode(), hashlib.sha1 > AttributeError: 'bytes' object has no attribute 'encode' > ####### > > Thanks, > Neel Patel > > On Wed, Oct 19, 2016 at 5:12 PM, Fahar Abbas <[email protected] > > wrote: > >> >> >> On Wed, Oct 19, 2016 at 4:03 PM, Fahar Abbas < >> [email protected]> wrote: >> >>> >>> >>> On Wed, Oct 19, 2016 at 3:55 PM, Ashesh Vashi < >>> [email protected]> wrote: >>> >>>> Hi Fahar, >>>> >>>> Please log the case on redmine. >>>> >>> https://redmine.postgresql.org/issues/1871 >>> >>>> Please find the attached patch, please apply it locally, and test it. >>>> >>>> And, please update the case, and this mail chain accordingly. >>>> >>> This is resolved now and no error message displayed when we apply the >> patch that is already shared. >> >>> >>>> Sure Will test the patch and update the status accordingly. >>> >>>> -- >>>> >>>> Thanks & Regards, >>>> >>>> Ashesh Vashi >>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>> <http://www.enterprisedb.com; >>>> >>>> >>>> *http://www.linkedin.com/in/asheshvashi* >>>> <http://www.linkedin.com/in/asheshvashi; >>>> >>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Here is the output of if we copy config_local.py and execute python >>>>> setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> python setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> User can not do any setup for web based now. >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> >>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Dave, >>>>>> >>>>>> Testing Environment >>>>>> >>>>>> Ubuntu 16.04 Linux 64: >>>>>> -------------------------------- >>>>>> >>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>> >>>>>> >>>>>> 1) Install GIT >>>>>> >>>>>> sudo apt-get install git >>>>>> >>>>>> 2) Install pip3 >>>>>> >>>>>> sudo apt-get install python3-pip >>>>>> >>>>>> 3) Install virtualenv >>>>>> >>>>>> sudo pip3 install virtualenv >>>>>> >>>>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>>>> module >>>>>> >>>>>> sudo apt-get install libpq-dev >>>>>> >>>>>> sudo apt-get install python3-dev >>>>>> >>>>>> 5) Create virtual environment >>>>>> >>>>>> virtualenv -p python3 venv >>>>>> >>>>>> 6) Create mkdir Projects >>>>>> >>>>>> 7) Clone git repo in Projects >>>>>> >>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>> >>>>>> 8) activate virtual environment >>>>>> >>>>>> source venv/bin/activate >>>>>> >>>>>> 9) Install modules >>>>>> >>>>>> pip3 install -r requirements_py3.txt >>>>>> >>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>> Projects\pgAdmin4\web * >>>>>> >>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>> python setup.py >>>>>> >>>>>> If user does not create config_local.py and do Python setup.py for >>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>> >>>>>> Here is the output: >>>>>> ------------------------- >>>>>> >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>> >>>>>> >>>>>> Is this expected? >>>>>> >>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Sure, >>>>>>> >>>>>>> Will test this thoroughly after complete investigation. >>>>>>> >>>>>>> Kind Regards, >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Patch applied. >>>>>>>> >>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>> >>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>> >>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>> installations, if you have included them in 1.0) >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Dave, >>>>>>>>> >>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave, >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>> >>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>> happy with it? >>>>>>>>>>>>> >>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>> >>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>> 'web' mode. >>>>>>>>>>>> - Apply the patch >>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>> >>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>> >>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>> the Security object. >>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hmm. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>> >>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>>>> >>>>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>>>> Sigh... Needs more thought. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>> >>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>> installations/databases. >>>>>>>>>> >>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>> >>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>> >>>>>>>>> Please find the updated patch. >>>>>>>>> >>>>>>>>> Otherwise - looks good to me. >>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks & Regards, >>>>>>>>> >>>>>>>>> Ashesh Vashi >>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>> >>>>>>>>> >>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>> >>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>> >>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>> appreciated)! >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 13:04 Dave Page <[email protected]> parent: Ashesh Vashi <[email protected]> 1 sibling, 1 reply; 33+ messages in thread From: Dave Page @ 2016-10-19 13:04 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: Fahar Abbas <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Patch applied. On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < [email protected]> wrote: > Hi Fahar, > > Please log the case on redmine. > Please find the attached patch, please apply it locally, and test it. > > And, please update the case, and this mail chain accordingly. > > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > <http://www.enterprisedb.com; > > > *http://www.linkedin.com/in/asheshvashi* > <http://www.linkedin.com/in/asheshvashi; > > On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas <[email protected] > > wrote: > >> Here is the output of if we copy config_local.py and execute python >> setup.py >> pgAdmin 4 - Application Initialisation >> ====================================== >> >> >> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not >> exist. >> Entering initial setup mode... >> NOTE: Configuring authentication for SERVER mode. >> >> >> Enter the email address and password to use for the initial pgAdmin >> user account: >> >> Email address: [email protected] >> Password: >> Retype password: >> Traceback (most recent call last): >> File "setup.py", line 449, in <module> >> do_setup(app) >> File "setup.py", line 96, in do_setup >> password = encrypt_password(p1) >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 150, in encrypt_password >> signed = get_hmac(password).decode('ascii') >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 108, in get_hmac >> 'set to "%s"' % _security.password_hash) >> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >> python setup.py >> pgAdmin 4 - Application Initialisation >> ====================================== >> >> User can not do any setup for web based now. >> >> >> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not >> exist. >> Entering initial setup mode... >> NOTE: Configuring authentication for SERVER mode. >> >> >> Enter the email address and password to use for the initial pgAdmin >> user account: >> >> Email address: [email protected] >> Password: >> Retype password: >> Traceback (most recent call last): >> File "setup.py", line 449, in <module> >> do_setup(app) >> File "setup.py", line 96, in do_setup >> password = encrypt_password(p1) >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 150, in encrypt_password >> signed = get_hmac(password).decode('ascii') >> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >> line 108, in get_hmac >> 'set to "%s"' % _security.password_hash) >> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >> >> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >> [email protected]> wrote: >> >>> Dave, >>> >>> Testing Environment >>> >>> Ubuntu 16.04 Linux 64: >>> -------------------------------- >>> >>> pg-AdminIV Development Environment Setup for Ubuntu : >>> >>> >>> 1) Install GIT >>> >>> sudo apt-get install git >>> >>> 2) Install pip3 >>> >>> sudo apt-get install python3-pip >>> >>> 3) Install virtualenv >>> >>> sudo pip3 install virtualenv >>> >>> 4) install below dependency as it is required for psycopg2 & pycrypto >>> module >>> >>> sudo apt-get install libpq-dev >>> >>> sudo apt-get install python3-dev >>> >>> 5) Create virtual environment >>> >>> virtualenv -p python3 venv >>> >>> 6) Create mkdir Projects >>> >>> 7) Clone git repo in Projects >>> >>> git clone http://git.postgresql.org/git/pgadmin4.git >>> >>> 8) activate virtual environment >>> >>> source venv/bin/activate >>> >>> 9) Install modules >>> >>> pip3 install -r requirements_py3.txt >>> >>> *10) Edit the config.py file to config_local.py resides in >>> Projects\pgAdmin4\web * >>> >>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>> python setup.py >>> >>> If user does not create config_local.py and do Python setup.py for new >>> Development then SECURITY_PASSWORD_SALT message is also displayed: >>> >>> Here is the output: >>> ------------------------- >>> >>> python setup.py >>> pgAdmin 4 - Application Initialisation >>> ====================================== >>> >>> >>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>> not exist. >>> Entering initial setup mode... >>> NOTE: Configuring authentication for SERVER mode. >>> >>> >>> Enter the email address and password to use for the initial pgAdmin >>> user account: >>> >>> Email address: [email protected] >>> Password: >>> Retype password: >>> Traceback (most recent call last): >>> File "setup.py", line 449, in <module> >>> do_setup(app) >>> File "setup.py", line 96, in do_setup >>> password = encrypt_password(p1) >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 150, in encrypt_password >>> signed = get_hmac(password).decode('ascii') >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 108, in get_hmac >>> 'set to "%s"' % _security.password_hash) >>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>> >>> >>> Is this expected? >>> >>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Sure, >>>> >>>> Will test this thoroughly after complete investigation. >>>> >>>> Kind Regards, >>>> >>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >>>> >>>>> Patch applied. >>>>> >>>>> Fahar, can you please test this thoroughly in desktop and server >>>>> modes, with both fresh and upgraded installations? >>>>> >>>>> https://redmine.postgresql.org/issues/1849 >>>>> >>>>> Packagers: This change means that packages are no longer forced to >>>>> create a config_local.py file, and there is no longer any need to >>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>> installations, if you have included them in 1.0) >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Dave, >>>>>> >>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> >>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Dave, >>>>>>>>> >>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Ashesh, >>>>>>>>>> >>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>> happy with it? >>>>>>>>>> >>>>>>>>> Overall the patch looked good to me. >>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>>>> 'runtime'. >>>>>>>>> >>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>> 'web' mode. >>>>>>>>> - Apply the patch >>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>> - Open pgAdmin home page. >>>>>>>>> - Log out (if already login). >>>>>>>>> >>>>>>>>> And, you will see an exception. >>>>>>>>> >>>>>>>>> I have figure out the issue with the patch. >>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>>>>>>> Security object. >>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>> >>>>>>>> >>>>>>>> Hmm. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>> these configurations from the database. >>>>>>>>> I have attached a addon patch for the same. >>>>>>>>> >>>>>>>> >>>>>>>> OK, thanks. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Now - I run into another issue. >>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>> >>>>>>>>> I think - we need to think about different strategy for upgrading >>>>>>>>> the configuration file in the 'web' mode. >>>>>>>>> I was thinking - we can store the existing security configurations >>>>>>>>> in the database during upgrade process in 'web' mode. >>>>>>>>> >>>>>>>> >>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>> >>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>> Sigh... Needs more thought. >>>>>>>> >>>>>>> >>>>>>> OK, so I've been thinking about this and experimenting for a couple >>>>>>> of hours, as well as annoying the crap out of Magnus by thinking out loud >>>>>>> in his general direction, and it looks like this isn't a major problem as >>>>>>> from what I can see, SECURITY_PASSWORD_SALT is (aside from really being a >>>>>>> key not a salt) not the only salting that's done. >>>>>>> >>>>>>> It looks like it's used system-wide as the key to generate an HMAC >>>>>>> of the users password, which is then passed to passlib which salts and >>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>> database and created the same user accounts with the same passwords again, >>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>> there is something else ensuring the hashes are unique across different >>>>>>> installations/databases. >>>>>>> >>>>>>> So, I believe we can do as you suggest and migrate existing values >>>>>>> for SECURITY_PASSWORD_SALT, given that there's clearly some other per user >>>>>>> and per installation/database salting going on anyway. New installations >>>>>>> can have the random value for SECURITY_PASSWORD_SALT. >>>>>>> >>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>>>> upgrade mode, which was wrong added in my addon patch. >>>>>> >>>>>> Please find the updated patch. >>>>>> >>>>>> Otherwise - looks good to me. >>>>>> Please commit the new patch (if you're ok with the change). >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & Regards, >>>>>> >>>>>> Ashesh Vashi >>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>> <http://www.enterprisedb.com/; >>>>>> >>>>>> >>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>> >>>>>>> >>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, >>>>>>> as they're used for purposes that are essentially ephemeral, and thus can >>>>>>> be changed during an upgrade. >>>>>>> >>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>> >>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>> appreciated)! >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Dave Page >>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>> Twitter: @pgsnake >>>>>>> >>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> The Enterprise PostgreSQL Company >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-19 13:10 Dave Page <[email protected]> parent: Sandeep Thakkar <[email protected]> 0 siblings, 0 replies; 33+ messages in thread From: Dave Page @ 2016-10-19 13:10 UTC (permalink / raw) To: Sandeep Thakkar <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Syed Fahar Abbas <[email protected]>; Hamid Quddus Akhtar <[email protected]> Thanks, applied. On Wed, Oct 19, 2016 at 1:39 PM, Sandeep Thakkar < [email protected]> wrote: > Here is the patch where we remove the config_local.py being created during > packaging. The mac build script missed creating config_distro.py earlier > and it has been take care of now. Please review the attached patch. > > I'll also make the changes in the EDB packaging scripts where we bundle > pgAdmin in PG server and EPAS Meta. > > On Wed, Oct 19, 2016 at 4:35 PM, Sandeep Thakkar < > [email protected]> wrote: > >> Hi Dave, >> >> On Wed, Oct 19, 2016 at 1:57 PM, Dave Page <[email protected]> wrote: >> >>> Patch applied. >>> >>> Fahar, can you please test this thoroughly in desktop and server modes, >>> with both fresh and upgraded installations? >>> >>> https://redmine.postgresql.org/issues/1849 >>> >>> Packagers: This change means that packages are no longer forced to >>> create a config_local.py file, and there is no longer any need to >>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>> installations, if you have included them in 1.0) >>> >>> OK. Will remove config_local.py from the packaging. We do not set the >> mentioned directives in the config. >> >> >>> Thanks. >>> >>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>> [email protected]> wrote: >>> >>>> Hi Dave, >>>> >>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >>>> >>>>> Hi >>>>> >>>>> >>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Dave, >>>>>>> >>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Ashesh, >>>>>>>> >>>>>>>> Can you please review the attached patch, and apply if you're happy >>>>>>>> with it? >>>>>>>> >>>>>>> Overall the patch looked good to me. >>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>> 'runtime'. >>>>>>> >>>>>>> Steps for reproduction on existing pgAdmin 4 environment with 'web' >>>>>>> mode. >>>>>>> - Apply the patch >>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>> - Open pgAdmin home page. >>>>>>> - Log out (if already login). >>>>>>> >>>>>>> And, you will see an exception. >>>>>>> >>>>>>> I have figure out the issue with the patch. >>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the >>>>>>> Security object. >>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>> >>>>>> >>>>>> Hmm. >>>>>> >>>>>> >>>>>>> >>>>>>> I had moved the Security object initialization after fetching these >>>>>>> configurations from the database. >>>>>>> I have attached a addon patch for the same. >>>>>>> >>>>>> >>>>>> OK, thanks. >>>>>> >>>>>> >>>>>>> >>>>>>> Now - I run into another issue. >>>>>>> Because - the existing password was hashed using the old >>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>> >>>>>>> I think - we need to think about different strategy for upgrading >>>>>>> the configuration file in the 'web' mode. >>>>>>> I was thinking - we can store the existing security configurations >>>>>>> in the database during upgrade process in 'web' mode. >>>>>>> >>>>>> >>>>>> My concern with that is that we'll likely be storing the default >>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>> >>>>>> I guess what we need to do is re-encrypt the password during the >>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>> Sigh... Needs more thought. >>>>>> >>>>> >>>>> OK, so I've been thinking about this and experimenting for a couple of >>>>> hours, as well as annoying the crap out of Magnus by thinking out loud in >>>>> his general direction, and it looks like this isn't a major problem as from >>>>> what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key >>>>> not a salt) not the only salting that's done. >>>>> >>>>> It looks like it's used system-wide as the key to generate an HMAC of >>>>> the users password, which is then passed to passlib which salts and hashes >>>>> it. I did some testing, and found that two users with the same password end >>>>> up with different hashes in the database, so clearly there is also per-user >>>>> salting happening. I also created two users, then dropped the database and >>>>> created the same user accounts with the same passwords again, and found >>>>> that the resulting hashes were different in both databases - thus there is >>>>> something else ensuring the hashes are unique across different >>>>> installations/databases. >>>>> >>>>> So, I believe we can do as you suggest and migrate existing values for >>>>> SECURITY_PASSWORD_SALT, given that there's clearly some other per user and >>>>> per installation/database salting going on anyway. New installations can >>>>> have the random value for SECURITY_PASSWORD_SALT. >>>>> >>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>> upgrade mode, which was wrong added in my addon patch. >>>> >>>> Please find the updated patch. >>>> >>>> Otherwise - looks good to me. >>>> Please commit the new patch (if you're ok with the change). >>>> >>>> >>>> -- >>>> >>>> Thanks & Regards, >>>> >>>> Ashesh Vashi >>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>> <http://www.enterprisedb.com/; >>>> >>>> >>>> *http://www.linkedin.com/in/asheshvashi* >>>> <http://www.linkedin.com/in/asheshvashi; >>>> >>>>> >>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, >>>>> as they're used for purposes that are essentially ephemeral, and thus can >>>>> be changed during an upgrade. >>>>> >>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>> >>>>> Patch attached - please review (Ashesh, but others too would be >>>>> appreciated)! >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>>> >>>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> >> >> >> -- >> Sandeep Thakkar >> >> >> >> > > > -- > Sandeep Thakkar > > > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 05:33 Fahar Abbas <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-20 05:33 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Hi Dave, I have reopened following RM: ================================ https://redmine.postgresql.org/issues/1849 On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: > Patch applied. > > On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < > [email protected]> wrote: > >> Hi Fahar, >> >> Please log the case on redmine. >> Please find the attached patch, please apply it locally, and test it. >> >> And, please update the case, and this mail chain accordingly. >> >> -- >> >> Thanks & Regards, >> >> Ashesh Vashi >> EnterpriseDB INDIA: Enterprise PostgreSQL Company >> <http://www.enterprisedb.com; >> >> >> *http://www.linkedin.com/in/asheshvashi* >> <http://www.linkedin.com/in/asheshvashi; >> >> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >> [email protected]> wrote: >> >>> Here is the output of if we copy config_local.py and execute python >>> setup.py >>> pgAdmin 4 - Application Initialisation >>> ====================================== >>> >>> >>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>> not exist. >>> Entering initial setup mode... >>> NOTE: Configuring authentication for SERVER mode. >>> >>> >>> Enter the email address and password to use for the initial pgAdmin >>> user account: >>> >>> Email address: [email protected] >>> Password: >>> Retype password: >>> Traceback (most recent call last): >>> File "setup.py", line 449, in <module> >>> do_setup(app) >>> File "setup.py", line 96, in do_setup >>> password = encrypt_password(p1) >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 150, in encrypt_password >>> signed = get_hmac(password).decode('ascii') >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 108, in get_hmac >>> 'set to "%s"' % _security.password_hash) >>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>> python setup.py >>> pgAdmin 4 - Application Initialisation >>> ====================================== >>> >>> User can not do any setup for web based now. >>> >>> >>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>> not exist. >>> Entering initial setup mode... >>> NOTE: Configuring authentication for SERVER mode. >>> >>> >>> Enter the email address and password to use for the initial pgAdmin >>> user account: >>> >>> Email address: [email protected] >>> Password: >>> Retype password: >>> Traceback (most recent call last): >>> File "setup.py", line 449, in <module> >>> do_setup(app) >>> File "setup.py", line 96, in do_setup >>> password = encrypt_password(p1) >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 150, in encrypt_password >>> signed = get_hmac(password).decode('ascii') >>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>> line 108, in get_hmac >>> 'set to "%s"' % _security.password_hash) >>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>> >>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Dave, >>>> >>>> Testing Environment >>>> >>>> Ubuntu 16.04 Linux 64: >>>> -------------------------------- >>>> >>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>> >>>> >>>> 1) Install GIT >>>> >>>> sudo apt-get install git >>>> >>>> 2) Install pip3 >>>> >>>> sudo apt-get install python3-pip >>>> >>>> 3) Install virtualenv >>>> >>>> sudo pip3 install virtualenv >>>> >>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>> module >>>> >>>> sudo apt-get install libpq-dev >>>> >>>> sudo apt-get install python3-dev >>>> >>>> 5) Create virtual environment >>>> >>>> virtualenv -p python3 venv >>>> >>>> 6) Create mkdir Projects >>>> >>>> 7) Clone git repo in Projects >>>> >>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>> >>>> 8) activate virtual environment >>>> >>>> source venv/bin/activate >>>> >>>> 9) Install modules >>>> >>>> pip3 install -r requirements_py3.txt >>>> >>>> *10) Edit the config.py file to config_local.py resides in >>>> Projects\pgAdmin4\web * >>>> >>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>> python setup.py >>>> >>>> If user does not create config_local.py and do Python setup.py for new >>>> Development then SECURITY_PASSWORD_SALT message is also displayed: >>>> >>>> Here is the output: >>>> ------------------------- >>>> >>>> python setup.py >>>> pgAdmin 4 - Application Initialisation >>>> ====================================== >>>> >>>> >>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>> not exist. >>>> Entering initial setup mode... >>>> NOTE: Configuring authentication for SERVER mode. >>>> >>>> >>>> Enter the email address and password to use for the initial pgAdmin >>>> user account: >>>> >>>> Email address: [email protected] >>>> Password: >>>> Retype password: >>>> Traceback (most recent call last): >>>> File "setup.py", line 449, in <module> >>>> do_setup(app) >>>> File "setup.py", line 96, in do_setup >>>> password = encrypt_password(p1) >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 150, in encrypt_password >>>> signed = get_hmac(password).decode('ascii') >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 108, in get_hmac >>>> 'set to "%s"' % _security.password_hash) >>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>> >>>> >>>> Is this expected? >>>> >>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Sure, >>>>> >>>>> Will test this thoroughly after complete investigation. >>>>> >>>>> Kind Regards, >>>>> >>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >>>>> >>>>>> Patch applied. >>>>>> >>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>> modes, with both fresh and upgraded installations? >>>>>> >>>>>> https://redmine.postgresql.org/issues/1849 >>>>>> >>>>>> Packagers: This change means that packages are no longer forced to >>>>>> create a config_local.py file, and there is no longer any need to >>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>> installations, if you have included them in 1.0) >>>>>> >>>>>> Thanks. >>>>>> >>>>>> >>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Dave, >>>>>>> >>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> >>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Dave, >>>>>>>>>> >>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Ashesh, >>>>>>>>>>> >>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>> happy with it? >>>>>>>>>>> >>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>>>>> 'runtime'. >>>>>>>>>> >>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>> 'web' mode. >>>>>>>>>> - Apply the patch >>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>> - Log out (if already login). >>>>>>>>>> >>>>>>>>>> And, you will see an exception. >>>>>>>>>> >>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>> the Security object. >>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hmm. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>> these configurations from the database. >>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK, thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Now - I run into another issue. >>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>> >>>>>>>>>> I think - we need to think about different strategy for upgrading >>>>>>>>>> the configuration file in the 'web' mode. >>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>> >>>>>>>>> >>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>> >>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>> Sigh... Needs more thought. >>>>>>>>> >>>>>>>> >>>>>>>> OK, so I've been thinking about this and experimenting for a couple >>>>>>>> of hours, as well as annoying the crap out of Magnus by thinking out loud >>>>>>>> in his general direction, and it looks like this isn't a major problem as >>>>>>>> from what I can see, SECURITY_PASSWORD_SALT is (aside from really being a >>>>>>>> key not a salt) not the only salting that's done. >>>>>>>> >>>>>>>> It looks like it's used system-wide as the key to generate an HMAC >>>>>>>> of the users password, which is then passed to passlib which salts and >>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>> installations/databases. >>>>>>>> >>>>>>>> So, I believe we can do as you suggest and migrate existing values >>>>>>>> for SECURITY_PASSWORD_SALT, given that there's clearly some other per user >>>>>>>> and per installation/database salting going on anyway. New installations >>>>>>>> can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>> >>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>>>>> upgrade mode, which was wrong added in my addon patch. >>>>>>> >>>>>>> Please find the updated patch. >>>>>>> >>>>>>> Otherwise - looks good to me. >>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> >>>>>>> Ashesh Vashi >>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>> <http://www.enterprisedb.com/; >>>>>>> >>>>>>> >>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>> >>>>>>>> >>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>> thus can be changed during an upgrade. >>>>>>>> >>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>> >>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>> appreciated)! >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 05:45 Murtuza Zabuawala <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 2 replies; 33+ messages in thread From: Murtuza Zabuawala @ 2016-10-20 05:45 UTC (permalink / raw) To: Fahar Abbas <[email protected]>; +Cc: Dave Page <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Hi, PFA patch to fix the issue for Pyhton3. RM#1849 -- Regards, Murtuza Zabuawala EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas <[email protected]> wrote: > Hi Dave, > > I have reopened following RM: > ================================ > https://redmine.postgresql.org/issues/1849 > > On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: > >> Patch applied. >> >> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >> [email protected]> wrote: >> >>> Hi Fahar, >>> >>> Please log the case on redmine. >>> Please find the attached patch, please apply it locally, and test it. >>> >>> And, please update the case, and this mail chain accordingly. >>> >>> -- >>> >>> Thanks & Regards, >>> >>> Ashesh Vashi >>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>> <http://www.enterprisedb.com; >>> >>> >>> *http://www.linkedin.com/in/asheshvashi* >>> <http://www.linkedin.com/in/asheshvashi; >>> >>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Here is the output of if we copy config_local.py and execute python >>>> setup.py >>>> pgAdmin 4 - Application Initialisation >>>> ====================================== >>>> >>>> >>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>> not exist. >>>> Entering initial setup mode... >>>> NOTE: Configuring authentication for SERVER mode. >>>> >>>> >>>> Enter the email address and password to use for the initial pgAdmin >>>> user account: >>>> >>>> Email address: [email protected] >>>> Password: >>>> Retype password: >>>> Traceback (most recent call last): >>>> File "setup.py", line 449, in <module> >>>> do_setup(app) >>>> File "setup.py", line 96, in do_setup >>>> password = encrypt_password(p1) >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 150, in encrypt_password >>>> signed = get_hmac(password).decode('ascii') >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 108, in get_hmac >>>> 'set to "%s"' % _security.password_hash) >>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>> python setup.py >>>> pgAdmin 4 - Application Initialisation >>>> ====================================== >>>> >>>> User can not do any setup for web based now. >>>> >>>> >>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>> not exist. >>>> Entering initial setup mode... >>>> NOTE: Configuring authentication for SERVER mode. >>>> >>>> >>>> Enter the email address and password to use for the initial pgAdmin >>>> user account: >>>> >>>> Email address: [email protected] >>>> Password: >>>> Retype password: >>>> Traceback (most recent call last): >>>> File "setup.py", line 449, in <module> >>>> do_setup(app) >>>> File "setup.py", line 96, in do_setup >>>> password = encrypt_password(p1) >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 150, in encrypt_password >>>> signed = get_hmac(password).decode('ascii') >>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>> line 108, in get_hmac >>>> 'set to "%s"' % _security.password_hash) >>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not >>>> be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>> >>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Dave, >>>>> >>>>> Testing Environment >>>>> >>>>> Ubuntu 16.04 Linux 64: >>>>> -------------------------------- >>>>> >>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>> >>>>> >>>>> 1) Install GIT >>>>> >>>>> sudo apt-get install git >>>>> >>>>> 2) Install pip3 >>>>> >>>>> sudo apt-get install python3-pip >>>>> >>>>> 3) Install virtualenv >>>>> >>>>> sudo pip3 install virtualenv >>>>> >>>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>>> module >>>>> >>>>> sudo apt-get install libpq-dev >>>>> >>>>> sudo apt-get install python3-dev >>>>> >>>>> 5) Create virtual environment >>>>> >>>>> virtualenv -p python3 venv >>>>> >>>>> 6) Create mkdir Projects >>>>> >>>>> 7) Clone git repo in Projects >>>>> >>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>> >>>>> 8) activate virtual environment >>>>> >>>>> source venv/bin/activate >>>>> >>>>> 9) Install modules >>>>> >>>>> pip3 install -r requirements_py3.txt >>>>> >>>>> *10) Edit the config.py file to config_local.py resides in >>>>> Projects\pgAdmin4\web * >>>>> >>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>> python setup.py >>>>> >>>>> If user does not create config_local.py and do Python setup.py for new >>>>> Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>> >>>>> Here is the output: >>>>> ------------------------- >>>>> >>>>> python setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>> >>>>> >>>>> Is this expected? >>>>> >>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Sure, >>>>>> >>>>>> Will test this thoroughly after complete investigation. >>>>>> >>>>>> Kind Regards, >>>>>> >>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >>>>>> >>>>>>> Patch applied. >>>>>>> >>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>> modes, with both fresh and upgraded installations? >>>>>>> >>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>> >>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>> installations, if you have included them in 1.0) >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Dave, >>>>>>>> >>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> >>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Dave, >>>>>>>>>>> >>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>> >>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>> happy with it? >>>>>>>>>>>> >>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with >>>>>>>>>>> 'runtime'. >>>>>>>>>>> >>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>> 'web' mode. >>>>>>>>>>> - Apply the patch >>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>> - Log out (if already login). >>>>>>>>>>> >>>>>>>>>>> And, you will see an exception. >>>>>>>>>>> >>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>> the Security object. >>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hmm. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>> these configurations from the database. >>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>> >>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>>> >>>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>>> Sigh... Needs more thought. >>>>>>>>>> >>>>>>>>> >>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>> >>>>>>>>> It looks like it's used system-wide as the key to generate an HMAC >>>>>>>>> of the users password, which is then passed to passlib which salts and >>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>> installations/databases. >>>>>>>>> >>>>>>>>> So, I believe we can do as you suggest and migrate existing values >>>>>>>>> for SECURITY_PASSWORD_SALT, given that there's clearly some other per user >>>>>>>>> and per installation/database salting going on anyway. New installations >>>>>>>>> can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>> >>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during >>>>>>>> upgrade mode, which was wrong added in my addon patch. >>>>>>>> >>>>>>>> Please find the updated patch. >>>>>>>> >>>>>>>> Otherwise - looks good to me. >>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks & Regards, >>>>>>>> >>>>>>>> Ashesh Vashi >>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>> <http://www.enterprisedb.com/; >>>>>>>> >>>>>>>> >>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>> >>>>>>>>> >>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>> thus can be changed during an upgrade. >>>>>>>>> >>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>> >>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>> appreciated)! >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dave Page >>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>> Twitter: @pgsnake >>>>>>>>> >>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Dave Page >>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>> Twitter: @pgsnake >>>>>>> >>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> The Enterprise PostgreSQL Company >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/octet-stream] python3_compatible.patch (1.6K, 3-python3_compatible.patch) download | inline diff: diff --git a/web/setup.py b/web/setup.py index c3d3b36..80b11fe 100755 --- a/web/setup.py +++ b/web/setup.py @@ -350,17 +350,20 @@ CREATE TABLE keys ( PRIMARY KEY (name)) """) - sql = "INSERT INTO keys (name, value) VALUES ('CSRF_SESSION_KEY', '%s')" % base64.urlsafe_b64encode(os.urandom(32)) + sql = "INSERT INTO keys (name, value) VALUES ('CSRF_SESSION_KEY', '%s')" \ + % base64.urlsafe_b64encode(os.urandom(32)).decode() db.engine.execute(sql) - sql = "INSERT INTO keys (name, value) VALUES ('SECRET_KEY', '%s')" % base64.urlsafe_b64encode(os.urandom(32)) + sql = "INSERT INTO keys (name, value) VALUES ('SECRET_KEY', '%s')" % \ + base64.urlsafe_b64encode(os.urandom(32)).decode() db.engine.execute(sql) # If SECURITY_PASSWORD_SALT is not in the config, but we're upgrading, then it must (unless the # user edited the main config - which they shouldn't have done) have been at it's default # value, so we'll use that. Otherwise, use whatever we can find in the config. if hasattr(config, 'SECURITY_PASSWORD_SALT'): - sql = "INSERT INTO keys (name, value) VALUES ('SECURITY_PASSWORD_SALT', '%s')" % config.SECURITY_PASSWORD_SALT + sql = "INSERT INTO keys (name, value) VALUES ('SECURITY_PASSWORD_SALT', '%s')" \ + % config.SECURITY_PASSWORD_SALT else: sql = "INSERT INTO keys (name, value) VALUES ('SECURITY_PASSWORD_SALT', 'SuperSecret3')" db.engine.execute(sql) ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 05:56 Fahar Abbas <[email protected]> parent: Murtuza Zabuawala <[email protected]> 1 sibling, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-20 05:56 UTC (permalink / raw) To: Murtuza Zabuawala <[email protected]>; +Cc: Dave Page <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Murtaza, I have applied this patch and there is no success on new pgAdmin4 setup as well as existing pgAdmin4 setup. On Thu, Oct 20, 2016 at 10:45 AM, Murtuza Zabuawala < [email protected]> wrote: > Hi, > > PFA patch to fix the issue for Pyhton3. > RM#1849 > > -- > Regards, > Murtuza Zabuawala > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < > [email protected]> wrote: > >> Hi Dave, >> >> I have reopened following RM: >> ================================ >> https://redmine.postgresql.org/issues/1849 >> >> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >> >>> Patch applied. >>> >>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>> [email protected]> wrote: >>> >>>> Hi Fahar, >>>> >>>> Please log the case on redmine. >>>> Please find the attached patch, please apply it locally, and test it. >>>> >>>> And, please update the case, and this mail chain accordingly. >>>> >>>> -- >>>> >>>> Thanks & Regards, >>>> >>>> Ashesh Vashi >>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>> <http://www.enterprisedb.com; >>>> >>>> >>>> *http://www.linkedin.com/in/asheshvashi* >>>> <http://www.linkedin.com/in/asheshvashi; >>>> >>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Here is the output of if we copy config_local.py and execute python >>>>> setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> python setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> User can not do any setup for web based now. >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> >>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Dave, >>>>>> >>>>>> Testing Environment >>>>>> >>>>>> Ubuntu 16.04 Linux 64: >>>>>> -------------------------------- >>>>>> >>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>> >>>>>> >>>>>> 1) Install GIT >>>>>> >>>>>> sudo apt-get install git >>>>>> >>>>>> 2) Install pip3 >>>>>> >>>>>> sudo apt-get install python3-pip >>>>>> >>>>>> 3) Install virtualenv >>>>>> >>>>>> sudo pip3 install virtualenv >>>>>> >>>>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>>>> module >>>>>> >>>>>> sudo apt-get install libpq-dev >>>>>> >>>>>> sudo apt-get install python3-dev >>>>>> >>>>>> 5) Create virtual environment >>>>>> >>>>>> virtualenv -p python3 venv >>>>>> >>>>>> 6) Create mkdir Projects >>>>>> >>>>>> 7) Clone git repo in Projects >>>>>> >>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>> >>>>>> 8) activate virtual environment >>>>>> >>>>>> source venv/bin/activate >>>>>> >>>>>> 9) Install modules >>>>>> >>>>>> pip3 install -r requirements_py3.txt >>>>>> >>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>> Projects\pgAdmin4\web * >>>>>> >>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>> python setup.py >>>>>> >>>>>> If user does not create config_local.py and do Python setup.py for >>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>> >>>>>> Here is the output: >>>>>> ------------------------- >>>>>> >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>> >>>>>> >>>>>> Is this expected? >>>>>> >>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Sure, >>>>>>> >>>>>>> Will test this thoroughly after complete investigation. >>>>>>> >>>>>>> Kind Regards, >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Patch applied. >>>>>>>> >>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>> >>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>> >>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>> installations, if you have included them in 1.0) >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Dave, >>>>>>>>> >>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave, >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>> >>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>> happy with it? >>>>>>>>>>>>> >>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>> >>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>> 'web' mode. >>>>>>>>>>>> - Apply the patch >>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>> >>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>> >>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>> the Security object. >>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hmm. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>> >>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>>>> >>>>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>>>> Sigh... Needs more thought. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>> >>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>> installations/databases. >>>>>>>>>> >>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>> >>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>> >>>>>>>>> Please find the updated patch. >>>>>>>>> >>>>>>>>> Otherwise - looks good to me. >>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks & Regards, >>>>>>>>> >>>>>>>>> Ashesh Vashi >>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>> >>>>>>>>> >>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>> >>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>> >>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>> appreciated)! >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 06:00 Murtuza Zabuawala <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 3 replies; 33+ messages in thread From: Murtuza Zabuawala @ 2016-10-20 06:00 UTC (permalink / raw) To: Fahar Abbas <[email protected]>; +Cc: Dave Page <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Could you delete 'keys' table from pgadmin4.db file & try again? -- Regards, Murtuza Zabuawala EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Thu, Oct 20, 2016 at 11:26 AM, Fahar Abbas <[email protected]> wrote: > Murtaza, > > I have applied this patch and there is no success on new pgAdmin4 setup as > well as existing pgAdmin4 setup. > > On Thu, Oct 20, 2016 at 10:45 AM, Murtuza Zabuawala <murtuza.zabuawala@ > enterprisedb.com> wrote: > >> Hi, >> >> PFA patch to fix the issue for Pyhton3. >> RM#1849 >> >> -- >> Regards, >> Murtuza Zabuawala >> EnterpriseDB: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >> [email protected]> wrote: >> >>> Hi Dave, >>> >>> I have reopened following RM: >>> ================================ >>> https://redmine.postgresql.org/issues/1849 >>> >>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >>> >>>> Patch applied. >>>> >>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>> [email protected]> wrote: >>>> >>>>> Hi Fahar, >>>>> >>>>> Please log the case on redmine. >>>>> Please find the attached patch, please apply it locally, and test it. >>>>> >>>>> And, please update the case, and this mail chain accordingly. >>>>> >>>>> -- >>>>> >>>>> Thanks & Regards, >>>>> >>>>> Ashesh Vashi >>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>> <http://www.enterprisedb.com; >>>>> >>>>> >>>>> *http://www.linkedin.com/in/asheshvashi* >>>>> <http://www.linkedin.com/in/asheshvashi; >>>>> >>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Here is the output of if we copy config_local.py and execute python >>>>>> setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> User can not do any setup for web based now. >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> >>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Dave, >>>>>>> >>>>>>> Testing Environment >>>>>>> >>>>>>> Ubuntu 16.04 Linux 64: >>>>>>> -------------------------------- >>>>>>> >>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>> >>>>>>> >>>>>>> 1) Install GIT >>>>>>> >>>>>>> sudo apt-get install git >>>>>>> >>>>>>> 2) Install pip3 >>>>>>> >>>>>>> sudo apt-get install python3-pip >>>>>>> >>>>>>> 3) Install virtualenv >>>>>>> >>>>>>> sudo pip3 install virtualenv >>>>>>> >>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>> pycrypto module >>>>>>> >>>>>>> sudo apt-get install libpq-dev >>>>>>> >>>>>>> sudo apt-get install python3-dev >>>>>>> >>>>>>> 5) Create virtual environment >>>>>>> >>>>>>> virtualenv -p python3 venv >>>>>>> >>>>>>> 6) Create mkdir Projects >>>>>>> >>>>>>> 7) Clone git repo in Projects >>>>>>> >>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>> >>>>>>> 8) activate virtual environment >>>>>>> >>>>>>> source venv/bin/activate >>>>>>> >>>>>>> 9) Install modules >>>>>>> >>>>>>> pip3 install -r requirements_py3.txt >>>>>>> >>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>> Projects\pgAdmin4\web * >>>>>>> >>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>> python setup.py >>>>>>> >>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>> >>>>>>> Here is the output: >>>>>>> ------------------------- >>>>>>> >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>> >>>>>>> >>>>>>> Is this expected? >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Sure, >>>>>>>> >>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>> >>>>>>>> Kind Regards, >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Patch applied. >>>>>>>>> >>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>> >>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>> >>>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Dave, >>>>>>>>>> >>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>>> happy with it? >>>>>>>>>>>>>> >>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>> >>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>>> 'web' mode. >>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>> >>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>> >>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>>> the Security object. >>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hmm. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>> >>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>> problem. >>>>>>>>>>>> >>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>> >>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>> installations/databases. >>>>>>>>>>> >>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>> >>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>> >>>>>>>>>> Please find the updated patch. >>>>>>>>>> >>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks & Regards, >>>>>>>>>> >>>>>>>>>> Ashesh Vashi >>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>> >>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>> >>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>>> appreciated)! >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dave Page >>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>> Twitter: @pgsnake >>>>>>>>> >>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 06:38 Fahar Abbas <[email protected]> parent: Murtuza Zabuawala <[email protected]> 2 siblings, 0 replies; 33+ messages in thread From: Fahar Abbas @ 2016-10-20 06:38 UTC (permalink / raw) To: Murtuza Zabuawala <[email protected]>; +Cc: Dave Page <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> I tried different variations and when launch pgAdmin4 with web browser still exception displayed when user deleted .pgadmin folder Here is the output: ------------------------ python pgAdmin4.py Starting pgAdmin 4. Please navigate to http://localhost:5050 in your browser. Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/usr/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.5/socketserver.py", line 628, in process_request_thread self.handle_error(request, client_address) File "/usr/lib/python3.5/socketserver.py", line 625, in process_request_thread self.finish_request(request, client_address) File "/usr/lib/python3.5/socketserver.py", line 354, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python3.5/socketserver.py", line 681, in __init__ self.handle() File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", line 200, in handle rv = BaseHTTPRequestHandler.handle(self) File "/usr/lib/python3.5/http/server.py", line 415, in handle self.handle_one_request() File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", line 235, in handle_one_request return self.run_wsgi() File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", line 177, in run_wsgi execute(self.server.app) File "/home/fahar/venv/lib/python3.5/site-packages/werkzeug/serving.py", line 165, in execute application_iter = app(environ, start_response) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 2000, in __call__ return self.wsgi_app(environ, start_response) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1991, in wsgi_app response = self.make_response(self.handle_exception(e)) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1567, in handle_exception reraise(exc_type, exc_value, tb) File "/home/fahar/venv/lib/python3.5/site-packages/flask/_compat.py", line 33, in reraise raise value File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1988, in wsgi_app response = self.full_dispatch_request() File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1643, in full_dispatch_request response = self.process_response(response) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 1864, in process_response self.save_session(ctx.session, response) File "/home/fahar/venv/lib/python3.5/site-packages/flask/app.py", line 926, in save_session return self.session_interface.save_session(self, session, response) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 267, in save_session self.manager.put(session) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 144, in put self.parent.put(session) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 214, in put session.sign(self.secret) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 71, in sign self.hmac_digest = _calc_hmac('%s:%s' % (self.sid, self.randval), secret) File "/home/fahar/Projects/pgadmin4/web/pgadmin/utils/session.py", line 44, in _calc_hmac secret.encode(), body.encode(), hashlib.sha1 AttributeError: 'bytes' object has no attribute 'encode' On Thu, Oct 20, 2016 at 11:00 AM, Murtuza Zabuawala < [email protected]> wrote: > Could you delete 'keys' table from pgadmin4.db file & try again? > > -- > Regards, > Murtuza Zabuawala > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Thu, Oct 20, 2016 at 11:26 AM, Fahar Abbas < > [email protected]> wrote: > >> Murtaza, >> >> I have applied this patch and there is no success on new pgAdmin4 setup >> as well as existing pgAdmin4 setup. >> >> On Thu, Oct 20, 2016 at 10:45 AM, Murtuza Zabuawala < >> [email protected]> wrote: >> >>> Hi, >>> >>> PFA patch to fix the issue for Pyhton3. >>> RM#1849 >>> >>> -- >>> Regards, >>> Murtuza Zabuawala >>> EnterpriseDB: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Hi Dave, >>>> >>>> I have reopened following RM: >>>> ================================ >>>> https://redmine.postgresql.org/issues/1849 >>>> >>>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >>>> >>>>> Patch applied. >>>>> >>>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Fahar, >>>>>> >>>>>> Please log the case on redmine. >>>>>> Please find the attached patch, please apply it locally, and test it. >>>>>> >>>>>> And, please update the case, and this mail chain accordingly. >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & Regards, >>>>>> >>>>>> Ashesh Vashi >>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>> <http://www.enterprisedb.com; >>>>>> >>>>>> >>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>> >>>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Here is the output of if we copy config_local.py and execute python >>>>>>> setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> User can not do any setup for web based now. >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Dave, >>>>>>>> >>>>>>>> Testing Environment >>>>>>>> >>>>>>>> Ubuntu 16.04 Linux 64: >>>>>>>> -------------------------------- >>>>>>>> >>>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>>> >>>>>>>> >>>>>>>> 1) Install GIT >>>>>>>> >>>>>>>> sudo apt-get install git >>>>>>>> >>>>>>>> 2) Install pip3 >>>>>>>> >>>>>>>> sudo apt-get install python3-pip >>>>>>>> >>>>>>>> 3) Install virtualenv >>>>>>>> >>>>>>>> sudo pip3 install virtualenv >>>>>>>> >>>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>>> pycrypto module >>>>>>>> >>>>>>>> sudo apt-get install libpq-dev >>>>>>>> >>>>>>>> sudo apt-get install python3-dev >>>>>>>> >>>>>>>> 5) Create virtual environment >>>>>>>> >>>>>>>> virtualenv -p python3 venv >>>>>>>> >>>>>>>> 6) Create mkdir Projects >>>>>>>> >>>>>>>> 7) Clone git repo in Projects >>>>>>>> >>>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>>> >>>>>>>> 8) activate virtual environment >>>>>>>> >>>>>>>> source venv/bin/activate >>>>>>>> >>>>>>>> 9) Install modules >>>>>>>> >>>>>>>> pip3 install -r requirements_py3.txt >>>>>>>> >>>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>>> Projects\pgAdmin4\web * >>>>>>>> >>>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>>> python setup.py >>>>>>>> >>>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>>> >>>>>>>> Here is the output: >>>>>>>> ------------------------- >>>>>>>> >>>>>>>> python setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: [email protected] >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in <module> >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>>> >>>>>>>> >>>>>>>> Is this expected? >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Sure, >>>>>>>>> >>>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>>> >>>>>>>>> Kind Regards, >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Patch applied. >>>>>>>>>> >>>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>>> >>>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>>> >>>>>>>>>> Packagers: This change means that packages are no longer forced >>>>>>>>>> to create a config_local.py file, and there is no longer any need to >>>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Dave, >>>>>>>>>>> >>>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected] >>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can you please review the attached patch, and apply if >>>>>>>>>>>>>>> you're happy with it? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>>>> 'web' mode. >>>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>>> >>>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after >>>>>>>>>>>>>> initializing the Security object. >>>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hmm. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> OK, thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>>> problem. >>>>>>>>>>>>> >>>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>>> >>>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>>> installations/databases. >>>>>>>>>>>> >>>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>>> >>>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>>> >>>>>>>>>>> Please find the updated patch. >>>>>>>>>>> >>>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> Thanks & Regards, >>>>>>>>>>> >>>>>>>>>>> Ashesh Vashi >>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>>> >>>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>>> >>>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>>>> appreciated)! >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Dave Page >>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>> >>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Syed Fahar Abbas >>>>>>>>> Quality Management Group >>>>>>>>> >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>> Website: www.enterprisedb.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 06:45 Dave Page <[email protected]> parent: Murtuza Zabuawala <[email protected]> 2 siblings, 0 replies; 33+ messages in thread From: Dave Page @ 2016-10-20 06:45 UTC (permalink / raw) To: Murtuza Zabuawala <[email protected]>; +Cc: Fahar Abbas <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> I think you'll also need to set the version back to 13 in the version table. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK:http://www.enterprisedb.com The Enterprise PostgreSQL Company > On 20 Oct 2016, at 07:00, Murtuza Zabuawala <[email protected]> wrote: > > Could you delete 'keys' table from pgadmin4.db file & try again? > > -- > Regards, > Murtuza Zabuawala > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > >> On Thu, Oct 20, 2016 at 11:26 AM, Fahar Abbas <[email protected]> wrote: >> Murtaza, >> >> I have applied this patch and there is no success on new pgAdmin4 setup as well as existing pgAdmin4 setup. >> >>> On Thu, Oct 20, 2016 at 10:45 AM, Murtuza Zabuawala <[email protected]> wrote: >>> Hi, >>> >>> PFA patch to fix the issue for Pyhton3. >>> RM#1849 >>> >>> -- >>> Regards, >>> Murtuza Zabuawala >>> EnterpriseDB: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>>> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas <[email protected]> wrote: >>>> Hi Dave, >>>> >>>> I have reopened following RM: >>>> ================================ >>>> https://redmine.postgresql.org/issues/1849 >>>> >>>>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >>>>> Patch applied. >>>>> >>>>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi <[email protected]> wrote: >>>>>> Hi Fahar, >>>>>> >>>>>> Please log the case on redmine. >>>>>> Please find the attached patch, please apply it locally, and test it. >>>>>> >>>>>> And, please update the case, and this mail chain accordingly. >>>>>> >>>>>> -- >>>>>> Thanks & Regards, >>>>>> >>>>>> Ashesh Vashi >>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>> >>>>>> http://www.linkedin.com/in/asheshvashi >>>>>> >>>>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas <[email protected]> wrote: >>>>>>> Here is the output of if we copy config_local.py and execute python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> User can not do any setup for web based now. >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas <[email protected]> wrote: >>>>>>>> Dave, >>>>>>>> >>>>>>>> Testing Environment >>>>>>>> >>>>>>>> Ubuntu 16.04 Linux 64: >>>>>>>> -------------------------------- >>>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>>> >>>>>>>> 1) Install GIT >>>>>>>> sudo apt-get install git >>>>>>>> 2) Install pip3 >>>>>>>> sudo apt-get install python3-pip >>>>>>>> 3) Install virtualenv >>>>>>>> sudo pip3 install virtualenv >>>>>>>> 4) install below dependency as it is required for psycopg2 & pycrypto module >>>>>>>> sudo apt-get install libpq-dev >>>>>>>> sudo apt-get install python3-dev >>>>>>>> 5) Create virtual environment >>>>>>>> virtualenv -p python3 venv >>>>>>>> 6) Create mkdir Projects >>>>>>>> 7) Clone git repo in Projects >>>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>>> 8) activate virtual environment >>>>>>>> source venv/bin/activate >>>>>>>> 9) Install modules >>>>>>>> pip3 install -r requirements_py3.txt >>>>>>>> 10) Edit the config.py file to config_local.py resides in Projects\pgAdmin4\web >>>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>>> python setup.py >>>>>>>> >>>>>>>> If user does not create config_local.py and do Python setup.py for new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>>> >>>>>>>> Here is the output: >>>>>>>> ------------------------- >>>>>>>> >>>>>>>> python setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial pgAdmin user account: >>>>>>>> >>>>>>>> Email address: [email protected] >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in <module> >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must not be None when the value of `SECURITY_PASSWORD_HASH` is set to "pbkdf2_sha512" >>>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>>> >>>>>>>> >>>>>>>> Is this expected? >>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas <[email protected]> wrote: >>>>>>>>> Sure, >>>>>>>>> >>>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>>> >>>>>>>>> Kind Regards, >>>>>>>>> >>>>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> wrote: >>>>>>>>>> Patch applied. >>>>>>>>>> >>>>>>>>>> Fahar, can you please test this thoroughly in desktop and server modes, with both fresh and upgraded installations? >>>>>>>>>> >>>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>>> >>>>>>>>>> Packagers: This change means that packages are no longer forced to create a config_local.py file, and there is no longer any need to explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY and CSRF_SESSION_KEY in the config (in fact, they should be removed for new installations, if you have included them in 1.0) >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi <[email protected]> wrote: >>>>>>>>>>> Hi Dave, >>>>>>>>>>> >>>>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> wrote: >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi <[email protected]> wrote: >>>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> wrote: >>>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can you please review the attached patch, and apply if you're happy with it? >>>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen with 'runtime'. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with 'web' mode. >>>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>>> >>>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing the Security object. >>>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>> >>>>>>>>>>>>> Hmm. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I had moved the Security object initialization after fetching these configurations from the database. >>>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>> >>>>>>>>>>>>> OK, thanks. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>>> Because - the existing password was hashed using the old SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think - we need to think about different strategy for upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>>> I was thinking - we can store the existing security configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>> >>>>>>>>>>>>> My concern with that is that we'll likely be storing the default config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>>>>>> >>>>>>>>>>>>> I guess what we need to do is re-encrypt the password during the upgrade - however, that makes me think; we then have both the key and the encrypted passwords in the same database which is clearly not a good idea. Sigh... Needs more thought. >>>>>>>>>>>> >>>>>>>>>>>> OK, so I've been thinking about this and experimenting for a couple of hours, as well as annoying the crap out of Magnus by thinking out loud in his general direction, and it looks like this isn't a major problem as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being a key not a salt) not the only salting that's done. >>>>>>>>>>>> >>>>>>>>>>>> It looks like it's used system-wide as the key to generate an HMAC of the users password, which is then passed to passlib which salts and hashes it. I did some testing, and found that two users with the same password end up with different hashes in the database, so clearly there is also per-user salting happening. I also created two users, then dropped the database and created the same user accounts with the same passwords again, and found that the resulting hashes were different in both databases - thus there is something else ensuring the hashes are unique across different installations/databases. >>>>>>>>>>>> >>>>>>>>>>>> So, I believe we can do as you suggest and migrate existing values for SECURITY_PASSWORD_SALT, given that there's clearly some other per user and per installation/database salting going on anyway. New installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>> >>>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>>> >>>>>>>>>>> Please find the updated patch. >>>>>>>>>>> >>>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Thanks & Regards, >>>>>>>>>>> >>>>>>>>>>> Ashesh Vashi >>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>>> http://www.linkedin.com/in/asheshvashi >>>>>>>>>>>> >>>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues either, as they're used for purposes that are essentially ephemeral, and thus can be changed during an upgrade. >>>>>>>>>>>> >>>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>>> >>>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be appreciated)! >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Dave Page >>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>> >>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Syed Fahar Abbas >>>>>>>>> Quality Management Group >>>>>>>>> >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com > ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 07:30 Fahar Abbas <[email protected]> parent: Murtuza Zabuawala <[email protected]> 2 siblings, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-20 07:30 UTC (permalink / raw) To: Murtuza Zabuawala <[email protected]>; +Cc: Dave Page <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Murtaza, Once i delete key table from pgAdmin4.db file then i can launch pgAdmin4 with terminal as well as web browser for existing pgAdmin4 setup. In case of fresh setup once we delete key table and apply latest patch following message displayed: python pgAdmin4.py Traceback (most recent call last): File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context context) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute cursor.execute(statement, parameters) sqlite3.OperationalError: no such table: keys The above exception was the direct cause of the following exception: Traceback (most recent call last): File "pgAdmin4.py", line 46, in <module> app = create_app() File "/home/fahar/Projects/pgadmin4/web/pgadmin/__init__.py", line 224, in create_app config.CSRF_SESSION_KEY = Keys.query.filter_by(name = 'CSRF_SESSION_KEY').first().value File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2659, in first ret = list(self[0:1]) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2457, in __getitem__ return list(res) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2761, in __iter__ return self._execute_and_instances(context) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2776, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 914, in execute return meth(self, multiparams, params) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement compiled_sql, distilled_params File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context context) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception exc_info File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 185, in reraise raise value.with_traceback(tb) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context context) File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: keys [SQL: 'SELECT keys.name AS keys_name, keys.value AS keys_value \nFROM keys \nWHERE keys.name = ?\n LIMIT ? OFFSET ?'] [parameters: ('CSRF_SESSION_KEY', 1, 0)] On Thu, Oct 20, 2016 at 11:00 AM, Murtuza Zabuawala < [email protected]> wrote: > Could you delete 'keys' table from pgadmin4.db file & try again? > > -- > Regards, > Murtuza Zabuawala > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Thu, Oct 20, 2016 at 11:26 AM, Fahar Abbas < > [email protected]> wrote: > >> Murtaza, >> >> I have applied this patch and there is no success on new pgAdmin4 setup >> as well as existing pgAdmin4 setup. >> >> On Thu, Oct 20, 2016 at 10:45 AM, Murtuza Zabuawala < >> [email protected]> wrote: >> >>> Hi, >>> >>> PFA patch to fix the issue for Pyhton3. >>> RM#1849 >>> >>> -- >>> Regards, >>> Murtuza Zabuawala >>> EnterpriseDB: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Hi Dave, >>>> >>>> I have reopened following RM: >>>> ================================ >>>> https://redmine.postgresql.org/issues/1849 >>>> >>>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >>>> >>>>> Patch applied. >>>>> >>>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Fahar, >>>>>> >>>>>> Please log the case on redmine. >>>>>> Please find the attached patch, please apply it locally, and test it. >>>>>> >>>>>> And, please update the case, and this mail chain accordingly. >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & Regards, >>>>>> >>>>>> Ashesh Vashi >>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>> <http://www.enterprisedb.com; >>>>>> >>>>>> >>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>> >>>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Here is the output of if we copy config_local.py and execute python >>>>>>> setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> User can not do any setup for web based now. >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Dave, >>>>>>>> >>>>>>>> Testing Environment >>>>>>>> >>>>>>>> Ubuntu 16.04 Linux 64: >>>>>>>> -------------------------------- >>>>>>>> >>>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>>> >>>>>>>> >>>>>>>> 1) Install GIT >>>>>>>> >>>>>>>> sudo apt-get install git >>>>>>>> >>>>>>>> 2) Install pip3 >>>>>>>> >>>>>>>> sudo apt-get install python3-pip >>>>>>>> >>>>>>>> 3) Install virtualenv >>>>>>>> >>>>>>>> sudo pip3 install virtualenv >>>>>>>> >>>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>>> pycrypto module >>>>>>>> >>>>>>>> sudo apt-get install libpq-dev >>>>>>>> >>>>>>>> sudo apt-get install python3-dev >>>>>>>> >>>>>>>> 5) Create virtual environment >>>>>>>> >>>>>>>> virtualenv -p python3 venv >>>>>>>> >>>>>>>> 6) Create mkdir Projects >>>>>>>> >>>>>>>> 7) Clone git repo in Projects >>>>>>>> >>>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>>> >>>>>>>> 8) activate virtual environment >>>>>>>> >>>>>>>> source venv/bin/activate >>>>>>>> >>>>>>>> 9) Install modules >>>>>>>> >>>>>>>> pip3 install -r requirements_py3.txt >>>>>>>> >>>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>>> Projects\pgAdmin4\web * >>>>>>>> >>>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>>> python setup.py >>>>>>>> >>>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>>> >>>>>>>> Here is the output: >>>>>>>> ------------------------- >>>>>>>> >>>>>>>> python setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: [email protected] >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in <module> >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>>> >>>>>>>> >>>>>>>> Is this expected? >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Sure, >>>>>>>>> >>>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>>> >>>>>>>>> Kind Regards, >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Patch applied. >>>>>>>>>> >>>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>>> >>>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>>> >>>>>>>>>> Packagers: This change means that packages are no longer forced >>>>>>>>>> to create a config_local.py file, and there is no longer any need to >>>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Dave, >>>>>>>>>>> >>>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected] >>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can you please review the attached patch, and apply if >>>>>>>>>>>>>>> you're happy with it? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>>>> 'web' mode. >>>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>>> >>>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after >>>>>>>>>>>>>> initializing the Security object. >>>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hmm. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> OK, thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>>> problem. >>>>>>>>>>>>> >>>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>>> >>>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>>> installations/databases. >>>>>>>>>>>> >>>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>>> >>>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>>> >>>>>>>>>>> Please find the updated patch. >>>>>>>>>>> >>>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> Thanks & Regards, >>>>>>>>>>> >>>>>>>>>>> Ashesh Vashi >>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>>> >>>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>>> >>>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>>>> appreciated)! >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Dave Page >>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>> >>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Syed Fahar Abbas >>>>>>>>> Quality Management Group >>>>>>>>> >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>> Website: www.enterprisedb.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 07:36 Ashesh Vashi <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 0 replies; 33+ messages in thread From: Ashesh Vashi @ 2016-10-20 07:36 UTC (permalink / raw) To: Fahar Abbas <[email protected]>; +Cc: Murtuza Zabuawala <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> On Thu, Oct 20, 2016 at 1:00 PM, Fahar Abbas <[email protected]> wrote: > Murtaza, > > Once i delete key table from pgAdmin4.db file then i can launch pgAdmin4 > with terminal as well as web browser for existing pgAdmin4 setup. > > > In case of fresh setup once we delete key table and apply latest patch > following message displayed: > > python pgAdmin4.py > Traceback (most recent call last): > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", > line 1139, in _execute_context > context) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/default.py", > line 450, in do_execute > cursor.execute(statement, parameters) > sqlite3.OperationalError: no such table: keys > That's because - you've not yet update the ConfigDB to 13 in the version table in the sqlite configuration table. And, the next exception, you're describing, is result of that. -- Thanks & Regards, Ashesh Vashi EnterpriseDB INDIA: Enterprise PostgreSQL Company <http://www.enterprisedb.com/; *http://www.linkedin.com/in/asheshvashi* <http://www.linkedin.com/in/asheshvashi; > > The above exception was the direct cause of the following exception: > > Traceback (most recent call last): > File "pgAdmin4.py", line 46, in <module> > app = create_app() > File "/home/fahar/Projects/pgadmin4/web/pgadmin/__init__.py", line 224, > in create_app > config.CSRF_SESSION_KEY = Keys.query.filter_by(name = > 'CSRF_SESSION_KEY').first().value > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", > line 2659, in first > ret = list(self[0:1]) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", > line 2457, in __getitem__ > return list(res) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", > line 2761, in __iter__ > return self._execute_and_instances(context) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", > line 2776, in _execute_and_instances > result = conn.execute(querycontext.statement, self._params) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", > line 914, in execute > return meth(self, multiparams, params) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/sql/elements.py", > line 323, in _execute_on_connection > return connection._execute_clauseelement(self, multiparams, params) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", > line 1010, in _execute_clauseelement > compiled_sql, distilled_params > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", > line 1146, in _execute_context > context) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", > line 1341, in _handle_dbapi_exception > exc_info > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", > line 202, in raise_from_cause > reraise(type(exception), exception, tb=exc_tb, cause=cause) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", > line 185, in reraise > raise value.with_traceback(tb) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", > line 1139, in _execute_context > context) > File "/home/fahar/venv/lib/python3.5/site-packages/sqlalchemy/engine/default.py", > line 450, in do_execute > cursor.execute(statement, parameters) > sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such > table: keys [SQL: 'SELECT keys.name AS keys_name, keys.value AS > keys_value \nFROM keys \nWHERE keys.name = ?\n LIMIT ? OFFSET ?'] > [parameters: ('CSRF_SESSION_KEY', 1, 0)] > > > On Thu, Oct 20, 2016 at 11:00 AM, Murtuza Zabuawala <murtuza.zabuawala@ > enterprisedb.com> wrote: > >> Could you delete 'keys' table from pgadmin4.db file & try again? >> >> -- >> Regards, >> Murtuza Zabuawala >> EnterpriseDB: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Thu, Oct 20, 2016 at 11:26 AM, Fahar Abbas < >> [email protected]> wrote: >> >>> Murtaza, >>> >>> I have applied this patch and there is no success on new pgAdmin4 setup >>> as well as existing pgAdmin4 setup. >>> >>> On Thu, Oct 20, 2016 at 10:45 AM, Murtuza Zabuawala < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> PFA patch to fix the issue for Pyhton3. >>>> RM#1849 >>>> >>>> -- >>>> Regards, >>>> Murtuza Zabuawala >>>> EnterpriseDB: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>>> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Hi Dave, >>>>> >>>>> I have reopened following RM: >>>>> ================================ >>>>> https://redmine.postgresql.org/issues/1849 >>>>> >>>>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >>>>> >>>>>> Patch applied. >>>>>> >>>>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Fahar, >>>>>>> >>>>>>> Please log the case on redmine. >>>>>>> Please find the attached patch, please apply it locally, and test it. >>>>>>> >>>>>>> And, please update the case, and this mail chain accordingly. >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> >>>>>>> Ashesh Vashi >>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>> <http://www.enterprisedb.com; >>>>>>> >>>>>>> >>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Here is the output of if we copy config_local.py and execute python >>>>>>>> setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: [email protected] >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in <module> >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> python setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> User can not do any setup for web based now. >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: [email protected] >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in <module> >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Dave, >>>>>>>>> >>>>>>>>> Testing Environment >>>>>>>>> >>>>>>>>> Ubuntu 16.04 Linux 64: >>>>>>>>> -------------------------------- >>>>>>>>> >>>>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>>>> >>>>>>>>> >>>>>>>>> 1) Install GIT >>>>>>>>> >>>>>>>>> sudo apt-get install git >>>>>>>>> >>>>>>>>> 2) Install pip3 >>>>>>>>> >>>>>>>>> sudo apt-get install python3-pip >>>>>>>>> >>>>>>>>> 3) Install virtualenv >>>>>>>>> >>>>>>>>> sudo pip3 install virtualenv >>>>>>>>> >>>>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>>>> pycrypto module >>>>>>>>> >>>>>>>>> sudo apt-get install libpq-dev >>>>>>>>> >>>>>>>>> sudo apt-get install python3-dev >>>>>>>>> >>>>>>>>> 5) Create virtual environment >>>>>>>>> >>>>>>>>> virtualenv -p python3 venv >>>>>>>>> >>>>>>>>> 6) Create mkdir Projects >>>>>>>>> >>>>>>>>> 7) Clone git repo in Projects >>>>>>>>> >>>>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>>>> >>>>>>>>> 8) activate virtual environment >>>>>>>>> >>>>>>>>> source venv/bin/activate >>>>>>>>> >>>>>>>>> 9) Install modules >>>>>>>>> >>>>>>>>> pip3 install -r requirements_py3.txt >>>>>>>>> >>>>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>>>> Projects\pgAdmin4\web * >>>>>>>>> >>>>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>>>> python setup.py >>>>>>>>> >>>>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>>>> >>>>>>>>> Here is the output: >>>>>>>>> ------------------------- >>>>>>>>> >>>>>>>>> python setup.py >>>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>>> ====================================== >>>>>>>>> >>>>>>>>> >>>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>>> does not exist. >>>>>>>>> Entering initial setup mode... >>>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>>> >>>>>>>>> >>>>>>>>> Enter the email address and password to use for the initial >>>>>>>>> pgAdmin user account: >>>>>>>>> >>>>>>>>> Email address: [email protected] >>>>>>>>> Password: >>>>>>>>> Retype password: >>>>>>>>> Traceback (most recent call last): >>>>>>>>> File "setup.py", line 449, in <module> >>>>>>>>> do_setup(app) >>>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>>> password = encrypt_password(p1) >>>>>>>>> File "/home/fahar/venv/lib/python3. >>>>>>>>> 5/site-packages/flask_security/utils.py", line 150, in >>>>>>>>> encrypt_password >>>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>>> File "/home/fahar/venv/lib/python3. >>>>>>>>> 5/site-packages/flask_security/utils.py", line 108, in get_hmac >>>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` >>>>>>>>> must not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>>> "pbkdf2_sha512" >>>>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>>>> >>>>>>>>> >>>>>>>>> Is this expected? >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Sure, >>>>>>>>>> >>>>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>>>> >>>>>>>>>> Kind Regards, >>>>>>>>>> >>>>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Patch applied. >>>>>>>>>>> >>>>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>>>> >>>>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>>>> >>>>>>>>>>> Packagers: This change means that packages are no longer forced >>>>>>>>>>> to create a config_local.py file, and there is no longer any need to >>>>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave, >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Can you please review the attached patch, and apply if >>>>>>>>>>>>>>>> you're happy with it? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment >>>>>>>>>>>>>>> with 'web' mode. >>>>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after >>>>>>>>>>>>>>> initializing the Security object. >>>>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hmm. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I had moved the Security object initialization after >>>>>>>>>>>>>>> fetching these configurations from the database. >>>>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> OK, thanks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>>>> problem. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>>>> >>>>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>>>> installations/databases. >>>>>>>>>>>>> >>>>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>>>> >>>>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>>>> >>>>>>>>>>>> Please find the updated patch. >>>>>>>>>>>> >>>>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> Thanks & Regards, >>>>>>>>>>>> >>>>>>>>>>>> Ashesh Vashi >>>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>>>> >>>>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>>>> >>>>>>>>>>>>> Patch attached - please review (Ashesh, but others too would >>>>>>>>>>>>> be appreciated)! >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Dave Page >>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>>> >>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Syed Fahar Abbas >>>>>>>>>> Quality Management Group >>>>>>>>>> >>>>>>>>>> EnterpriseDB Corporation >>>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>>> Website: www.enterprisedb.com >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Syed Fahar Abbas >>>>>>>>> Quality Management Group >>>>>>>>> >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>> Website: www.enterprisedb.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 07:53 Ashesh Vashi <[email protected]> parent: Murtuza Zabuawala <[email protected]> 1 sibling, 1 reply; 33+ messages in thread From: Ashesh Vashi @ 2016-10-20 07:53 UTC (permalink / raw) To: Murtuza Zabuawala <[email protected]>; Fahar Abbas <[email protected]>; +Cc: Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> You were missing the other places, before the function - do_setup(..) call, where we are generating the other keys automatically. I've checked-in the code. Fahar, If you want to test the issue properly, please follow the below steps: - git checkout c4f1b8eb112e99d228c40a412020fa616dbe44f6 This was the commit-id, where we have the configuration schema version '13'. - Delete the existing pgadmin4.db - Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in config_local.py. - Execute the command - 'python setup.py'. - Now - you've configuration file with old schema. - 'git checkout master' to go to the latest code. - Make sure - you've latest code. 'git pull'. - Now - rerun 'python setup.py/pgAdmin4.py'. It should work. -- Thanks & Regards, Ashesh Vashi EnterpriseDB INDIA: Enterprise PostgreSQL Company <http://www.enterprisedb.com; *http://www.linkedin.com/in/asheshvashi* <http://www.linkedin.com/in/asheshvashi; On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabuawala < [email protected]> wrote: > Hi, > > PFA patch to fix the issue for Pyhton3. > RM#1849 > > -- > Regards, > Murtuza Zabuawala > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < > [email protected]> wrote: > >> Hi Dave, >> >> I have reopened following RM: >> ================================ >> https://redmine.postgresql.org/issues/1849 >> >> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >> >>> Patch applied. >>> >>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>> [email protected]> wrote: >>> >>>> Hi Fahar, >>>> >>>> Please log the case on redmine. >>>> Please find the attached patch, please apply it locally, and test it. >>>> >>>> And, please update the case, and this mail chain accordingly. >>>> >>>> -- >>>> >>>> Thanks & Regards, >>>> >>>> Ashesh Vashi >>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>> <http://www.enterprisedb.com; >>>> >>>> >>>> *http://www.linkedin.com/in/asheshvashi* >>>> <http://www.linkedin.com/in/asheshvashi; >>>> >>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Here is the output of if we copy config_local.py and execute python >>>>> setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> python setup.py >>>>> pgAdmin 4 - Application Initialisation >>>>> ====================================== >>>>> >>>>> User can not do any setup for web based now. >>>>> >>>>> >>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>> not exist. >>>>> Entering initial setup mode... >>>>> NOTE: Configuring authentication for SERVER mode. >>>>> >>>>> >>>>> Enter the email address and password to use for the initial >>>>> pgAdmin user account: >>>>> >>>>> Email address: [email protected] >>>>> Password: >>>>> Retype password: >>>>> Traceback (most recent call last): >>>>> File "setup.py", line 449, in <module> >>>>> do_setup(app) >>>>> File "setup.py", line 96, in do_setup >>>>> password = encrypt_password(p1) >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 150, in encrypt_password >>>>> signed = get_hmac(password).decode('ascii') >>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>> line 108, in get_hmac >>>>> 'set to "%s"' % _security.password_hash) >>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>> "pbkdf2_sha512" >>>>> >>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Dave, >>>>>> >>>>>> Testing Environment >>>>>> >>>>>> Ubuntu 16.04 Linux 64: >>>>>> -------------------------------- >>>>>> >>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>> >>>>>> >>>>>> 1) Install GIT >>>>>> >>>>>> sudo apt-get install git >>>>>> >>>>>> 2) Install pip3 >>>>>> >>>>>> sudo apt-get install python3-pip >>>>>> >>>>>> 3) Install virtualenv >>>>>> >>>>>> sudo pip3 install virtualenv >>>>>> >>>>>> 4) install below dependency as it is required for psycopg2 & pycrypto >>>>>> module >>>>>> >>>>>> sudo apt-get install libpq-dev >>>>>> >>>>>> sudo apt-get install python3-dev >>>>>> >>>>>> 5) Create virtual environment >>>>>> >>>>>> virtualenv -p python3 venv >>>>>> >>>>>> 6) Create mkdir Projects >>>>>> >>>>>> 7) Clone git repo in Projects >>>>>> >>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>> >>>>>> 8) activate virtual environment >>>>>> >>>>>> source venv/bin/activate >>>>>> >>>>>> 9) Install modules >>>>>> >>>>>> pip3 install -r requirements_py3.txt >>>>>> >>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>> Projects\pgAdmin4\web * >>>>>> >>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>> python setup.py >>>>>> >>>>>> If user does not create config_local.py and do Python setup.py for >>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>> >>>>>> Here is the output: >>>>>> ------------------------- >>>>>> >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>> >>>>>> >>>>>> Is this expected? >>>>>> >>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Sure, >>>>>>> >>>>>>> Will test this thoroughly after complete investigation. >>>>>>> >>>>>>> Kind Regards, >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Patch applied. >>>>>>>> >>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>> >>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>> >>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>> installations, if you have included them in 1.0) >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Dave, >>>>>>>>> >>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave, >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>> >>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>> happy with it? >>>>>>>>>>>>> >>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>> >>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>> 'web' mode. >>>>>>>>>>>> - Apply the patch >>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>> >>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>> >>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>> the Security object. >>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hmm. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>> >>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> My concern with that is that we'll likely be storing the default >>>>>>>>>>> config values in many cases, thus for those users, perpetuating the problem. >>>>>>>>>>> >>>>>>>>>>> I guess what we need to do is re-encrypt the password during the >>>>>>>>>>> upgrade - however, that makes me think; we then have both the key and the >>>>>>>>>>> encrypted passwords in the same database which is clearly not a good idea. >>>>>>>>>>> Sigh... Needs more thought. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>> >>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>> installations/databases. >>>>>>>>>> >>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>> >>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>> >>>>>>>>> Please find the updated patch. >>>>>>>>> >>>>>>>>> Otherwise - looks good to me. >>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks & Regards, >>>>>>>>> >>>>>>>>> Ashesh Vashi >>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>> >>>>>>>>> >>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>> >>>>>>>>>> >>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>> >>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>> >>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>> appreciated)! >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 08:03 Fahar Abbas <[email protected]> parent: Ashesh Vashi <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-20 08:03 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: Murtuza Zabuawala <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Thanks Ashesh for your steps and will also test with new setup( on fresh machine) and our customers can face this problem with new setup and will update the status accordingly. On Thu, Oct 20, 2016 at 12:53 PM, Ashesh Vashi < [email protected]> wrote: > You were missing the other places, before the function - do_setup(..) > call, where we are generating the other keys automatically. > I've checked-in the code. > > Fahar, > > If you want to test the issue properly, please follow the below steps: > - git checkout c4f1b8eb112e99d228c40a412020fa616dbe44f6 > This was the commit-id, where we have the configuration schema version > '13'. > - Delete the existing pgadmin4.db > - Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in > config_local.py. > - Execute the command - 'python setup.py'. > - Now - you've configuration file with old schema. > - 'git checkout master' to go to the latest code. > - Make sure - you've latest code. 'git pull'. > - Now - rerun 'python setup.py/pgAdmin4.py'. It should work. > > > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > <http://www.enterprisedb.com; > > > *http://www.linkedin.com/in/asheshvashi* > <http://www.linkedin.com/in/asheshvashi; > > On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabuawala <murtuza.zabuawala@ > enterprisedb.com> wrote: > >> Hi, >> >> PFA patch to fix the issue for Pyhton3. >> RM#1849 >> >> -- >> Regards, >> Murtuza Zabuawala >> EnterpriseDB: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >> [email protected]> wrote: >> >>> Hi Dave, >>> >>> I have reopened following RM: >>> ================================ >>> https://redmine.postgresql.org/issues/1849 >>> >>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >>> >>>> Patch applied. >>>> >>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>> [email protected]> wrote: >>>> >>>>> Hi Fahar, >>>>> >>>>> Please log the case on redmine. >>>>> Please find the attached patch, please apply it locally, and test it. >>>>> >>>>> And, please update the case, and this mail chain accordingly. >>>>> >>>>> -- >>>>> >>>>> Thanks & Regards, >>>>> >>>>> Ashesh Vashi >>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>> <http://www.enterprisedb.com; >>>>> >>>>> >>>>> *http://www.linkedin.com/in/asheshvashi* >>>>> <http://www.linkedin.com/in/asheshvashi; >>>>> >>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>> [email protected]> wrote: >>>>> >>>>>> Here is the output of if we copy config_local.py and execute python >>>>>> setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> python setup.py >>>>>> pgAdmin 4 - Application Initialisation >>>>>> ====================================== >>>>>> >>>>>> User can not do any setup for web based now. >>>>>> >>>>>> >>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' does >>>>>> not exist. >>>>>> Entering initial setup mode... >>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>> >>>>>> >>>>>> Enter the email address and password to use for the initial >>>>>> pgAdmin user account: >>>>>> >>>>>> Email address: [email protected] >>>>>> Password: >>>>>> Retype password: >>>>>> Traceback (most recent call last): >>>>>> File "setup.py", line 449, in <module> >>>>>> do_setup(app) >>>>>> File "setup.py", line 96, in do_setup >>>>>> password = encrypt_password(p1) >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 150, in encrypt_password >>>>>> signed = get_hmac(password).decode('ascii') >>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>> line 108, in get_hmac >>>>>> 'set to "%s"' % _security.password_hash) >>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>> "pbkdf2_sha512" >>>>>> >>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Dave, >>>>>>> >>>>>>> Testing Environment >>>>>>> >>>>>>> Ubuntu 16.04 Linux 64: >>>>>>> -------------------------------- >>>>>>> >>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>> >>>>>>> >>>>>>> 1) Install GIT >>>>>>> >>>>>>> sudo apt-get install git >>>>>>> >>>>>>> 2) Install pip3 >>>>>>> >>>>>>> sudo apt-get install python3-pip >>>>>>> >>>>>>> 3) Install virtualenv >>>>>>> >>>>>>> sudo pip3 install virtualenv >>>>>>> >>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>> pycrypto module >>>>>>> >>>>>>> sudo apt-get install libpq-dev >>>>>>> >>>>>>> sudo apt-get install python3-dev >>>>>>> >>>>>>> 5) Create virtual environment >>>>>>> >>>>>>> virtualenv -p python3 venv >>>>>>> >>>>>>> 6) Create mkdir Projects >>>>>>> >>>>>>> 7) Clone git repo in Projects >>>>>>> >>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>> >>>>>>> 8) activate virtual environment >>>>>>> >>>>>>> source venv/bin/activate >>>>>>> >>>>>>> 9) Install modules >>>>>>> >>>>>>> pip3 install -r requirements_py3.txt >>>>>>> >>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>> Projects\pgAdmin4\web * >>>>>>> >>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>> python setup.py >>>>>>> >>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>> >>>>>>> Here is the output: >>>>>>> ------------------------- >>>>>>> >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>> >>>>>>> >>>>>>> Is this expected? >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Sure, >>>>>>>> >>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>> >>>>>>>> Kind Regards, >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Patch applied. >>>>>>>>> >>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>> >>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>> >>>>>>>>> Packagers: This change means that packages are no longer forced to >>>>>>>>> create a config_local.py file, and there is no longer any need to >>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Dave, >>>>>>>>>> >>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can you please review the attached patch, and apply if you're >>>>>>>>>>>>>> happy with it? >>>>>>>>>>>>>> >>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>> >>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>>> 'web' mode. >>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>> >>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>> >>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after initializing >>>>>>>>>>>>> the Security object. >>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hmm. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>> >>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>> problem. >>>>>>>>>>>> >>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>> >>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>> installations/databases. >>>>>>>>>>> >>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>> >>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>> >>>>>>>>>> Please find the updated patch. >>>>>>>>>> >>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks & Regards, >>>>>>>>>> >>>>>>>>>> Ashesh Vashi >>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>> >>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>> >>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>>> appreciated)! >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dave Page >>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>> Twitter: @pgsnake >>>>>>>>> >>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Syed Fahar Abbas >>>>>> Quality Management Group >>>>>> >>>>>> EnterpriseDB Corporation >>>>>> Phone Office: +92-51-835-8874 >>>>>> Phone Direct: +92-51-8466803 >>>>>> Mobile: +92-333-5409707 >>>>>> Skype ID: syed.fahar.abbas >>>>>> Website: www.enterprisedb.com >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >>> >>> -- >>> Syed Fahar Abbas >>> Quality Management Group >>> >>> EnterpriseDB Corporation >>> Phone Office: +92-51-835-8874 >>> Phone Direct: +92-51-8466803 >>> Mobile: +92-333-5409707 >>> Skype ID: syed.fahar.abbas >>> Website: www.enterprisedb.com >>> >> >> > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 08:35 Fahar Abbas <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 1 reply; 33+ messages in thread From: Fahar Abbas @ 2016-10-20 08:35 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: Murtuza Zabuawala <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> Thanks pgAdmin4 Development team! User can launch pgAdmin4 with web browser with no issues with fresh setup, i tried on Ubuntu 16.04 Linux 64(Python 3.5). Will also try on OS X with Python 3 and will let you know. On Thu, Oct 20, 2016 at 1:03 PM, Fahar Abbas <[email protected]> wrote: > Thanks Ashesh for your steps and will also test with new setup( on fresh > machine) and our customers can face this problem with new setup and will > update the status accordingly. > > On Thu, Oct 20, 2016 at 12:53 PM, Ashesh Vashi < > [email protected]> wrote: > >> You were missing the other places, before the function - do_setup(..) >> call, where we are generating the other keys automatically. >> I've checked-in the code. >> >> Fahar, >> >> If you want to test the issue properly, please follow the below steps: >> - git checkout c4f1b8eb112e99d228c40a412020fa616dbe44f6 >> This was the commit-id, where we have the configuration schema version >> '13'. >> - Delete the existing pgadmin4.db >> - Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in >> config_local.py. >> - Execute the command - 'python setup.py'. >> - Now - you've configuration file with old schema. >> - 'git checkout master' to go to the latest code. >> - Make sure - you've latest code. 'git pull'. >> - Now - rerun 'python setup.py/pgAdmin4.py'. It should work. >> >> >> -- >> >> Thanks & Regards, >> >> Ashesh Vashi >> EnterpriseDB INDIA: Enterprise PostgreSQL Company >> <http://www.enterprisedb.com; >> >> >> *http://www.linkedin.com/in/asheshvashi* >> <http://www.linkedin.com/in/asheshvashi; >> >> On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabuawala < >> [email protected]> wrote: >> >>> Hi, >>> >>> PFA patch to fix the issue for Pyhton3. >>> RM#1849 >>> >>> -- >>> Regards, >>> Murtuza Zabuawala >>> EnterpriseDB: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >>> [email protected]> wrote: >>> >>>> Hi Dave, >>>> >>>> I have reopened following RM: >>>> ================================ >>>> https://redmine.postgresql.org/issues/1849 >>>> >>>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >>>> >>>>> Patch applied. >>>>> >>>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Fahar, >>>>>> >>>>>> Please log the case on redmine. >>>>>> Please find the attached patch, please apply it locally, and test it. >>>>>> >>>>>> And, please update the case, and this mail chain accordingly. >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & Regards, >>>>>> >>>>>> Ashesh Vashi >>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>> <http://www.enterprisedb.com; >>>>>> >>>>>> >>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>> >>>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Here is the output of if we copy config_local.py and execute python >>>>>>> setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> python setup.py >>>>>>> pgAdmin 4 - Application Initialisation >>>>>>> ====================================== >>>>>>> >>>>>>> User can not do any setup for web based now. >>>>>>> >>>>>>> >>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>> does not exist. >>>>>>> Entering initial setup mode... >>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>> >>>>>>> >>>>>>> Enter the email address and password to use for the initial >>>>>>> pgAdmin user account: >>>>>>> >>>>>>> Email address: [email protected] >>>>>>> Password: >>>>>>> Retype password: >>>>>>> Traceback (most recent call last): >>>>>>> File "setup.py", line 449, in <module> >>>>>>> do_setup(app) >>>>>>> File "setup.py", line 96, in do_setup >>>>>>> password = encrypt_password(p1) >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 150, in encrypt_password >>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>> line 108, in get_hmac >>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>> "pbkdf2_sha512" >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Dave, >>>>>>>> >>>>>>>> Testing Environment >>>>>>>> >>>>>>>> Ubuntu 16.04 Linux 64: >>>>>>>> -------------------------------- >>>>>>>> >>>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>>> >>>>>>>> >>>>>>>> 1) Install GIT >>>>>>>> >>>>>>>> sudo apt-get install git >>>>>>>> >>>>>>>> 2) Install pip3 >>>>>>>> >>>>>>>> sudo apt-get install python3-pip >>>>>>>> >>>>>>>> 3) Install virtualenv >>>>>>>> >>>>>>>> sudo pip3 install virtualenv >>>>>>>> >>>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>>> pycrypto module >>>>>>>> >>>>>>>> sudo apt-get install libpq-dev >>>>>>>> >>>>>>>> sudo apt-get install python3-dev >>>>>>>> >>>>>>>> 5) Create virtual environment >>>>>>>> >>>>>>>> virtualenv -p python3 venv >>>>>>>> >>>>>>>> 6) Create mkdir Projects >>>>>>>> >>>>>>>> 7) Clone git repo in Projects >>>>>>>> >>>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>>> >>>>>>>> 8) activate virtual environment >>>>>>>> >>>>>>>> source venv/bin/activate >>>>>>>> >>>>>>>> 9) Install modules >>>>>>>> >>>>>>>> pip3 install -r requirements_py3.txt >>>>>>>> >>>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>>> Projects\pgAdmin4\web * >>>>>>>> >>>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>>> python setup.py >>>>>>>> >>>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>>> >>>>>>>> Here is the output: >>>>>>>> ------------------------- >>>>>>>> >>>>>>>> python setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: [email protected] >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in <module> >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>>> >>>>>>>> >>>>>>>> Is this expected? >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Sure, >>>>>>>>> >>>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>>> >>>>>>>>> Kind Regards, >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Patch applied. >>>>>>>>>> >>>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>>> >>>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>>> >>>>>>>>>> Packagers: This change means that packages are no longer forced >>>>>>>>>> to create a config_local.py file, and there is no longer any need to >>>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Dave, >>>>>>>>>>> >>>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page <[email protected] >>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can you please review the attached patch, and apply if >>>>>>>>>>>>>>> you're happy with it? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment with >>>>>>>>>>>>>> 'web' mode. >>>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>>> >>>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after >>>>>>>>>>>>>> initializing the Security object. >>>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hmm. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I had moved the Security object initialization after fetching >>>>>>>>>>>>>> these configurations from the database. >>>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> OK, thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>>> problem. >>>>>>>>>>>>> >>>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>>> >>>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>>> installations/databases. >>>>>>>>>>>> >>>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>>> >>>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>>> >>>>>>>>>>> Please find the updated patch. >>>>>>>>>>> >>>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> Thanks & Regards, >>>>>>>>>>> >>>>>>>>>>> Ashesh Vashi >>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>>> >>>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>>> >>>>>>>>>>>> Patch attached - please review (Ashesh, but others too would be >>>>>>>>>>>> appreciated)! >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Dave Page >>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>> >>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Dave Page >>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>> Twitter: @pgsnake >>>>>>>>>> >>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Syed Fahar Abbas >>>>>>>>> Quality Management Group >>>>>>>>> >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>> Website: www.enterprisedb.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Syed Fahar Abbas >>>>>>> Quality Management Group >>>>>>> >>>>>>> EnterpriseDB Corporation >>>>>>> Phone Office: +92-51-835-8874 >>>>>>> Phone Direct: +92-51-8466803 >>>>>>> Mobile: +92-333-5409707 >>>>>>> Skype ID: syed.fahar.abbas >>>>>>> Website: www.enterprisedb.com >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>> >>>> >>>> >>>> -- >>>> Syed Fahar Abbas >>>> Quality Management Group >>>> >>>> EnterpriseDB Corporation >>>> Phone Office: +92-51-835-8874 >>>> Phone Direct: +92-51-8466803 >>>> Mobile: +92-333-5409707 >>>> Skype ID: syed.fahar.abbas >>>> Website: www.enterprisedb.com >>>> >>> >>> >> > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
* Re: RM1849: Auto-generating security keys @ 2016-10-20 10:54 Fahar Abbas <[email protected]> parent: Fahar Abbas <[email protected]> 0 siblings, 0 replies; 33+ messages in thread From: Fahar Abbas @ 2016-10-20 10:54 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: Murtuza Zabuawala <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers; Josh Berkus <[email protected]>; Devrim GÜNDÜZ <[email protected]>; Magnus Hagander <[email protected]>; Sandeep Thakkar <[email protected]>; Hamid Quddus Akhtar <[email protected]> All, I have not seen any issue in pgAdmin4 on MAC OS X(Python 3.5) Platform as well hence 1849 is resolved. Kind Regards, On Thu, Oct 20, 2016 at 1:35 PM, Fahar Abbas <[email protected]> wrote: > Thanks pgAdmin4 Development team! > > User can launch pgAdmin4 with web browser with no issues with fresh setup, > i tried on Ubuntu 16.04 Linux 64(Python 3.5). Will also try on OS X with > Python 3 and will let you know. > > On Thu, Oct 20, 2016 at 1:03 PM, Fahar Abbas <[email protected] > > wrote: > >> Thanks Ashesh for your steps and will also test with new setup( on fresh >> machine) and our customers can face this problem with new setup and will >> update the status accordingly. >> >> On Thu, Oct 20, 2016 at 12:53 PM, Ashesh Vashi < >> [email protected]> wrote: >> >>> You were missing the other places, before the function - do_setup(..) >>> call, where we are generating the other keys automatically. >>> I've checked-in the code. >>> >>> Fahar, >>> >>> If you want to test the issue properly, please follow the below steps: >>> - git checkout c4f1b8eb112e99d228c40a412020fa616dbe44f6 >>> This was the commit-id, where we have the configuration schema version >>> '13'. >>> - Delete the existing pgadmin4.db >>> - Set CSRF_SESSION_KEY, SECRET_KEY, SECURITY_PASSWORD_SALT in >>> config_local.py. >>> - Execute the command - 'python setup.py'. >>> - Now - you've configuration file with old schema. >>> - 'git checkout master' to go to the latest code. >>> - Make sure - you've latest code. 'git pull'. >>> - Now - rerun 'python setup.py/pgAdmin4.py'. It should work. >>> >>> >>> -- >>> >>> Thanks & Regards, >>> >>> Ashesh Vashi >>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>> <http://www.enterprisedb.com; >>> >>> >>> *http://www.linkedin.com/in/asheshvashi* >>> <http://www.linkedin.com/in/asheshvashi; >>> >>> On Thu, Oct 20, 2016 at 11:15 AM, Murtuza Zabuawala < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> PFA patch to fix the issue for Pyhton3. >>>> RM#1849 >>>> >>>> -- >>>> Regards, >>>> Murtuza Zabuawala >>>> EnterpriseDB: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>>> On Thu, Oct 20, 2016 at 11:03 AM, Fahar Abbas < >>>> [email protected]> wrote: >>>> >>>>> Hi Dave, >>>>> >>>>> I have reopened following RM: >>>>> ================================ >>>>> https://redmine.postgresql.org/issues/1849 >>>>> >>>>> On Wed, Oct 19, 2016 at 6:04 PM, Dave Page <[email protected]> wrote: >>>>> >>>>>> Patch applied. >>>>>> >>>>>> On Wed, Oct 19, 2016 at 11:55 AM, Ashesh Vashi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Fahar, >>>>>>> >>>>>>> Please log the case on redmine. >>>>>>> Please find the attached patch, please apply it locally, and test it. >>>>>>> >>>>>>> And, please update the case, and this mail chain accordingly. >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> >>>>>>> Ashesh Vashi >>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>> <http://www.enterprisedb.com; >>>>>>> >>>>>>> >>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>> >>>>>>> On Wed, Oct 19, 2016 at 3:47 PM, Fahar Abbas < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Here is the output of if we copy config_local.py and execute python >>>>>>>> setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: [email protected] >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in <module> >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> python setup.py >>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>> ====================================== >>>>>>>> >>>>>>>> User can not do any setup for web based now. >>>>>>>> >>>>>>>> >>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>> does not exist. >>>>>>>> Entering initial setup mode... >>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>> >>>>>>>> >>>>>>>> Enter the email address and password to use for the initial >>>>>>>> pgAdmin user account: >>>>>>>> >>>>>>>> Email address: [email protected] >>>>>>>> Password: >>>>>>>> Retype password: >>>>>>>> Traceback (most recent call last): >>>>>>>> File "setup.py", line 449, in <module> >>>>>>>> do_setup(app) >>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>> password = encrypt_password(p1) >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 150, in encrypt_password >>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>> File "/home/fahar/venv/lib/python3.5/site-packages/flask_security/utils.py", >>>>>>>> line 108, in get_hmac >>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` must >>>>>>>> not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>> "pbkdf2_sha512" >>>>>>>> >>>>>>>> On Wed, Oct 19, 2016 at 3:03 PM, Fahar Abbas < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Dave, >>>>>>>>> >>>>>>>>> Testing Environment >>>>>>>>> >>>>>>>>> Ubuntu 16.04 Linux 64: >>>>>>>>> -------------------------------- >>>>>>>>> >>>>>>>>> pg-AdminIV Development Environment Setup for Ubuntu : >>>>>>>>> >>>>>>>>> >>>>>>>>> 1) Install GIT >>>>>>>>> >>>>>>>>> sudo apt-get install git >>>>>>>>> >>>>>>>>> 2) Install pip3 >>>>>>>>> >>>>>>>>> sudo apt-get install python3-pip >>>>>>>>> >>>>>>>>> 3) Install virtualenv >>>>>>>>> >>>>>>>>> sudo pip3 install virtualenv >>>>>>>>> >>>>>>>>> 4) install below dependency as it is required for psycopg2 & >>>>>>>>> pycrypto module >>>>>>>>> >>>>>>>>> sudo apt-get install libpq-dev >>>>>>>>> >>>>>>>>> sudo apt-get install python3-dev >>>>>>>>> >>>>>>>>> 5) Create virtual environment >>>>>>>>> >>>>>>>>> virtualenv -p python3 venv >>>>>>>>> >>>>>>>>> 6) Create mkdir Projects >>>>>>>>> >>>>>>>>> 7) Clone git repo in Projects >>>>>>>>> >>>>>>>>> git clone http://git.postgresql.org/git/pgadmin4.git >>>>>>>>> >>>>>>>>> 8) activate virtual environment >>>>>>>>> >>>>>>>>> source venv/bin/activate >>>>>>>>> >>>>>>>>> 9) Install modules >>>>>>>>> >>>>>>>>> pip3 install -r requirements_py3.txt >>>>>>>>> >>>>>>>>> *10) Edit the config.py file to config_local.py resides in >>>>>>>>> Projects\pgAdmin4\web * >>>>>>>>> >>>>>>>>> 11)Now run setup.py file (\Projects\pgAdmin4\web) >>>>>>>>> python setup.py >>>>>>>>> >>>>>>>>> If user does not create config_local.py and do Python setup.py for >>>>>>>>> new Development then SECURITY_PASSWORD_SALT message is also displayed: >>>>>>>>> >>>>>>>>> Here is the output: >>>>>>>>> ------------------------- >>>>>>>>> >>>>>>>>> python setup.py >>>>>>>>> pgAdmin 4 - Application Initialisation >>>>>>>>> ====================================== >>>>>>>>> >>>>>>>>> >>>>>>>>> The configuration database - '/home/fahar/.pgadmin/pgadmin4.db' >>>>>>>>> does not exist. >>>>>>>>> Entering initial setup mode... >>>>>>>>> NOTE: Configuring authentication for SERVER mode. >>>>>>>>> >>>>>>>>> >>>>>>>>> Enter the email address and password to use for the initial >>>>>>>>> pgAdmin user account: >>>>>>>>> >>>>>>>>> Email address: [email protected] >>>>>>>>> Password: >>>>>>>>> Retype password: >>>>>>>>> Traceback (most recent call last): >>>>>>>>> File "setup.py", line 449, in <module> >>>>>>>>> do_setup(app) >>>>>>>>> File "setup.py", line 96, in do_setup >>>>>>>>> password = encrypt_password(p1) >>>>>>>>> File "/home/fahar/venv/lib/python3. >>>>>>>>> 5/site-packages/flask_security/utils.py", line 150, in >>>>>>>>> encrypt_password >>>>>>>>> signed = get_hmac(password).decode('ascii') >>>>>>>>> File "/home/fahar/venv/lib/python3. >>>>>>>>> 5/site-packages/flask_security/utils.py", line 108, in get_hmac >>>>>>>>> 'set to "%s"' % _security.password_hash) >>>>>>>>> RuntimeError: The configuration value `SECURITY_PASSWORD_SALT` >>>>>>>>> must not be None when the value of `SECURITY_PASSWORD_HASH` is set to >>>>>>>>> "pbkdf2_sha512" >>>>>>>>> (venv) fahar@fahar-virtual-machine:~/Projects/pgadmin4/web$ >>>>>>>>> >>>>>>>>> >>>>>>>>> Is this expected? >>>>>>>>> >>>>>>>>> On Wed, Oct 19, 2016 at 1:37 PM, Fahar Abbas < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Sure, >>>>>>>>>> >>>>>>>>>> Will test this thoroughly after complete investigation. >>>>>>>>>> >>>>>>>>>> Kind Regards, >>>>>>>>>> >>>>>>>>>> On Wed, Oct 19, 2016 at 1:27 PM, Dave Page <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Patch applied. >>>>>>>>>>> >>>>>>>>>>> Fahar, can you please test this thoroughly in desktop and server >>>>>>>>>>> modes, with both fresh and upgraded installations? >>>>>>>>>>> >>>>>>>>>>> https://redmine.postgresql.org/issues/1849 >>>>>>>>>>> >>>>>>>>>>> Packagers: This change means that packages are no longer forced >>>>>>>>>>> to create a config_local.py file, and there is no longer any need to >>>>>>>>>>> explicitly set SECURITY_PASSWORD_SALT, SECURITY_KEY >>>>>>>>>>> and CSRF_SESSION_KEY in the config (in fact, they should be removed for new >>>>>>>>>>> installations, if you have included them in 1.0) >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Oct 19, 2016 at 6:46 AM, Ashesh Vashi < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave, >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Oct 15, 2016 at 8:02 AM, Dave Page <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Friday, October 14, 2016, Dave Page <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thursday, October 13, 2016, Ashesh Vashi < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Dave, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Oct 11, 2016 at 9:10 PM, Dave Page < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Ashesh, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Can you please review the attached patch, and apply if >>>>>>>>>>>>>>>> you're happy with it? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Overall the patch looked good to me. >>>>>>>>>>>>>>> But - I encounter an issue in 'web' mode, which wont happen >>>>>>>>>>>>>>> with 'runtime'. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Steps for reproduction on existing pgAdmin 4 environment >>>>>>>>>>>>>>> with 'web' mode. >>>>>>>>>>>>>>> - Apply the patch >>>>>>>>>>>>>>> - Start the pgAdmin4 application (stand alone application). >>>>>>>>>>>>>>> - Open pgAdmin home page. >>>>>>>>>>>>>>> - Log out (if already login). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> And, you will see an exception. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I have figure out the issue with the patch. >>>>>>>>>>>>>>> We were setting the SECURITY_PASSWORD_SALT, after >>>>>>>>>>>>>>> initializing the Security object. >>>>>>>>>>>>>>> Hence - it could not set the SECURITY_KEY, and >>>>>>>>>>>>>>> SECURITY_PASSWORD_SALT properly. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hmm. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I had moved the Security object initialization after >>>>>>>>>>>>>>> fetching these configurations from the database. >>>>>>>>>>>>>>> I have attached a addon patch for the same. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> OK, thanks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Now - I run into another issue. >>>>>>>>>>>>>>> Because - the existing password was hashed using the old >>>>>>>>>>>>>>> SECURITY_PASSWORD_SALT, I am no more able to login to pgAdmin 4. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think - we need to think about different strategy for >>>>>>>>>>>>>>> upgrading the configuration file in the 'web' mode. >>>>>>>>>>>>>>> I was thinking - we can store the existing security >>>>>>>>>>>>>>> configurations in the database during upgrade process in 'web' mode. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> My concern with that is that we'll likely be storing the >>>>>>>>>>>>>> default config values in many cases, thus for those users, perpetuating the >>>>>>>>>>>>>> problem. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I guess what we need to do is re-encrypt the password during >>>>>>>>>>>>>> the upgrade - however, that makes me think; we then have both the key and >>>>>>>>>>>>>> the encrypted passwords in the same database which is clearly not a good >>>>>>>>>>>>>> idea. Sigh... Needs more thought. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> OK, so I've been thinking about this and experimenting for a >>>>>>>>>>>>> couple of hours, as well as annoying the crap out of Magnus by thinking out >>>>>>>>>>>>> loud in his general direction, and it looks like this isn't a major problem >>>>>>>>>>>>> as from what I can see, SECURITY_PASSWORD_SALT is (aside from really being >>>>>>>>>>>>> a key not a salt) not the only salting that's done. >>>>>>>>>>>>> >>>>>>>>>>>>> It looks like it's used system-wide as the key to generate an >>>>>>>>>>>>> HMAC of the users password, which is then passed to passlib which salts and >>>>>>>>>>>>> hashes it. I did some testing, and found that two users with the same >>>>>>>>>>>>> password end up with different hashes in the database, so clearly there is >>>>>>>>>>>>> also per-user salting happening. I also created two users, then dropped the >>>>>>>>>>>>> database and created the same user accounts with the same passwords again, >>>>>>>>>>>>> and found that the resulting hashes were different in both databases - thus >>>>>>>>>>>>> there is something else ensuring the hashes are unique across different >>>>>>>>>>>>> installations/databases. >>>>>>>>>>>>> >>>>>>>>>>>>> So, I believe we can do as you suggest and migrate existing >>>>>>>>>>>>> values for SECURITY_PASSWORD_SALT, given that there's clearly some other >>>>>>>>>>>>> per user and per installation/database salting going on anyway. New >>>>>>>>>>>>> installations can have the random value for SECURITY_PASSWORD_SALT. >>>>>>>>>>>>> >>>>>>>>>>>> We do not need to generate the random SECURITY_PASSWORD_SALT >>>>>>>>>>>> during upgrade mode, which was wrong added in my addon patch. >>>>>>>>>>>> >>>>>>>>>>>> Please find the updated patch. >>>>>>>>>>>> >>>>>>>>>>>> Otherwise - looks good to me. >>>>>>>>>>>> Please commit the new patch (if you're ok with the change). >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> Thanks & Regards, >>>>>>>>>>>> >>>>>>>>>>>> Ashesh Vashi >>>>>>>>>>>> EnterpriseDB INDIA: Enterprise PostgreSQL Company >>>>>>>>>>>> <http://www.enterprisedb.com/; >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> *http://www.linkedin.com/in/asheshvashi* >>>>>>>>>>>> <http://www.linkedin.com/in/asheshvashi; >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I don't believe SECURITY_KEY and CSRF_SESSION_KEY are issues >>>>>>>>>>>>> either, as they're used for purposes that are essentially ephemeral, and >>>>>>>>>>>>> thus can be changed during an upgrade. >>>>>>>>>>>>> >>>>>>>>>>>>> Adding Magnus as I'd appreciate any thoughts he may have. >>>>>>>>>>>>> >>>>>>>>>>>>> Patch attached - please review (Ashesh, but others too would >>>>>>>>>>>>> be appreciated)! >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Dave Page >>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>>> >>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Syed Fahar Abbas >>>>>>>>>> Quality Management Group >>>>>>>>>> >>>>>>>>>> EnterpriseDB Corporation >>>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>>> Website: www.enterprisedb.com >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Syed Fahar Abbas >>>>>>>>> Quality Management Group >>>>>>>>> >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>>> Mobile: +92-333-5409707 >>>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>>> Website: www.enterprisedb.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Syed Fahar Abbas >>>>>>>> Quality Management Group >>>>>>>> >>>>>>>> EnterpriseDB Corporation >>>>>>>> Phone Office: +92-51-835-8874 >>>>>>>> Phone Direct: +92-51-8466803 >>>>>>>> Mobile: +92-333-5409707 >>>>>>>> Skype ID: syed.fahar.abbas >>>>>>>> Website: www.enterprisedb.com >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Syed Fahar Abbas >>>>> Quality Management Group >>>>> >>>>> EnterpriseDB Corporation >>>>> Phone Office: +92-51-835-8874 >>>>> Phone Direct: +92-51-8466803 >>>>> Mobile: +92-333-5409707 >>>>> Skype ID: syed.fahar.abbas >>>>> Website: www.enterprisedb.com >>>>> >>>> >>>> >>> >> >> >> -- >> Syed Fahar Abbas >> Quality Management Group >> >> EnterpriseDB Corporation >> Phone Office: +92-51-835-8874 >> Phone Direct: +92-51-8466803 >> Mobile: +92-333-5409707 >> Skype ID: syed.fahar.abbas >> Website: www.enterprisedb.com >> > > > > -- > Syed Fahar Abbas > Quality Management Group > > EnterpriseDB Corporation > Phone Office: +92-51-835-8874 > Phone Direct: +92-51-8466803 > Mobile: +92-333-5409707 > Skype ID: syed.fahar.abbas > Website: www.enterprisedb.com > -- Syed Fahar Abbas Quality Management Group EnterpriseDB Corporation Phone Office: +92-51-835-8874 Phone Direct: +92-51-8466803 Mobile: +92-333-5409707 Skype ID: syed.fahar.abbas Website: www.enterprisedb.com ^ permalink raw reply [nested|flat] 33+ messages in thread
end of thread, other threads:[~2016-10-20 10:54 UTC | newest] Thread overview: 33+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2016-10-11 15:40 RM1849: Auto-generating security keys Dave Page <[email protected]> 2016-10-13 14:16 ` Ashesh Vashi <[email protected]> 2016-10-14 17:27 ` Dave Page <[email protected]> 2016-10-15 02:32 ` Dave Page <[email protected]> 2016-10-19 05:46 ` Ashesh Vashi <[email protected]> 2016-10-19 08:27 ` Dave Page <[email protected]> 2016-10-19 08:37 ` Fahar Abbas <[email protected]> 2016-10-19 10:03 ` Fahar Abbas <[email protected]> 2016-10-19 10:17 ` Fahar Abbas <[email protected]> 2016-10-19 10:55 ` Ashesh Vashi <[email protected]> 2016-10-19 11:03 ` Fahar Abbas <[email protected]> 2016-10-19 11:42 ` Fahar Abbas <[email protected]> 2016-10-19 12:05 ` Dave Page <[email protected]> 2016-10-19 12:11 ` Neel Patel <[email protected]> 2016-10-19 12:19 ` Fahar Abbas <[email protected]> 2016-10-19 12:51 ` Fahar Abbas <[email protected]> 2016-10-19 12:56 ` Dave Page <[email protected]> 2016-10-19 13:04 ` Dave Page <[email protected]> 2016-10-20 05:33 ` Fahar Abbas <[email protected]> 2016-10-20 05:45 ` Murtuza Zabuawala <[email protected]> 2016-10-20 05:56 ` Fahar Abbas <[email protected]> 2016-10-20 06:00 ` Murtuza Zabuawala <[email protected]> 2016-10-20 06:38 ` Fahar Abbas <[email protected]> 2016-10-20 06:45 ` Dave Page <[email protected]> 2016-10-20 07:30 ` Fahar Abbas <[email protected]> 2016-10-20 07:36 ` Ashesh Vashi <[email protected]> 2016-10-20 07:53 ` Ashesh Vashi <[email protected]> 2016-10-20 08:03 ` Fahar Abbas <[email protected]> 2016-10-20 08:35 ` Fahar Abbas <[email protected]> 2016-10-20 10:54 ` Fahar Abbas <[email protected]> 2016-10-19 11:05 ` Sandeep Thakkar <[email protected]> 2016-10-19 12:39 ` Sandeep Thakkar <[email protected]> 2016-10-19 13:10 ` Dave Page <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox