Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d82OZ-0006yV-Gk for pgadmin-hackers@arkaria.postgresql.org; Tue, 09 May 2017 10:29:31 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1d82OX-0006AR-TQ for pgadmin-hackers@arkaria.postgresql.org; Tue, 09 May 2017 10:29:29 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1d82OW-00066p-84 for pgadmin-hackers@postgresql.org; Tue, 09 May 2017 10:29:28 +0000 Received: from mail-it0-x236.google.com ([2607:f8b0:4001:c0b::236]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1d82OR-0001zs-1V for pgadmin-hackers@postgresql.org; Tue, 09 May 2017 10:29:27 +0000 Received: by mail-it0-x236.google.com with SMTP id e65so53869844ita.1 for ; Tue, 09 May 2017 03:29:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pgadmin-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=mArTqW/iTZ77DUUB/m8XtZVsG9qqP7IVswkCbqLyzLg=; b=DcpL56k9KdWnSo6Y9GgdhJ493Mz9xhP68EVJ1z+czqlVyGToArCTFadqSV6UEyZwod wwMYcjOK12nxJVpwwFRrBAC272SWuVURLII1IncQQZ/qvFIHlF+cTeKTlfpc6fww6qZ5 F6wz1CuFLhuzYxZuol0n0f8H175yVfp7SaGwOKwxyPS1s/dS4rtp/iOS2rA+43Yj+hBZ dpMQiizTadPKFA88py54PbgtLq+/z4nR+SHXWYGqkW06fDGZqzbpHogDw7k4Is6Ygwx8 ozHjqN8gYBAoyJGpnUMIpTxFpBrFpnPzQx2BAcdS4HZ5Lqkjzvmh5bF++T7iOj1Be3gi kizA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=mArTqW/iTZ77DUUB/m8XtZVsG9qqP7IVswkCbqLyzLg=; b=RrJuqjrWx7UAs24DAFN8LxpBdlj57Yk0I52wrErEhGir3fD4abFxFwGhITskQIS5Jr 1WjdjVCFXA4nMp/HA3zMnkinHS3gHctXu+ihjN+YGyJDEeTzuZf2rLzPsCOQ18CnH89R D5uxKcXNa51URZ5sgncRsbhtWaKFUnbJAizX3XH4Qm95sAgaLJpGVjGEBQGyXaHpjEkc CpqQRin52IhAsZWnXBPinAE/t/kFakr1TnUtUYP+SG2cx6JUR81MDBxgFWFmDgZrO1wq 2KzFhCdCLkTSPg9Wa8dtVuHZGR0SI4++ktrAQd2uAfLoamYLIaJWB0vS7ePIjnNJENGe ZKnQ== X-Gm-Message-State: AODbwcAG2dnrLdzNMZwgC5H47KmasYEQeU1G5dY8L1qSl+U9VZFYrqiT B44d3ePcTZFbA2zUuAb2gp6dx5AZEvE+ X-Received: by 10.36.17.197 with SMTP id 188mr528492itf.28.1494325760716; Tue, 09 May 2017 03:29:20 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.174.167 with HTTP; Tue, 9 May 2017 03:29:20 -0700 (PDT) In-Reply-To: References: From: Dave Page Date: Tue, 9 May 2017 11:29:20 +0100 Message-ID: Subject: Re: [pgAdmin4][Patch][RM2257]: Query tool - Insert row doesn't use default values To: Surinder Kumar Cc: pgadmin-hackers Content-Type: multipart/alternative; boundary="001a11437efe258ad0054f14d474" X-Pg-Spam-Score: -2.3 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgadmin-hackers Precedence: bulk Sender: pgadmin-hackers-owner@postgresql.org --001a11437efe258ad0054f14d474 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, May 9, 2017 at 11:03 AM, Surinder Kumar < surinder.kumar@enterprisedb.com> wrote: > Hi Dave, > > On Mon, May 8, 2017 at 4:37 PM, Dave Page wrote: > >> >> >> On Mon, May 8, 2017 at 11:51 AM, Surinder Kumar < >> surinder.kumar@enterprisedb.com> wrote: >> >>> Hi >>> >>> On Mon, May 8, 2017 at 3:51 PM, Dave Page wrote: >>> >>>> Hi >>>> >>>> On Mon, May 8, 2017 at 11:13 AM, Surinder Kumar < >>>> surinder.kumar@enterprisedb.com> wrote: >>>> >>>>> Hi Dave, >>>>> >>>>> On Mon, May 8, 2017 at 3:28 PM, Dave Page wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> On Fri, May 5, 2017 at 12:52 PM, Surinder Kumar < >>>>>> surinder.kumar@enterprisedb.com> wrote: >>>>>> >>>>>>> Hi Dave, >>>>>>> >>>>>>> The support to handle [null] and [default] values is added for >>>>>>> following formatters: >>>>>>> - JsonFormatter >>>>>>> - NumbersFormatter >>>>>>> - CheckmarkFormatter >>>>>>> - TextFormatter >>>>>>> >>>>>>> Previously when a new row is added, it was not validating each and >>>>>>> every cell for columns with attribute not_null =3D true. >>>>>>> Introduced a new function validateRow(...) which is called on >>>>>>> adding a new row, it validates each cell with column having >>>>>>> attribute(not_null=3Dtrue). the corresponding cell will highlighted= and save >>>>>>> button will be disabled if value is [null]. >>>>>>> >>>>>> >>>>>> I'm not sure that behaviour is right. What I now see (given the tabl= e >>>>>> below) is that: >>>>>> >>>>>> - If I click in the id field of a new row, it forces me to enter a >>>>>> value or hit escape. Why is it trying to force me? It's a serial fie= ld, so >>>>>> will get a value on save anyway. >>>>>> >>>>> =E2=80=8BYes, It is because I am validating all cell values after it = renders. >>>>> It will reverted back. >>>>> >>>>>> >>>>>> - If I do enter a value in the ID field, focus jumps over all the >>>>>> other fields with either a default or no 'not null' option to the >>>>>> data_no_nulls column. Focus should always go to the next field. >>>>>> >>>>>> I think this addition can just be removed - I'm pretty sure the >>>>>> previous behaviour would have been what we want, with the additional >>>>>> formatters fixed. >>>>>> >>>>> =E2=80=8BBut If i remove this addition, the value for column like >>>>> 'data_no_nulls' =E2=80=8Bwill be set to '' (blank string), then on sa= ve its value >>>>> will be validated on the server side and whatever the error message i= s will >>>>> be returned back (the same behaviour as in pgAdmin3) >>>>> >>>> >>>> Which is fine I think. If you want to leave the validation there, >>>> that's also fine - but, it a) shouldn't require me to press Esc if I d= ecide >>>> not to fill in that value yet, and b) shouldn't change the tab order o= f the >>>> fields. It's also broken of course, in that it was trying to force me = to >>>> enter a value for a not-null field with a default. >>>> >>> =E2=80=8BYes, I will check if we just highlight the field and don't for= ce the >>> user to =E2=80=8Benter value. >>> This will enable user to edit any field using TAB key even if required >>> field is highlighted red. >>> Should the save button remains disable untill user enters any valid >>> value in 'data_no_nulls' column ? >>> >> >> I think that's fine, yes - as long as we get the validation right :-) >> > =E2=80=8BFor highlighting the error field and enable user to navigate to = other > cells using TAB key, > I spend some time looking and debugging into the code and found that it > requires lot of changes in slick.grid.js core functions as there are no > event listeners provided to =E2=80=8Blisten and changes in core file is n= ot > preferred. It may also break other functionalities as code is quite compl= ex. > So, I think we should validate data on server side and display any error > messages on UI. > OK. > >>>> >>> >>>>>> >>>>>>> >>>>>>> Now I will add more feature test cases for remaining formatters. >>>>>>> Will send separate patch for feature test cases once completed. >>>>>>> >>>>>> >>>>>> Thanks. >>>>>> >>>>>> >>>>>>> >>>>>>> Please review updated patch. >>>>>>> >>>>>>> >>>>>>> On Tue, May 2, 2017 at 5:57 PM, Surinder Kumar < >>>>>>> surinder.kumar@enterprisedb.com> wrote: >>>>>>> >>>>>>>> Hi Dave, >>>>>>>> >>>>>>>> On Tue, May 2, 2017 at 5:21 PM, Dave Page >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> This is looking much better now :-). Couple of thoughts and a bug= : >>>>>>>>> >>>>>>>>> - Only the TextFormatter seems to handle both [null] and [default= ] >>>>>>>>> values. Shouldn't all formatters do so (including Json and Checkm= ark)? >>>>>>>>> >>>>>>>> =E2=80=8BYes, I will apply the same changes for other formatters t= oo.=E2=80=8B >>>>>>>> >>>>>>>>> For example, "serial" columns currently get displayed as [null] >>>>>>>>> when left blank, but I would expect to see [default]. >>>>>>>>> >>>>>>>>> - I would suggest we put [null] and [default] in a lighter colour >>>>>>>>> - #999999. >>>>>>>>> >>>>>>>>> - With the feature test patch added, I seem to be consistently >>>>>>>>> getting the following failure (immediately after your new tests r= un): >>>>>>>>> >>>>>>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>>>> ERROR: runTest (pgadmin.feature_tests.xss_che >>>>>>>>> cks_panels_and_query_tool_test.CheckForXssFeatureTest) >>>>>>>>> Test XSS check for panels and query tool >>>>>>>>> ------------------------------------------------------------ >>>>>>>>> ---------- >>>>>>>>> Traceback (most recent call last): >>>>>>>>> File "/Users/dpage/git/pgadmin4/web >>>>>>>>> /regression/feature_utils/base_feature_test.py", line 42, in setU= p >>>>>>>>> self._screenshot() >>>>>>>>> File "/Users/dpage/git/pgadmin4/web >>>>>>>>> /regression/feature_utils/base_feature_test.py", line 92, in >>>>>>>>> _screenshot >>>>>>>>> python_version)) >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdr= iver.py", >>>>>>>>> line 802, in get_screenshot_as_file >>>>>>>>> png =3D self.get_screenshot_as_png() >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdr= iver.py", >>>>>>>>> line 821, in get_screenshot_as_png >>>>>>>>> return base64.b64decode(self.get_scre >>>>>>>>> enshot_as_base64().encode('ascii')) >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdr= iver.py", >>>>>>>>> line 831, in get_screenshot_as_base64 >>>>>>>>> return self.execute(Command.SCREENSHOT)['value'] >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdr= iver.py", >>>>>>>>> line 238, in execute >>>>>>>>> self.error_handler.check_response(response) >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/error= handler.py", >>>>>>>>> line 193, in check_response >>>>>>>>> raise exception_class(message, screen, stacktrace) >>>>>>>>> UnexpectedAlertPresentException: Alert Text: None >>>>>>>>> Message: unexpected alert open: {Alert text : Are you sure you >>>>>>>>> wish to close the pgAdmin 4 browser?} >>>>>>>>> (Session info: chrome=3D58.0.3029.81) >>>>>>>>> (Driver info: chromedriver=3D2.29.461585 >>>>>>>>> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=3DMac OS X >>>>>>>>> 10.12.3 x86_64) >>>>>>>>> >>>>>>>>> >>>>>>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>>>>>> ERROR: runTest (pgadmin.feature_tests.xss_che >>>>>>>>> cks_pgadmin_debugger_test.CheckDebuggerForXssFeatureTest) >>>>>>>>> Test table DDL generation >>>>>>>>> ------------------------------------------------------------ >>>>>>>>> ---------- >>>>>>>>> Traceback (most recent call last): >>>>>>>>> File "/Users/dpage/git/pgadmin4/web >>>>>>>>> /regression/feature_utils/base_feature_test.py", line 42, in setU= p >>>>>>>>> self._screenshot() >>>>>>>>> File "/Users/dpage/git/pgadmin4/web >>>>>>>>> /regression/feature_utils/base_feature_test.py", line 92, in >>>>>>>>> _screenshot >>>>>>>>> python_version)) >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdr= iver.py", >>>>>>>>> line 802, in get_screenshot_as_file >>>>>>>>> png =3D self.get_screenshot_as_png() >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdr= iver.py", >>>>>>>>> line 821, in get_screenshot_as_png >>>>>>>>> return base64.b64decode(self.get_scre >>>>>>>>> enshot_as_base64().encode('ascii')) >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdr= iver.py", >>>>>>>>> line 831, in get_screenshot_as_base64 >>>>>>>>> return self.execute(Command.SCREENSHOT)['value'] >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdr= iver.py", >>>>>>>>> line 238, in execute >>>>>>>>> self.error_handler.check_response(response) >>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/error= handler.py", >>>>>>>>> line 193, in check_response >>>>>>>>> raise exception_class(message, screen, stacktrace) >>>>>>>>> UnexpectedAlertPresentException: Alert Text: None >>>>>>>>> Message: unexpected alert open: {Alert text : Are you sure you >>>>>>>>> wish to close the pgAdmin 4 browser?} >>>>>>>>> (Session info: chrome=3D58.0.3029.81) >>>>>>>>> (Driver info: chromedriver=3D2.29.461585 >>>>>>>>> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=3DMac OS X >>>>>>>>> 10.12.3 x86_64) >>>>>>>>> >>>>>>>> =E2=80=8BSure. I will fix this.=E2=80=8B >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Apr 28, 2017 at 10:19 AM, Surinder Kumar < >>>>>>>>> surinder.kumar@enterprisedb.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Dave, >>>>>>>>>> >>>>>>>>>> Please find updated patch for RM case and a separate patch for >>>>>>>>>> Feature tests. >>>>>>>>>> >>>>>>>>>> *Python:* >>>>>>>>>> >>>>>>>>>> - Added [default] label for cells with default values while >>>>>>>>>> inserting a new row. >>>>>>>>>> >>>>>>>>>> - Introduced a FieldValidator function for cells that don't allo= w >>>>>>>>>> null values. If user tries to insert null value, field with be h= ighlighted >>>>>>>>>> with red borders around. >>>>>>>>>> >>>>>>>>>> =E2=80=8B- >>>>>>>>>> If a cell contains blank string('') and when we set it to null, >>>>>>>>>> the change into the cell is not detected. It was because the com= parison >>>>>>>>>> for (defaultValue =3D=3D null) return true if defaultValue is >>>>>>>>>> undefined. Hence _.isNull(value) is used to fix this. >>>>>>>>>> >>>>>>>>>> *Feature Test cases:* >>>>>>>>>> >>>>>>>>>> - Introduced a new method create_table_with_query(server, >>>>>>>>>> db_name, query) in test_utils.py which executes the given query >>>>>>>>>> on connected server. >>>>>>>>>> >>>>>>>>>> - Added a new file test_data.json that has test data for test >>>>>>>>>> cases. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Apr 7, 2017 at 2:21 PM, Dave Page >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> On Sat, Apr 1, 2017 at 12:45 PM, Surinder Kumar >>>>>>>>>>> wrote: >>>>>>>>>>> > Hi >>>>>>>>>>> > >>>>>>>>>>> > Issues fixed: >>>>>>>>>>> > >>>>>>>>>>> > 1. If a column is defined with a default modifier, there is >>>>>>>>>>> now way to >>>>>>>>>>> > insert the row with those defaults. >>>>>>>>>>> > The column will be left blank and it will take default value >>>>>>>>>>> automatically. >>>>>>>>>>> > >>>>>>>>>>> > 2. If a column has a not-null constraint then an error is >>>>>>>>>>> returned and the >>>>>>>>>>> > row is not inserted. >>>>>>>>>>> > The column will be left blank >>>>>>>>>>> > >>>>>>>>>>> > The default values for new added rows will be displayed on >>>>>>>>>>> refresh/execute. >>>>>>>>>>> > >>>>>>>>>>> > Please find attached patch and review. >>>>>>>>>>> >>>>>>>>>>> This largely works as expected, but there is some weirdness. I >>>>>>>>>>> have a >>>>>>>>>>> test table that looks like this: >>>>>>>>>>> >>>>>>>>>>> CREATE TABLE public.defaults >>>>>>>>>>> ( >>>>>>>>>>> id bigint NOT NULL DEFAULT nextval('defaults_id_seq'::reg >>>>>>>>>>> class), >>>>>>>>>>> data_default_nulls text COLLATE pg_catalog."default" DEFAUL= T >>>>>>>>>>> 'abc123'::text, >>>>>>>>>>> data_default_no_nulls text COLLATE pg_catalog."default" NOT >>>>>>>>>>> NULL >>>>>>>>>>> DEFAULT 'def456'::text, >>>>>>>>>>> data_nulls text COLLATE pg_catalog."default", >>>>>>>>>>> data_no_nulls text COLLATE pg_catalog."default" NOT NULL, >>>>>>>>>>> CONSTRAINT defaults_pkey PRIMARY KEY (id) >>>>>>>>>>> ) >>>>>>>>>>> >>>>>>>>>>> Remember that the expected behaviour is: >>>>>>>>>>> >>>>>>>>>>> - Set a value to empty to update the column to null. >>>>>>>>>>> - Set a value to '' to update the column to an empty string >>>>>>>>>>> - Set a value to anything else to update the column to that val= ue >>>>>>>>>>> >>>>>>>>>>> 1) In a row with values in each column, if I try to set the >>>>>>>>>>> value of >>>>>>>>>>> data_default_nulls to null, the query executed is: >>>>>>>>>>> >>>>>>>>>>> UPDATE public.defaults SET >>>>>>>>>>> data_default_nulls =3D '' WHERE >>>>>>>>>>> id =3D '2'; >>>>>>>>>>> >>>>>>>>>>> 2) If I do the same in the data_nulls column, the value is >>>>>>>>>>> immediately >>>>>>>>>>> shown as [null] and the query executed is: >>>>>>>>>>> >>>>>>>>>>> UPDATE public.defaults SET >>>>>>>>>>> data_nulls =3D NULL WHERE >>>>>>>>>>> id =3D '2'; >>>>>>>>>>> >>>>>>>>>>> 3) If I then edit the value in data_default_nulls, it shows the >>>>>>>>>>> current value as ''. Removing the quotes (to set it to null) >>>>>>>>>>> doesn't >>>>>>>>>>> get detected as a change. >>>>>>>>>>> >>>>>>>>>> =E2=80=8B=E2=80=8BTaken care. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 4) When I manually executed "update defaults set >>>>>>>>>>> data_default_nulls =3D >>>>>>>>>>> null where id =3D 2" in a query tool window, I got: >>>>>>>>>>> >>>>>>>>>>> 2017-04-07 09:43:02,987: INFO werkzeug: 127.0.0.1 - - >>>>>>>>>>> [07/Apr/2017 >>>>>>>>>>> 09:43:02] "GET /sqleditor/columns/8745675 HTTP/1.1" 500 - >>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>>>> dmin4/lib/python2.7/site-packages/flask/app.py", >>>>>>>>>>> line 2000, in __call__ >>>>>>>>>>> return self.wsgi_app(environ, start_response) >>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>>>> dmin4/lib/python2.7/site-packages/flask/app.py", >>>>>>>>>>> line 1991, in wsgi_app >>>>>>>>>>> response =3D self.make_response(self.handle_exception(e)) >>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>>>> dmin4/lib/python2.7/site-packages/flask/app.py", >>>>>>>>>>> line 1567, in handle_exception >>>>>>>>>>> reraise(exc_type, exc_value, tb) >>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>>>> dmin4/lib/python2.7/site-packages/flask/app.py", >>>>>>>>>>> line 1988, in wsgi_app >>>>>>>>>>> response =3D self.full_dispatch_request() >>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>>>> dmin4/lib/python2.7/site-packages/flask/app.py", >>>>>>>>>>> line 1641, in full_dispatch_request >>>>>>>>>>> rv =3D self.handle_user_exception(e) >>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>>>> dmin4/lib/python2.7/site-packages/flask/app.py", >>>>>>>>>>> line 1544, in handle_user_exception >>>>>>>>>>> reraise(exc_type, exc_value, tb) >>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>>>> dmin4/lib/python2.7/site-packages/flask/app.py", >>>>>>>>>>> line 1639, in full_dispatch_request >>>>>>>>>>> rv =3D self.dispatch_request() >>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>>>> dmin4/lib/python2.7/site-packages/flask/app.py", >>>>>>>>>>> line 1625, in dispatch_request >>>>>>>>>>> return self.view_functions[rule.endpoint](**req.view_args) >>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga >>>>>>>>>>> dmin4/lib/python2.7/site-packages/flask_login.py", >>>>>>>>>>> line 792, in decorated_view >>>>>>>>>>> return func(*args, **kwargs) >>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web >>>>>>>>>>> /pgadmin/tools/sqleditor/__init__.py", >>>>>>>>>>> line 452, in get_columns >>>>>>>>>>> tid=3Dcommand_obj.obj_id) >>>>>>>>>>> AttributeError: 'QueryToolCommand' object has no attribute >>>>>>>>>>> 'obj_id' >>>>>>>>>>> >>>>>>>>>> =E2=80=8BFixed.=E2=80=8B >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 5) When I run the query again in pgAdmin III, then refresh the >>>>>>>>>>> data in >>>>>>>>>>> pgAdmin 4, the data_default_nulls column is displayed without t= he >>>>>>>>>>> [null] marker (despite having a null value, which I confirmed i= n >>>>>>>>>>> pgAdmin 3). >>>>>>>>>>> >>>>>>>>>> =E2=80=8BFixed.=E2=80=8B >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I'm sure there are other combinations of issues here. Please fi= x >>>>>>>>>>> and >>>>>>>>>>> thoroughly re-test to ensure behaviour is consistent - and to >>>>>>>>>>> avoid >>>>>>>>>>> future issues, please add some appropriate feature tests to che= ck >>>>>>>>>>> nulls, defaults and empty strings are properly handled in view, >>>>>>>>>>> insert >>>>>>>>>>> and updates. Murtuza recently wrote some feature tests for the >>>>>>>>>>> query >>>>>>>>>>> tool - you should be able to use those as a starting point. >>>>>>>>>>> >>>>>>>>>> =E2=80=8BAdded feature tests=E2=80=8B >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Dave Page >>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>> Twitter: @pgsnake >>>>>>>>> >>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > --=20 Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company --001a11437efe258ad0054f14d474 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Tue, May 9, 2017 at 11:03 AM, Surinder Kumar <<= a href=3D"mailto:surinder.kumar@enterprisedb.com" target=3D"_blank">surinde= r.kumar@enterprisedb.com> wrote:
Hi Dave,

O= n Mon, May 8, 2017 at 4:37 PM, Dave Page <dpage@pgadmin.org>= wrote:


On Mon, May 8, 2017 at 11:51 AM, Sur= inder Kumar <surinder.kumar@enterprisedb.com> wrote:
Hi

On Mon, May 8, 2017 at 3:51 PM, Dave Page <dpage@pgadmin= .org> wrote:
Hi

On Mon, May 8, 2017 at 11:13 AM, Surinder = Kumar <surinder.kumar@enterprisedb.com> w= rote:
Hi Dave,

On Mon, May= 8, 2017 at 3:28 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi
=
On Fri, May 5, 2017 at 12:52 PM,= Surinder Kumar <surinder.kumar@enterprisedb.com>= ; wrote:
Hi Dave,

The support to handle [null] and [default]= values is added for following formatters:
=C2=A0-=C2=A0JsonFormatter
=C2=A0-= =C2=A0NumbersFormatter
=C2=A0-=C2=A0Che= ckmarkFormatter
=C2=A0-=C2=A0TextFormat= ter

Previously when a new row is added, it was not validating each and ev= ery cell for columns with attribute not_null =3D true.
Introduced= a new function=C2=A0validateRow(...)=C2=A0which is called on adding a n= ew row, it validates each cell with column having attribute(not_null=3Dtrue= ). the corresponding cell will highlighted and save button will be disabled= if value is [null].

I&#= 39;m not sure that behaviour is right. What I now see (given the table belo= w) is that:

- If I click in the id field of a new = row, it forces me to enter a value or hit escape. Why is it trying to force= me? It's a serial field, so will get a value on save anyway.
=E2=80=8BY= es, It is because I am validating all cell values after it renders. It will= reverted back.

- If I do enter= a value in the ID field, focus jumps over all the other fields with either= a default or no 'not null' option to the data_no_nulls column. Foc= us should always go to the next field.

I think thi= s addition can just be removed - I'm pretty sure the previous behaviour= would have been what we want, with the additional formatters fixed.=C2=A0<= /div>
= =E2=80=8BBut If i remove this addition, the value for column like 'data= _no_nulls' =E2=80=8Bwill be set to '' (blank string), then on s= ave its value will be validated on the server side and whatever the error m= essage is will be returned back (the same behaviour as in pgAdmin3)

Which is fine I thi= nk. If you want to leave the validation there, that's also fine - but, = it a) shouldn't require me to press Esc if I decide not to fill in that= value yet, and b) shouldn't change the tab order of the fields. It'= ;s also broken of course, in that it was trying to force me to enter a valu= e for a not-null field with a default.
=
=E2=80=8BYes, I will check if we just= highlight the field and don't force the user to =E2=80=8Benter value.<= /div>
This will enable user to edit any field= using TAB key even if required field is highlighted red.
Should the save button remains disable untill user ent= ers any valid value in 'data_no_nulls' column ?

I think that's fin= e, yes - as long as we get the validation right :-)=C2=A0
=
=E2=80=8BFor = highlighting the error field and enable user to navigate to other cells usi= ng TAB key,
I spend some time looking a= nd debugging into the code and found that it requires lot of changes in sli= ck.grid.js core functions as there are no event listeners provided to =E2= =80=8Blisten and changes in core file is not preferred. It may also break o= ther functionalities as code is quite complex.
So, I think we should validate data on server side and display any = error messages on UI.

OK.
=C2=A0
=
=C2=A0
=C2=A0

Now I will add more feature test cas= es for remaining formatters. Will send separate patch for feature test case= s once completed.

Thanks= .
=C2=A0

Please review updated patch.


On Tue, May 2, 2017 at 5:57 PM, Surinder= Kumar <surinder.kumar@enterprisedb.com> = wrote:
= Hi Dave,

On Tue, May 2, 2017 at 5:21 PM, Dave Page <dpage@pgadmin.org> wrote:
Hi,

Th= is is looking much better now :-). Couple of thoughts and a bug:
=
- Only the TextFormatter seems to handle both [null] and [de= fault] values. Shouldn't all formatters do so (including Json and Check= mark)?
=E2=80= =8BYes, I will apply the same changes for other formatters too.=E2=80=8B
For example, "serial" column= s currently get displayed as [null] when left blank, but I would expect to = see [default].=C2=A0

- I would suggest we put [nul= l] and [default] in a lighter colour - #999999.

- = With the feature test patch added, I seem to be consistently getting the fo= llowing failure (immediately after your new tests run):

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D
ERROR: runTest (pgadmin.feature_tests.xss_checks_p= anels_and_query_tool_test.CheckForXssFeatureTest)
Test XSS c= heck for panels and query tool
----------------------------------------------------------------------
Traceback (m= ost recent call last):
=C2=A0 File "/Users/dpage/git/= pgadmin4/web/regression/feature_utils/base_feature_test.py",= line 42, in setUp
=C2=A0 =C2=A0 self._screenshot()
=C2= =A0 File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/= base_feature_test.py", line 92, in _screenshot
=C2=A0 = =C2=A0 python_version))
=C2=A0 File "/Users/dpage/.virtualen= vs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote= /webdriver.py", line 802, in get_screenshot_as_file
=C2= =A0 =C2=A0 png =3D self.get_screenshot_as_png()
=C2=A0 File "= ;/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/s= elenium/webdriver/remote/webdriver.py", line 821, in get_screensh= ot_as_png
=C2=A0 =C2=A0 return base64.b64decode(self.get_screenshot_as_base64().encode('ascii'))
=C2=A0 File &qu= ot;/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages= /selenium/webdriver/remote/webdriver.py", line 831, in get_screen= shot_as_base64
=C2=A0 =C2=A0 return self.execute(Command.SCREENSH= OT)['value']
=C2=A0 File "/Users/dpage/.virtual= envs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remo= te/webdriver.py", line 238, in execute
=C2=A0 =C2=A0 se= lf.error_handler.check_response(response)
=C2=A0 File "= /Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/se= lenium/webdriver/remote/errorhandler.py", line 193, in check_resp= onse
=C2=A0 =C2=A0 raise exception_class(message, screen, stacktr= ace)
UnexpectedAlertPresentException: Alert Text: None
=
Message: unexpected alert open: {Alert text : Are you sure you wish to= close the pgAdmin 4 browser?}
=C2=A0 (Session info: chrome=3D58.= 0.3029.81)
=C2=A0 (Driver info: chromedriver=3D2.29.461585 (0be2c= d95f834e9ee7c46bcc7cf405b483f5ae83b),platform=3DMac OS X 10.12.3 x86_6= 4)


=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
ERROR: runTest (pgadmin= .feature_tests.xss_checks_pgadmin_debugger_test.CheckDebuggerForX= ssFeatureTest)
Test table DDL generation
--------------= --------------------------------------------------------
Traceback (most recent call last):
=C2=A0 File &qu= ot;/Users/dpage/git/pgadmin4/web/regression/feature_utils/base_fe= ature_test.py", line 42, in setUp
=C2=A0 =C2=A0 self._screen= shot()
=C2=A0 File "/Users/dpage/git/pgadmin4/web/regre= ssion/feature_utils/base_feature_test.py", line 92, in _screensho= t
=C2=A0 =C2=A0 python_version))
=C2=A0 File "/Use= rs/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/seleni= um/webdriver/remote/webdriver.py", line 802, in get_screenshot_as= _file
=C2=A0 =C2=A0 png =3D self.get_screenshot_as_png()
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/s= ite-packages/selenium/webdriver/remote/webdriver.py", line 8= 21, in get_screenshot_as_png
=C2=A0 =C2=A0 return base64.b64decod= e(self.get_screenshot_as_base64().encode('ascii'))
<= div>=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7= /site-packages/selenium/webdriver/remote/webdriver.py", line= 831, in get_screenshot_as_base64
=C2=A0 =C2=A0 return self.execu= te(Command.SCREENSHOT)['value']
=C2=A0 File "/U= sers/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/sele= nium/webdriver/remote/webdriver.py", line 238, in execute
=C2=A0 =C2=A0 self.error_handler.check_response(response)
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/si= te-packages/selenium/webdriver/remote/errorhandler.py", line= 193, in check_response
=C2=A0 =C2=A0 raise exception_class(messa= ge, screen, stacktrace)
UnexpectedAlertPresentException: Ale= rt Text: None
Message: unexpected alert open: {Alert text : Are y= ou sure you wish to close the pgAdmin 4 browser?}
=C2=A0 (Session= info: chrome=3D58.0.3029.81)
=C2=A0 (Driver info: chromedriver= =3D2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=3DM= ac OS X 10.12.3 x86_64)
=E2=80=8BSure. I will fix this.=E2=80=8B

Thanks!
<= br>

On Fri, = Apr 28, 2017 at 10:19 AM, Surinder Kumar <surinder.kumar@ent= erprisedb.com> wrote:
Hi Dave,

Please find updated patch for = RM case and a separate patch for Feature tests.

Python:

- Added= =C2=A0[default] label for cells with default= values while inserting a new row.

- Introduced a FieldValidator f= unction for cells that don't allow null values. If user tries to insert= null value, field with be highlighted with red =C2=A0 =C2=A0borders around= .

=E2=80=8B-=C2=A0
If a cell contains blank string(&#= 39;') and when we set it to null, the change into the cell is not detec= ted. It was because the comparison
for (defaultValue =3D=3D null) return= true if defaultValue is undefined. Hence _.isNull(value) is used to fix th= is.

Feature Test cases:

=C2=A0- Introduced a new method=C2=A0create_table_with_que= ry(server, db_name, query)=C2=A0 in test_utils.py which execute= s the given query on connected server.

=C2=A0- Add= ed a new file test_data.json that has test data for test cases.
<= br>

On Fri, Apr 7, 2017 at 2:21 PM, Dave Page= <dpage@pgadmin.org> wrote:
Hi

On Sat, Apr 1, 2017 at 12:45 PM, Surinder Kumar
<su= rinder.kumar@enterprisedb.com> wrote:
> Hi
>
> Issues fixed:
>
> 1. If a column is defined with a default modifier, there is now way to=
> insert the row with those defaults.
> The column will be left blank and it will take default value automatic= ally.
>
> 2. If a column has a not-null constraint then an error is returned and= the
> row is not inserted.
> The column will be left blank
>
> The default values for new added rows will be displayed on refresh/exe= cute.
>
> Please find attached patch and review.

This largely works as expected, but there is some weirdness. I have = a
test table that looks like this:

CREATE TABLE public.defaults
(
=C2=A0 =C2=A0 id bigint NOT NULL DEFAULT nextval('defaults_id_seq':= :regclass),
=C2=A0 =C2=A0 data_default_nulls text COLLATE pg_catalog."default"= ; DEFAULT 'abc123'::text,
=C2=A0 =C2=A0 data_default_no_nulls text COLLATE pg_catalog."default&q= uot; NOT NULL
DEFAULT 'def456'::text,
=C2=A0 =C2=A0 data_nulls text COLLATE pg_catalog."default",
=C2=A0 =C2=A0 data_no_nulls text COLLATE pg_catalog."default" NOT= NULL,
=C2=A0 =C2=A0 CONSTRAINT defaults_pkey PRIMARY KEY (id)
)

Remember that the expected behaviour is:

- Set a value to empty to update the column to null.
- Set a value to '' to update the column to an empty string
- Set a value to anything else to update the column to that value

1) In a row with values in each column, if I try to set the value of
data_default_nulls to null, the query executed is:

UPDATE public.defaults SET
data_default_nulls =3D '' WHERE
id =3D '2';

2) If I do the same in the data_nulls column, the value is immediately
shown as [null] and the query executed is:

UPDATE public.defaults SET
data_nulls =3D NULL WHERE
id =3D '2';

3) If I then edit the value in data_default_nulls, it shows the
current value as ''. Removing the quotes (to set it to null) doesn&= #39;t
get detected as a change.
=E2=80=8B=E2=80=8BTaken care.=C2=A0

4) When I manually executed "update defaults set data_default_nulls = =3D
null where id =3D 2" in a query tool window, I got:

2017-04-07 09:43:02,987: INFO werkzeug: 127.0.0.1 - - [07/Apr/2017
09:43:02] "GET /sqleditor/columns/8745675 HTTP/1.1" 500 -
Traceback (most recent call last):
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/sit= e-packages/flask/app.py",
line 2000, in __call__
=C2=A0 =C2=A0 return self.wsgi_app(environ, start_response)
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/sit= e-packages/flask/app.py",
line 1991, in wsgi_app
=C2=A0 =C2=A0 response =3D self.make_response(self.handle_exception(e)= )
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/sit= e-packages/flask/app.py",
line 1567, in handle_exception
=C2=A0 =C2=A0 reraise(exc_type, exc_value, tb)
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/sit= e-packages/flask/app.py",
line 1988, in wsgi_app
=C2=A0 =C2=A0 response =3D self.full_dispatch_request()
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/sit= e-packages/flask/app.py",
line 1641, in full_dispatch_request
=C2=A0 =C2=A0 rv =3D self.handle_user_exception(e)
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/sit= e-packages/flask/app.py",
line 1544, in handle_user_exception
=C2=A0 =C2=A0 reraise(exc_type, exc_value, tb)
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/sit= e-packages/flask/app.py",
line 1639, in full_dispatch_request
=C2=A0 =C2=A0 rv =3D self.dispatch_request()
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/sit= e-packages/flask/app.py",
line 1625, in dispatch_request
=C2=A0 =C2=A0 return self.view_functions[rule.endpoint](**req.view_arg= s)
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/sit= e-packages/flask_login.py",
line 792, in decorated_view
=C2=A0 =C2=A0 return func(*args, **kwargs)
=C2=A0 File "/Users/dpage/git/pgadmin4/web/pgadmin/tools/sqledito= r/__init__.py",
line 452, in get_columns
=C2=A0 =C2=A0 tid=3Dcommand_obj.obj_id)
AttributeError: 'QueryToolCommand' object has no attribute 'obj= _id'
=E2=80= =8BFixed.=E2=80=8B

5) When I run the query again in pgAdmin III, then refresh the data in
pgAdmin 4, the data_default_nulls column is displayed without the
[null] marker (despite having a null value, which I confirmed in
pgAdmin 3).
=E2=80=8B= Fixed.=E2=80=8B

I'm sure there are other combinations of issues here. Please fix and thoroughly re-test to ensure behaviour is consistent - and to avoid
future issues, please add some appropriate feature tests to check
nulls, defaults and empty strings are properly handled in view, insert
and updates. Murtuza recently wrote some feature tests for the query
tool - you should be able to use those as a starting point.
=E2=80=8BAdded feature tests=E2=80= =8B

Thanks.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twit= ter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise Post= greSQL Company



<= br clear=3D"all">

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake<= br>
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company=




--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB= UK: http://www.e= nterprisedb.com
The Enterprise PostgreSQL Company




--
Dave Page
Blog: http://pgsnake.blogspot.com<= br>Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterpri= se PostgreSQL Company




--
Dave Page
Blog: = http://pgsnake.bl= ogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com<= br>The Enterprise PostgreSQL Company
--001a11437efe258ad0054f14d474--