Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1geGez-0008Cu-1k for pgadmin-hackers@arkaria.postgresql.org; Tue, 01 Jan 2019 09:48:29 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1geGex-0005Ni-JI for pgadmin-hackers@arkaria.postgresql.org; Tue, 01 Jan 2019 09:48:27 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1geGew-0005Mu-NI for pgadmin-hackers@lists.postgresql.org; Tue, 01 Jan 2019 09:48:27 +0000 Received: from mail-qk1-x72b.google.com ([2607:f8b0:4864:20::72b]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1geGes-0000WJ-64 for pgadmin-hackers@postgresql.org; Tue, 01 Jan 2019 09:48:25 +0000 Received: by mail-qk1-x72b.google.com with SMTP id 189so16530831qkj.8 for ; Tue, 01 Jan 2019 01:48:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=2WSAUstt7iWFZAVCAt6CEQSoTQ3DlweqK7cX7ewavXU=; b=YPgLTcQTcW8MD5nee+XUYtKrI7lCbnU+GqTdpt99spTOUD+fUbuH5YEqlTWCwjYxSw aoYa5n/44XV1qNwglQsKqQwZHixAzIdaDxogv7nxHBf2yKrY+le0E+mlHND7hNeHC7BY J38BKazOTXuKyoU3UeUDWdoTXJbbYwL4S/Oc78oo5YKOZ69ACBGmmdWwhRSQzvPfkduu kszRba4ZZfzoj+aZzVPeQTAIuLMtPYLDNi4oKFvGK1sFkMCwkYqcd8TNs08wZ/weI7CJ NoPnAVvUEtnbuM9in+ZL+G77aDrcUTO5PfzAc2i9J/ENAKMs2UNzIYZRXY5Y0pOK/uEP nXPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2WSAUstt7iWFZAVCAt6CEQSoTQ3DlweqK7cX7ewavXU=; b=I0FrjhhX48UYYHsFAG5zgldttopMGvUBsoi8oU9bRQVMN99wn00Eg/ro4rJZ+1NRQA j69Xflwcc2pekVqj8AJ1Sj804nb+yk1aiBzaMpI8klQH1642Sh7hocsLboR9GF8mX+JX yvuvqQ63X5cd0xj/c6jaLVaDueeQQcyVP//NuGBnt2WTMiGcHHzlEmmDYZMOPKc67Vyk JgGqnKIajiBYeWcg2LNWPAo+TlxPSFLZi1rc921uRU7CFr9JT5+Q8GxkTVPNYFJC/+5h xIR1fDIn3WAnoJS1bZP5XCJ2sD9+VuYp27oLPBImSxZa2iebaXH+n/bI6ZkF3JdprTG/ 3DtQ== X-Gm-Message-State: AJcUukdsQ4QGrmWKEVbYkxcvkrs5OusXSAiiWZQvC5JrZPuMUsg60zvb d9feqkOGe5Noz+iXOXrGK9TgDVB1zFt3CJCCsQWTmv7VamU= X-Google-Smtp-Source: ALg8bN4URdd7dAODZVAk9vN3H60UcKTJjOKrr6x3tSsSxTEXFeBb0Fgc+i3CDtej72fsXhUbNu7e2FqqQsWyJudxj6o= X-Received: by 2002:a37:8541:: with SMTP id h62mr37291356qkd.213.1546336100226; Tue, 01 Jan 2019 01:48:20 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Akshay Joshi Date: Tue, 1 Jan 2019 15:18:08 +0530 Message-ID: Subject: Re: [pgAdmin4][Patch] - RM 3780 pgAdmin4 lacks ability to specify NULL values in CSV export To: Dave Page Cc: pgadmin-hackers Content-Type: multipart/alternative; boundary="000000000000f53471057e626dd0" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --000000000000f53471057e626dd0 Content-Type: text/plain; charset="UTF-8" Hi Dave/Hackers On Mon, Dec 24, 2018 at 7:47 AM Akshay Joshi wrote: > > > On Fri, 21 Dec 2018, 19:45 Dave Page >> >> >> On Fri, Dec 21, 2018 at 2:08 PM Akshay Joshi < >> akshay.joshi@enterprisedb.com> wrote: >> >>> >>> >>> On Fri, 21 Dec 2018, 18:52 Dave Page >> >>>> Hi >>>> >>>> Looks good - I just found one issue; please go ahead and commit (with a >>>> release notes update) when fixed. >>>> >>>> - If I set the NULL replacement value to an empty string, it defaults >>>> back to NULL. It's therefore not possible to replace null values with an >>>> empty value. >>>> >>> >>> If you remember this is the default behaviour of the Preferences >>> dialog. It sets the default value again if you set the blank value. Fahar >>> has raised the same issue before and we have rejected that. If you think we >>> can create the new RM for that behaviour and will commit the current patch. >>> >> >> Well, we rejected it for a specific case. In this specific case, it makes >> sense to be able to enter a blank value. >> >> Maybe we just need an "allow blank" flag on the preference object, that >> does nothing except allow you to remove the default (or any other) value >> without it being replaced again? Obviously the default still needs to work >> until a value is written to the config DB. >> > > Ok. If i understand correctly this new parameter works for all the > preference parameter if set to true. By default it's value is False and if > user wants to enter blank value they will ser the flag. > I think we should not add new preferences flag "allow blank". Meanwhile I have added new parameter "allow_blank" to the register function for preferences, so at the time of registering the preference we can set the value of "allow_blank" parameter, if blank value is acceptable. I have committed my patch with this changes. If you think we need that flag as a Preferences object then I'll create a new RM for it. > >> >>> >>>> Thanks! >>>> >>>> On Fri, Dec 21, 2018 at 12:57 PM Akshay Joshi < >>>> akshay.joshi@enterprisedb.com> wrote: >>>> >>>>> Hi Dave >>>>> >>>>> I have modified the condition, it won't through any exception. >>>>> Attached is the updated patch, please review it. >>>>> >>>>> On Fri, Dec 21, 2018 at 4:22 AM Dave Page wrote: >>>>> >>>>>> Sorry! Here is is. >>>>>> >>>>>> On Fri, Dec 21, 2018 at 12:12 PM Akshay Joshi < >>>>>> akshay.joshi@enterprisedb.com> wrote: >>>>>> >>>>>>> Hi Dave >>>>>>> >>>>>>> Can you please attached the updated patch with your changes. I'll >>>>>>> try to fix the exception. >>>>>>> >>>>>>> On Fri, Dec 21, 2018 at 3:57 AM Dave Page wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> Here's an updated patch as I've tweaked some of the wording. The >>>>>>>> screenshot probably isn't the right resolution, but as we're replacing them >>>>>>>> anyway it doesn't seem overly important. Feel free to fix if you like :-) >>>>>>>> >>>>>>>> With quoting set to either All or Strings, everything looks good. >>>>>>>> With it set to None, I still get an exception (below). The query I'm using >>>>>>>> is this: >>>>>>>> >>>>>>>> SELECT NULL::text, 1234::int, 'Foo bar'::text, E'Foo\nBar'::text >>>>>>>> >>>>>>>> Field separator: , >>>>>>>> Quote character: " >>>>>>>> Replace null's with: NULL >>>>>>>> >>>>>>>> Steps: >>>>>>>> >>>>>>>> 1) Run pgAdmin in Desktop mode. I'm running from within PyuCharms, >>>>>>>> using the venv detailed below. >>>>>>>> 2) Open the Query Tool on a PostgreSQL 9.6.10 database, running on >>>>>>>> MacOS 10.14.1 >>>>>>>> 3) Run the above query, wit quoting set to All and check the result >>>>>>>> in the grid. >>>>>>>> 4) Download the CSV file and check. >>>>>>>> 5) Open Preferences and set quoting to Strings. >>>>>>>> 6) Download the CSV file and check. >>>>>>>> 7) Open Preferences and set quoting to None. >>>>>>>> 8) Download the CSV file *exception occurs*. >>>>>>>> >>>>>>>> >>>>>>>> System info: >>>>>>>> >>>>>>>> (pgadmin4) dpage@hal:*~/git/pgadmin4*$ python --version >>>>>>>> >>>>>>>> Python 3.6.7 >>>>>>>> >>>>>>>> (pgadmin4) dpage@hal:*~/git/pgadmin4*$ pip freeze >>>>>>>> >>>>>>>> alabaster==0.7.11 >>>>>>>> >>>>>>>> alembic==1.0.0 >>>>>>>> >>>>>>>> asn1crypto==0.24.0 >>>>>>>> >>>>>>>> Babel==2.6.0 >>>>>>>> >>>>>>>> bcrypt==3.1.4 >>>>>>>> >>>>>>>> blinker==1.4 >>>>>>>> >>>>>>>> certifi==2018.8.24 >>>>>>>> >>>>>>>> cffi==1.11.5 >>>>>>>> >>>>>>>> chardet==3.0.4 >>>>>>>> >>>>>>>> chromedriver-installer==0.0.6 >>>>>>>> >>>>>>>> click==6.7 >>>>>>>> >>>>>>>> cryptography==2.3 >>>>>>>> >>>>>>>> docutils==0.14 >>>>>>>> >>>>>>>> extras==1.0.0 >>>>>>>> >>>>>>>> fixtures==3.0.0 >>>>>>>> >>>>>>>> Flask==0.12.4 >>>>>>>> >>>>>>>> Flask-BabelEx==0.9.3 >>>>>>>> >>>>>>>> Flask-Gravatar==0.5.0 >>>>>>>> >>>>>>>> Flask-HTMLmin==1.3.2 >>>>>>>> >>>>>>>> Flask-Login==0.3.2 >>>>>>>> >>>>>>>> Flask-Mail==0.9.1 >>>>>>>> >>>>>>>> Flask-Migrate==2.1.1 >>>>>>>> >>>>>>>> Flask-Paranoid==0.2.0 >>>>>>>> >>>>>>>> Flask-Principal==0.4.0 >>>>>>>> >>>>>>>> Flask-Security==3.0.0 >>>>>>>> >>>>>>>> Flask-SQLAlchemy==2.3.2 >>>>>>>> >>>>>>>> Flask-WTF==0.14.2 >>>>>>>> >>>>>>>> html5lib==1.0.1 >>>>>>>> >>>>>>>> htmlmin==0.1.12 >>>>>>>> >>>>>>>> idna==2.7 >>>>>>>> >>>>>>>> imagesize==1.1.0 >>>>>>>> >>>>>>>> itsdangerous==0.24 >>>>>>>> >>>>>>>> Jinja2==2.10 >>>>>>>> >>>>>>>> linecache2==1.0.0 >>>>>>>> >>>>>>>> Mako==1.0.7 >>>>>>>> >>>>>>>> MarkupSafe==1.0 >>>>>>>> >>>>>>>> packaging==18.0 >>>>>>>> >>>>>>>> paramiko==2.4.1 >>>>>>>> >>>>>>>> passlib==1.7.1 >>>>>>>> >>>>>>>> pbr==3.1.1 >>>>>>>> >>>>>>>> psutil==5.4.8 >>>>>>>> >>>>>>>> psycopg2==2.7.5 >>>>>>>> >>>>>>>> pyasn1==0.4.4 >>>>>>>> >>>>>>>> pycodestyle==2.3.1 >>>>>>>> >>>>>>>> pycparser==2.18 >>>>>>>> >>>>>>>> pycrypto==2.6.1 >>>>>>>> >>>>>>>> Pygments==2.2.0 >>>>>>>> >>>>>>>> PyNaCl==1.2.1 >>>>>>>> >>>>>>>> pyparsing==2.2.2 >>>>>>>> >>>>>>>> pyperclip==1.6.4 >>>>>>>> >>>>>>>> pyrsistent==0.14.2 >>>>>>>> >>>>>>>> python-dateutil==2.7.3 >>>>>>>> >>>>>>>> python-editor==1.0.3 >>>>>>>> >>>>>>>> python-mimeparse==1.6.0 >>>>>>>> >>>>>>>> pytz==2018.3 >>>>>>>> >>>>>>>> requests==2.19.1 >>>>>>>> >>>>>>>> selenium==3.14.1 >>>>>>>> >>>>>>>> simplejson==3.13.2 >>>>>>>> >>>>>>>> six==1.11.0 >>>>>>>> >>>>>>>> snowballstemmer==1.2.1 >>>>>>>> >>>>>>>> speaklater==1.3 >>>>>>>> >>>>>>>> Sphinx==1.8.2 >>>>>>>> >>>>>>>> sphinxcontrib-websupport==1.1.0 >>>>>>>> >>>>>>>> SQLAlchemy==1.2.10 >>>>>>>> >>>>>>>> sqlparse==0.2.4 >>>>>>>> >>>>>>>> sshtunnel==0.1.4 >>>>>>>> >>>>>>>> testscenarios==0.5.0 >>>>>>>> >>>>>>>> testtools==2.3.0 >>>>>>>> >>>>>>>> traceback2==1.4.0 >>>>>>>> >>>>>>>> unittest2==1.1.0 >>>>>>>> >>>>>>>> urllib3==1.23 >>>>>>>> >>>>>>>> webencodings==0.5.1 >>>>>>>> >>>>>>>> Werkzeug==0.14.1 >>>>>>>> >>>>>>>> WTForms==2.1 >>>>>>>> >>>>>>>> Exception: >>>>>>>> >>>>>>>> 2018-12-21 11:47:28,995: SQL pgadmin: Execute (with server cursor) >>>>>>>> for server #2 - CONN:8760231 (Query-id: 8649354): >>>>>>>> SELECT NULL::text, 1234::int, 'Foo bar'::text, E'Foo\nBar'::text >>>>>>>> 2018-12-21 11:47:29,001: INFO werkzeug: 127.0.0.1 - - [21/Dec/2018 >>>>>>>> 11:47:29] "GET >>>>>>>> /sqleditor/query_tool/download/2133388?query=SELECT%20NULL%3A%3Atext%2C%201234%3A%3Aint%2C%20%27Foo%20bar%27%3A%3Atext%2C%20E%27Foo%5CnBar%27%3A%3Atext&filename=data-1545392848979.csv >>>>>>>> HTTP/1.1" 500 - >>>>>>>> 2018-12-21 11:47:29,003: ERROR werkzeug: Error on request: >>>>>>>> Traceback (most recent call last): >>>>>>>> File >>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/serving.py", >>>>>>>> line 270, in run_wsgi >>>>>>>> execute(self.server.app) >>>>>>>> File >>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/serving.py", >>>>>>>> line 260, in execute >>>>>>>> for data in application_iter: >>>>>>>> File >>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/wsgi.py", >>>>>>>> line 870, in __next__ >>>>>>>> return self._next() >>>>>>>> File >>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/wrappers.py", >>>>>>>> line 82, in _iter_encoded >>>>>>>> for item in iterable: >>>>>>>> File >>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py", >>>>>>>> line 848, in gen >>>>>>>> csv_writer.writerows(results) >>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", line >>>>>>>> 761, in writerows >>>>>>>> return self.writer.writerows(map(self._dict_to_list, rowdicts)) >>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", line >>>>>>>> 268, in writerows >>>>>>>> self.writerow(row) >>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", line >>>>>>>> 261, in writerow >>>>>>>> row = [self.strategy.prepare(field, only=only) for field in row] >>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", line >>>>>>>> 261, in >>>>>>>> row = [self.strategy.prepare(field, only=only) for field in row] >>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", line >>>>>>>> 142, in prepare >>>>>>>> raise Error('No escapechar is set') >>>>>>>> _csv.Error: No escapechar is set >>>>>>>> >>>>>>>> On Thu, Dec 20, 2018 at 1:05 PM Akshay Joshi < >>>>>>>> akshay.joshi@enterprisedb.com> wrote: >>>>>>>> >>>>>>>>> Hi Dave >>>>>>>>> >>>>>>>>> On Thu, Dec 20, 2018 at 5:12 PM Akshay Joshi < >>>>>>>>> akshay.joshi@enterprisedb.com> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Dec 20, 2018 at 4:48 PM Dave Page >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi >>>>>>>>>>> >>>>>>>>>>> On Thu, Dec 20, 2018 at 10:09 AM Akshay Joshi < >>>>>>>>>>> akshay.joshi@enterprisedb.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Dave >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Dec 20, 2018 at 3:08 PM Dave Page >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi >>>>>>>>>>>>> >>>>>>>>>>>>> When testing with quoting set to None, quote = " and delimiter >>>>>>>>>>>>> = , I get the following exception when I try to download: >>>>>>>>>>>>> >>>>>>>>>>>>> 2018-12-20 09:34:02,547: SQL pgadmin: Execute (with server >>>>>>>>>>>>> cursor) for server #2 - CONN:354106 (Query-id: 4121147): >>>>>>>>>>>>> SELECT NULL::text, 1234::int, 'Foo bar'::text, >>>>>>>>>>>>> E'Foo\nBar'::text >>>>>>>>>>>>> 2018-12-20 09:34:02,570: INFO werkzeug: 127.0.0.1 - - >>>>>>>>>>>>> [20/Dec/2018 09:34:02] "GET >>>>>>>>>>>>> /sqleditor/query_tool/download/5610522?query=SELECT%20NULL%3A%3Atext%2C%201234%3A%3Aint%2C%20%27Foo%20bar%27%3A%3Atext%2C%20E%27Foo%5CnBar%27%3A%3Atext&filename=data-1545298442530.csv >>>>>>>>>>>>> HTTP/1.1" 500 - >>>>>>>>>>>>> 2018-12-20 09:34:02,572: ERROR werkzeug: Error on request: >>>>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>>>> File >>>>>>>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/serving.py", >>>>>>>>>>>>> line 270, in run_wsgi >>>>>>>>>>>>> execute(self.server.app) >>>>>>>>>>>>> File >>>>>>>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/serving.py", >>>>>>>>>>>>> line 260, in execute >>>>>>>>>>>>> for data in application_iter: >>>>>>>>>>>>> File >>>>>>>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/wsgi.py", >>>>>>>>>>>>> line 870, in __next__ >>>>>>>>>>>>> return self._next() >>>>>>>>>>>>> File >>>>>>>>>>>>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/wrappers.py", >>>>>>>>>>>>> line 82, in _iter_encoded >>>>>>>>>>>>> for item in iterable: >>>>>>>>>>>>> File >>>>>>>>>>>>> "/Users/dpage/git/pgadmin4/web/pgadmin/utils/driver/psycopg2/connection.py", >>>>>>>>>>>>> line 820, in gen >>>>>>>>>>>>> csv_writer.writerows(results) >>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", >>>>>>>>>>>>> line 748, in writerows >>>>>>>>>>>>> return self.writer.writerows(map(self._dict_to_list, >>>>>>>>>>>>> rowdicts)) >>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", >>>>>>>>>>>>> line 256, in writerows >>>>>>>>>>>>> self.writerow(row) >>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", >>>>>>>>>>>>> line 249, in writerow >>>>>>>>>>>>> row = [self.strategy.prepare(field, only=only) for field >>>>>>>>>>>>> in row] >>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", >>>>>>>>>>>>> line 249, in >>>>>>>>>>>>> row = [self.strategy.prepare(field, only=only) for field >>>>>>>>>>>>> in row] >>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", >>>>>>>>>>>>> line 136, in prepare >>>>>>>>>>>>> raise Error('No escapechar is set') >>>>>>>>>>>>> _csv.Error: No escapechar is set >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Not able to reproduce the above issue. I have tested it >>>>>>>>>>>> with the same setting as you mentioned. Please refer all the attached >>>>>>>>>>>> screenshots. Please specify the steps if they are different. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> When I have quoting set to All, the first column is returned >>>>>>>>>>>>> as "" >>>>>>>>>>>>> >>>>>>>>>>>>> dpage@hal:*~/Downloads*$ more data-1545298598112.csv >>>>>>>>>>>>> >>>>>>>>>>>>> "text","int4","text-2","text-3" >>>>>>>>>>>>> >>>>>>>>>>>>> "","1234","Foo bar","Foo >>>>>>>>>>>>> >>>>>>>>>>>>> Bar" >>>>>>>>>>>>> >>>>>>>>>>>>> Isn't the point for it to be NULL? >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> while quoting is set to ALL, all the data types has been >>>>>>>>>>>> quoted, so I thought null values should be replaced by "" instead of blank. >>>>>>>>>>>> But if you think null values shouldn't be quoted even if user select quote >>>>>>>>>>>> ALL, I'll fix it and resend the patch. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> So how would you distinguish NULL from an empty string? Isn't >>>>>>>>>>> that exactly what the bug is about? >>>>>>>>>>> >>>>>>>>>>> I still think we need a "Replace NULLs with" config option, and >>>>>>>>>>> regardless of quoting settings we always replace NULL values with whatever >>>>>>>>>>> that is set to - for which the user could then choose options like: >>>>>>>>>>> >>>>>>>>>>> NULL >>>>>>>>>>> "NULL" >>>>>>>>>>> "" >>>>>>>>>>> '' >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> We would never quote the NULL replacement value - if the user >>>>>>>>>>> wanted it to be quoted, they would include the quotes in the configured >>>>>>>>>>> string. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK, Will work on it and send the modified patch again. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Attached is the modified patch as per your suggestion. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Dec 18, 2018 at 11:13 AM Akshay Joshi < >>>>>>>>>>>>> akshay.joshi@enterprisedb.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Dave >>>>>>>>>>>>>> >>>>>>>>>>>>>> Attached is the modified patch to fix review comments. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Dec 18, 2018 at 3:00 PM Akshay Joshi < >>>>>>>>>>>>>> akshay.joshi@enterprisedb.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Dec 18, 2018 at 2:49 PM Dave Page >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tue, Dec 18, 2018 at 3:45 AM Akshay Joshi < >>>>>>>>>>>>>>>> akshay.joshi@enterprisedb.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Hackers, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Attached is the patch to fix RM #3780 pgAdmin4 lacks >>>>>>>>>>>>>>>>> ability to specify NULL values in CSV export. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please review it. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> A few points; >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - You've included code from backports.csv, but per the >>>>>>>>>>>>>>>> licence you need to include a description of the changes made. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sure. In that case I'll copy the complete file and >>>>>>>>>>>>>>> will do my changes which is of two lines only. With my patch I have remove >>>>>>>>>>>>>>> all the unwanted code from backport.csv. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - Shouldn't backports.csv be removed from requirements.txt, >>>>>>>>>>>>>>>> or is it used elsewhere? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes. Will do that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - If the previous point is true, then I'm fairly sure there >>>>>>>>>>>>>>>> is code in one or more of the many package build scripts that adds an >>>>>>>>>>>>>>>> __init__.py file to backports.csv in the venv that's created. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'll remove that code as well. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Dave Page >>>>>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> *Akshay Joshi* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> *Sr. Software Architect * >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> *Akshay Joshi* >>>>>>>>>>>>>> >>>>>>>>>>>>>> *Sr. Software Architect * >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Dave Page >>>>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>>>> >>>>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Akshay Joshi* >>>>>>>>>>>> >>>>>>>>>>>> *Sr. Software Architect * >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Dave Page >>>>>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>>>>> Twitter: @pgsnake >>>>>>>>>>> >>>>>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>>>>> The Enterprise PostgreSQL Company >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Akshay Joshi* >>>>>>>>>> >>>>>>>>>> *Sr. Software Architect * >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Akshay Joshi* >>>>>>>>> >>>>>>>>> *Sr. Software Architect * >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Akshay Joshi* >>>>>>> >>>>>>> *Sr. Software Architect * >>>>>>> >>>>>>> >>>>>>> >>>>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Akshay Joshi* >>>>> >>>>> *Sr. Software Architect * >>>>> >>>>> >>>>> >>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > -- *Akshay Joshi* *Sr. Software Architect * *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* --000000000000f53471057e626dd0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Dave/Hackers

On Mon, Dec 24, 2018 at 7:47 AM Akshay Joshi <akshay.joshi@enterprisedb.= com> wrote:


On Fri, 21 Dec 2018, 19:45 Dave Page <dpage@pgadmin.org wrote:

=

On Fri, Dec 21, 2018 = at 2:08 PM Akshay Joshi <akshay.joshi@enterprisedb.com>= ; wrote:


On Fri,= 21 Dec 2018, 18:52 Dave Page <dpage@pgadmin.org wrote:
Hi

=
Looks good - I just found one issue; please go ahead and commit = (with a release notes update) when fixed.

- If I s= et the NULL replacement value to an empty string, it defaults back to NULL.= It's therefore not possible to replace null values with an empty value= .

=C2=A0 =C2=A0 If you remember this is the default behaviour of th= e Preferences dialog. It sets the default value again if you set the blank = value. Fahar has raised the same issue before and we have rejected that. If= you think we can create the new RM for that behaviour and will commit the = current patch.

Well, we rejecte= d it for a specific case. In this specific case, it makes sense to be able = to enter a blank value.

Maybe we just need an &quo= t;allow blank" flag on the preference object, that does nothing except= allow you to remove the default (or any other) value without it being repl= aced again? Obviously the default still needs to work until a value is writ= ten to the config DB.=C2=A0

=C2=A0 =C2=A0 Ok. If i understand = correctly this new parameter works for all the preference parameter if set = to true. By default it's value is False and if user wants to enter blan= k value they will ser the flag.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 I think we should not add new preferences flag= "allow blank". Meanwhile I have added new parameter "allow_= blank" to the register function for preferences, so at the time of reg= istering the preference we can set the value of "allow_blank" par= ameter, if blank value is acceptable. I have committed my patch with this c= hanges.=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0If you th= ink we need that flag as a Preferences object then I'll create a new RM= for it.=C2=A0 =C2=A0=C2=A0
=C2=A0
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft:1px solid rgb(204,204,204);padding-left:1ex">

=
Thanks!

On Fri, Dec 21, 2018 at 12:57 PM Akshay Joshi <= akshay.joshi@enterprisedb.com> wrote:
Hi Dave

I have modified the condition, it won't through any exception.= Attached is the updated patch, please review it.

On Fri, Dec 21, 2018 at 4:22 AM Dave= Page <dpage@pgadmin.org> wrote:
Sorry! Here is is.
On Fri, Dec 21, 2018 at = 12:12 PM Akshay Joshi <akshay.joshi@enterprisedb.c= om> wrote:
Hi Dave

Can you pleas= e attached the updated patch with your changes. I'll try to fix the exc= eption.

On F= ri, Dec 21, 2018 at 3:57 AM Dave Page <dpage@pgadmin.org&g= t; wrote:
Hi

Here's an updated patch as I've tweaked some of the wordin= g. The screenshot probably isn't the right resolution, but as we're= replacing them anyway it doesn't seem overly important. Feel free to f= ix if you like :-)

With quoting set to either All = or Strings, everything looks good. With it set to None, I still get an exce= ption (below). The query I'm using is this:

SE= LECT NULL::text, 1234::int, 'Foo bar'::text, E'Foo\nBar'::t= ext

Field separator: ,
Quote charact= er: "
Replace null's with: NULL

Steps:

1) Run pgAdmin in Desktop mode. I'm ru= nning from within PyuCharms, using the venv detailed below.
2) Op= en the Query Tool on a PostgreSQL 9.6.10 database, running on MacOS 10.14.1=
3) Run the above query, wit quoting set to All and check the res= ult in the grid.
4) Download the CSV file and check.
5)= Open Preferences and set quoting to Strings.
6) Download the CSV= file and check.
7) Open Preferences and set quoting to None.
8) Download the CSV file *exception occurs*.


System info:

(pgadmin4) <= /span>dpage@hal:~/git/pgadmin4$ python --vers= ion

Python 3.6.7=

(pgadmin4) <= /span>dpage@hal:~/git/pgadmin4$ pip freeze

alabaster=3D= =3D0.7.11

alembic=3D= =3D1.0.0

asn1crypto= =3D=3D0.24.0

Babel=3D=3D2= .6.0

bcrypt=3D=3D= 3.1.4

blinker=3D= =3D1.4

certifi=3D= =3D2018.8.24

cffi=3D=3D1.= 11.5

chardet=3D= =3D3.0.4

chromedriver= -installer=3D=3D0.0.6

click=3D=3D6= .7

cryptography= =3D=3D2.3

docutils=3D= =3D0.14

extras=3D=3D= 1.0.0

fixtures=3D= =3D3.0.0

Flask=3D=3D0= .12.4

Flask-BabelE= x=3D=3D0.9.3

Flask-Gravat= ar=3D=3D0.5.0

Flask-HTMLmi= n=3D=3D1.3.2

Flask-Login= =3D=3D0.3.2

Flask-Mail= =3D=3D0.9.1

Flask-Migrat= e=3D=3D2.1.1

Flask-Parano= id=3D=3D0.2.0

Flask-Princi= pal=3D=3D0.4.0

Flask-Securi= ty=3D=3D3.0.0

Flask-SQLAlc= hemy=3D=3D2.3.2

Flask-WTF=3D= =3D0.14.2

html5lib=3D= =3D1.0.1

htmlmin=3D= =3D0.1.12

idna=3D=3D2.= 7

imagesize=3D= =3D1.1.0

itsdangerous= =3D=3D0.24

Jinja2=3D=3D= 2.10

linecache2= =3D=3D1.0.0

Mako=3D=3D1.= 0.7

MarkupSafe= =3D=3D1.0

packaging=3D= =3D18.0

paramiko=3D= =3D2.4.1

passlib=3D= =3D1.7.1

pbr=3D=3D3.1= .1

psutil=3D=3D= 5.4.8

psycopg2=3D= =3D2.7.5

pyasn1=3D=3D= 0.4.4

pycodestyle= =3D=3D2.3.1

pycparser=3D= =3D2.18

pycrypto=3D= =3D2.6.1

Pygments=3D= =3D2.2.0

PyNaCl=3D=3D= 1.2.1

pyparsing=3D= =3D2.2.2

pyperclip=3D= =3D1.6.4

pyrsistent= =3D=3D0.14.2

python-dateu= til=3D=3D2.7.3

python-edito= r=3D=3D1.0.3

python-mimep= arse=3D=3D1.6.0

pytz=3D=3D20= 18.3

requests=3D= =3D2.19.1

selenium=3D= =3D3.14.1

simplejson= =3D=3D3.13.2

six=3D=3D1.1= 1.0

snowballstem= mer=3D=3D1.2.1

speaklater= =3D=3D1.3

Sphinx=3D=3D= 1.8.2

sphinxcontri= b-websupport=3D=3D1.1.0

SQLAlchemy= =3D=3D1.2.10

sqlparse=3D= =3D0.2.4

sshtunnel=3D= =3D0.1.4

testscenario= s=3D=3D0.5.0

testtools=3D= =3D2.3.0

traceback2= =3D=3D1.4.0

unittest2=3D= =3D1.1.0

urllib3=3D= =3D1.23

webencodings= =3D=3D0.5.1

Werkzeug=3D= =3D0.14.1

WTForms=3D= =3D2.1


Exception:

2018-12-21 11:47:28,995: SQL pgadmin: Exe= cute (with server cursor) for server #2 - CONN:8760231 (Query-id: 8649354):=
SELECT NULL::text, 1234::int, 'Foo bar'::text, E'Foo= \nBar'::text
2018-12-21 11:47:29,001: INFO werkzeug: 127.0.0.1 - - [21/Dec/2018 11:47:29] "GET /sqleditor/query_tool/d= ownload/2133388?query=3DSELECT%20NULL%3A%3Atext%2C%201234%3A%3Aint%2C%20%27= Foo%20bar%27%3A%3Atext%2C%20E%27Foo%5CnBar%27%3A%3Atext&filename=3Ddata= -1545392848979.csv HTTP/1.1" 500 -
2018-12-21 11:47:29,003: = ERROR werkzeug: Error on request:
Traceback (most re= cent call last):
=C2=A0 File "/Users/dpage/.virtualenvs/pgad= min4/lib/python3.6/site-packages/werkzeug/serving.py", line 270, in ru= n_wsgi
=C2=A0 =C2=A0 execute(self.server.app)
=C2=A0 Fi= le "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/wer= kzeug/serving.py", line 260, in execute
=C2=A0 =C2=A0 for da= ta in application_iter:
=C2=A0 File "/Users/dpage/.virtualen= vs/pgadmin4/lib/python3.6/site-packages/werkzeug/wsgi.py", line 870, i= n __next__
=C2=A0 =C2=A0 return self._next()
=C2=A0 Fil= e "/Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werk= zeug/wrappers.py", line 82, in _iter_encoded
=C2=A0 =C2=A0 f= or item in iterable:
=C2=A0 File "/Users/dpage/git/pgadmin4/= web/pgadmin/utils/driver/psycopg2/connection.py", line 848, in gen
=C2=A0 =C2=A0 csv_writer.writerows(results)
=C2=A0 File &q= uot;/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", line 761, in = writerows
=C2=A0 =C2=A0 return self.writer.writerows(map(self._di= ct_to_list, rowdicts))
=C2=A0 File "/Users/dpage/git/pgadmin= 4/web/pgadmin/utils/csv.py", line 268, in writerows
=C2=A0 = =C2=A0 self.writerow(row)
=C2=A0 File "/Users/dpage/git/pgad= min4/web/pgadmin/utils/csv.py", line 261, in writerow
=C2=A0= =C2=A0 row =3D [self.strategy.prepare(field, only=3Donly) for field in row= ]
=C2=A0 File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/c= sv.py", line 261, in <listcomp>
=C2=A0 =C2=A0 row =3D = [self.strategy.prepare(field, only=3Donly) for field in row]
=C2= =A0 File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", li= ne 142, in prepare
=C2=A0 =C2=A0 raise Error('No escapechar i= s set')
_csv.Error: No escapechar is set

On Thu, Dec 20, 2018 at 1:05 PM Aksh= ay Joshi <akshay.joshi@enterprisedb.com> wr= ote:
Hi=C2=A0Dave

On Thu, Dec 20, 2018 at 5:12 PM Akshay Joshi <akshay.joshi@enterprisedb.com> wrote:
=

On Thu, Dec 20, 2= 018 at 4:48 PM Dave Page <dpage@pgadmin.org> wrote:
=
Hi=

On Thu, Dec 20, 2018 at= 10:09 AM Akshay Joshi <akshay.joshi@enterprisedb.= com> wrote:
Hi Dave

On Thu, Dec 20, 2018 at 3:08 PM Dave Page <dpage@pgadmin.org> wrote:
Hi

When testing with quoting set to None, quote =3D " and delimiter =3D= , I get the following exception when I try to download:

2018-12-20 09:34:02,547: SQL pgadmin: Execute (wit= h server cursor) for server #2 - CONN:354106 (Query-id: 4121147):
SELECT NULL::text, 1234::int, 'Foo bar'::text, E'Foo\nBar'= ::text
2018-12-20 09:34:02,570: INFO werkzeug: 127.0= .0.1 - - [20/Dec/2018 09:34:02] "GET /sqleditor/query_tool/download/56= 10522?query=3DSELECT%20NULL%3A%3Atext%2C%201234%3A%3Aint%2C%20%27Foo%20bar%= 27%3A%3Atext%2C%20E%27Foo%5CnBar%27%3A%3Atext&filename=3Ddata-154529844= 2530.csv HTTP/1.1" 500 -
2018-12-20 09:34:02,572: ERROR werkzeug: Error on request:
Traceback (most recent call = last):
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin4/lib/p= ython3.6/site-packages/werkzeug/serving.py", line 270, in run_wsgi
=C2=A0 =C2=A0 execute(self.server.app)
=C2=A0 File "/= Users/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/serv= ing.py", line 260, in execute
=C2=A0 =C2=A0 for data in appl= ication_iter:
=C2=A0 File "/Users/dpage/.virtualenvs/pgadmin= 4/lib/python3.6/site-packages/werkzeug/wsgi.py", line 870, in __next__=
=C2=A0 =C2=A0 return self._next()
=C2=A0 File "/U= sers/dpage/.virtualenvs/pgadmin4/lib/python3.6/site-packages/werkzeug/wrapp= ers.py", line 82, in _iter_encoded
=C2=A0 =C2=A0 for item in= iterable:
=C2=A0 File "/Users/dpage/git/pgadmin4/web/pgadmi= n/utils/driver/psycopg2/connection.py", line 820, in gen
=C2= =A0 =C2=A0 csv_writer.writerows(results)
=C2=A0 File "/Users= /dpage/git/pgadmin4/web/pgadmin/utils/csv.py", line 748, in writerows<= /div>
=C2=A0 =C2=A0 return self.writer.writerows(map(self._dict_to_list= , rowdicts))
=C2=A0 File "/Users/dpage/git/pgadmin4/web/pgad= min/utils/csv.py", line 256, in writerows
=C2=A0 =C2=A0 self= .writerow(row)
=C2=A0 File "/Users/dpage/git/pgadmin4/web/pg= admin/utils/csv.py", line 249, in writerow
=C2=A0 =C2=A0 row= =3D [self.strategy.prepare(field, only=3Donly) for field in row]
=C2=A0 File "/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py"= , line 249, in <listcomp>
=C2=A0 =C2=A0 row =3D [self.strat= egy.prepare(field, only=3Donly) for field in row]
=C2=A0 File &qu= ot;/Users/dpage/git/pgadmin4/web/pgadmin/utils/csv.py", line 136, in p= repare
=C2=A0 =C2=A0 raise Error('No escapechar is set')<= /div>
_csv.Error: No escapechar is set

=C2=A0 =C2=A0 Not able to reproduce the above issue= . I have tested it with the same setting as you mentioned. Please refer all= the attached screenshots. Please specify the steps if they are different.<= /div>

When I have quoting set to All, the = first column is returned as ""

dpage@hal:~/Downloads$ more data-1545298598112.csv=C2=A0

"text","int4",&quo= t;text-2","text-3"

"","1234","Fo= o bar","Foo

Bar"


Isn't the point for it= to be NULL?

= =C2=A0 =C2=A0 while quoting is set to ALL, all the data types has been quot= ed, so I thought null values should be replaced by "" instead of = blank. But if you think null values shouldn't be quoted even if user se= lect quote ALL, I'll fix it and resend the patch.

So how would you distinguish NULL from an empt= y string? Isn't that exactly what the bug is about?

I still think we need a "Replace NULLs with" config option,= and regardless of quoting settings we always replace NULL values with what= ever that is set to=C2=A0 - for which the user could then choose options li= ke:

NULL
"NULL"
&quo= t;"
''
<empty string>

=
We would never quote the NULL replacement value - if the user wa= nted it to be quoted, they would include the quotes in the configured strin= g.


=C2=A0 = =C2=A0OK, Will work on it and send the modified patch again.=C2=A0

=C2=A0 =C2=A0 =C2=A0 Attached is = the modified patch as per your suggestion.=C2=A0
=C2=A0

On Tue, De= c 18, 2018 at 11:13 AM Akshay Joshi <akshay.joshi@= enterprisedb.com> wrote:
Hi=C2=A0Dave

Attached i= s the modified patch to fix review comments.

On Tue, Dec 18, 2018 at 3:00 PM Akshay Joshi &= lt;akshay.joshi@enterprisedb.com> wrote:


On T= ue, Dec 18, 2018 at 2:49 PM Dave Page <dpage@pgadmin.org&g= t; wrote:
Hi

On Tue, De= c 18, 2018 at 3:45 AM Akshay Joshi <akshay.joshi@e= nterprisedb.com> wrote:
Hi Hackers,

=
Attached is the patch to fix RM #3780=C2=A0pgAdmin4 lacks ability to s= pecify NULL values in CSV export.

Please review = it.

A few points;

- You've included code from backports.csv, but per th= e licence you need to include a description of the changes made.

=C2=A0 =C2=A0 =C2=A0 Sure. In that = case I'll copy the complete file and will do my changes which is of two= lines only. With my patch I have remove all the unwanted code from backpor= t.csv.=C2=A0 =C2=A0 =C2=A0=C2=A0

- Shouldn't backports.csv be removed from requirements.txt, or is it = used elsewhere?

=C2=A0 = =C2=A0 =C2=A0Yes. Will do that.=C2=A0

- If the previous point is true, then I'm fairly sure there is c= ode in one or more of the many package build scripts that adds an __init__.= py file to backports.csv in the venv that's created.
<= /blockquote>

=C2=A0 =C2=A0 I'll remove that code as = well.=C2=A0=C2=A0
=C2=A0
--
Dave Page
= Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
=
EnterpriseDB UK: http://www.enterprisedb.com
The En= terprise PostgreSQL Company


--
Akshay Joshi<= /div>
Sr. Softwar= e Architect

Phone: +91 20-3058-9517
Mobile: +91 976-788-8246


--
Akshay Joshi
Sr. Software Architect
=

<= div>Phone: +91 20-305= 8-9517
Mobile: +91 976-788-8246


--
Dave Page
Blog: h= ttp://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK:= http://www.enterprisedb.com
The Enterprise PostgreSQL = Company


--
Akshay Joshi
Sr. Software Archi= tect

<= b>

Phone: +91 20-3058-9517
Mobile: +91 976-788-8246


--
Dave PageBlog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The E= nterprise PostgreSQL Company


--
<= div>Akshay Joshi
Sr. Software Architect


Phone: +91 20-3058-9517
Mobile: +91 976-788-8246=


--
Akshay Joshi
Sr. Software Architect <= br>


=
Pho= ne: +91 20-3058-9517
Mobile: +91 976-788-8246


--
Dave Page
Blog: <= a href=3D"http://pgsnake.blogspot.com" rel=3D"noreferrer noreferrer" target= =3D"_blank">http://pgsnake.blogspot.com
Twitter: @pgsnake

Ent= erpriseDB UK: http://www.enterprisedb.com
The Enterpris= e PostgreSQL Company


--
Akshay Joshi
=
Sr. Software Arc= hitect


Phone: +91 20-3058-9517
Mobile: +91 976-788-8246<= /div>


--
D= ave Page
Blog: http://pgsnake.blogspot.com
Twitter: = @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


--
=
Akshay Joshi
Sr. Software Architect


Phone: +91 20-3058-9517
Mobile: +91 976-788-824= 6


--
Dave Page
Blog: http://pgsnake.blogspot= .com
Twitter: @pgsnake

EnterpriseDB UK: http://w= ww.enterprisedb.com
The Enterprise PostgreSQL Company


--
Dave Page
Blog: http://pg= snake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http:= //www.enterprisedb.com
The Enterprise PostgreSQL Company


--
Akshay Joshi
Sr. Software Architect
=


Phone:= +91 20-3058-9517
Mobile: +91 976-788-8246
=
--000000000000f53471057e626dd0--