public inbox for [email protected]
help / color / mirror / Atom feed [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
19+ messages / 2 participants
[nested] [flat]
* [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2017-12-25 11:03 Khushboo Vashi <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Khushboo Vashi @ 2017-12-25 11:03 UTC (permalink / raw)
To: pgadmin-hackers
Hi,
Please find the attached patch to fix RM #2963: Backup database, Restore
database and Maintenance Database failed for é object.
Thanks,
Khushboo
Attachments:
[text/x-patch] RM_2963.patch (1.0K, 3-RM_2963.patch)
download | inline diff:
diff --git a/web/pgadmin/misc/bgprocess/processes.py b/web/pgadmin/misc/bgprocess/processes.py
index 209c2a1..a5d2655 100644
--- a/web/pgadmin/misc/bgprocess/processes.py
+++ b/web/pgadmin/misc/bgprocess/processes.py
@@ -164,12 +164,13 @@ class BatchProcess(object):
csv_writer.writerow(_args)
args_val = args_csv_io.getvalue().strip(str('\r\n'))
+ desc_str = dumps(self.desc)
j = Process(
pid=int(id), command=_cmd,
- arguments=args_val.decode('utf-8', 'replace') if IS_PY2 and hasattr(args_val, 'decode') \
- else args_val,
- logdir=log_dir, desc=dumps(self.desc), user_id=current_user.id
+ arguments=args_val.decode('utf-8', 'replace') if IS_PY2 and hasattr(args_val, 'decode') else args_val,
+ logdir=log_dir, desc=desc_str.decode('utf-8', 'replace') if IS_PY2 and hasattr(desc_str, 'decode')
+ else desc_str, user_id=current_user.id
)
db.session.add(j)
db.session.commit()
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-01-03 15:24 Dave Page <[email protected]>
parent: Khushboo Vashi <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Dave Page @ 2018-01-03 15:24 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
Hi
On Mon, Dec 25, 2017 at 11:03 AM, Khushboo Vashi <
[email protected]> wrote:
> Hi,
>
> Please find the attached patch to fix RM #2963: Backup database, Restore
> database and Maintenance Database failed for é object.
>
With the patch applied, I get:
2018-01-03 15:23:00,110: INFO pgadmin: Executing the process executor with
the arguments: ['/Users/dpage/.virtualenvs/pgadmin4/bin/python',
'/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/process_executor.py',
'/usr/local/pgsql/bin/pg_dump', u'--file', u'/Users/dpage/e.sql',
u'--host', u'localhost', u'--port', '5432', u'--username', u'postgres',
u'--no-password', u'--verbose', u'--format=c', u'--blobs', u'\xe9']
2018-01-03 15:23:00,117: INFO werkzeug: 127.0.0.1 - - [03/Jan/2018
15:23:00] "POST /backup/job/1/object HTTP/1.1" 200 -
Exception in thread Thread-6:
Traceback (most recent call last):
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
line 810, in __bootstrap_inner
self.run()
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
line 763, in run
self.__target(*self.__args, **self.__kwargs)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
line 602, in process_request_thread
self.handle_error(request, client_address)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
line 599, in process_request_thread
self.finish_request(request, client_address)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
line 655, in __init__
self.handle()
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
line 200, in handle
rv = BaseHTTPRequestHandler.handle(self)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py",
line 340, in handle
self.handle_one_request()
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
line 235, in handle_one_request
return self.run_wsgi()
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
line 177, in run_wsgi
execute(self.server.app)
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
line 165, in execute
application_iter = app(environ, start_response)
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 2000, in __call__
return self.wsgi_app(environ, start_response)
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1991, in wsgi_app
response = self.make_response(self.handle_exception(e))
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1567, in handle_exception
reraise(exc_type, exc_value, tb)
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1988, in wsgi_app
response = self.full_dispatch_request()
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1639, in full_dispatch_request
rv = self.dispatch_request()
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
line 1625, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File
"/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
line 792, in decorated_view
return func(*args, **kwargs)
File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/__init__.py",
line 62, in index
return make_response(response=BatchProcess.list())
File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/processes.py",
line 512, in list
desc = loads(p.desc)
File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
line 1381, in loads
file = StringIO(str)
UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in
position 138: ordinal not in range(128)
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-01-10 06:59 Khushboo Vashi <[email protected]>
parent: Dave Page <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Khushboo Vashi @ 2018-01-10 06:59 UTC (permalink / raw)
To: Dave Page <[email protected]>; +Cc: pgadmin-hackers
On Wed, Jan 3, 2018 at 8:54 PM, Dave Page <[email protected]> wrote:
> Hi
>
> On Mon, Dec 25, 2017 at 11:03 AM, Khushboo Vashi <
> [email protected]> wrote:
>
>> Hi,
>>
>> Please find the attached patch to fix RM #2963: Backup database, Restore
>> database and Maintenance Database failed for é object.
>>
>
> With the patch applied, I get:
>
> 2018-01-03 15:23:00,110: INFO pgadmin: Executing the process executor
> with the arguments: ['/Users/dpage/.virtualenvs/pgadmin4/bin/python',
> '/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/process_executor.py',
> '/usr/local/pgsql/bin/pg_dump', u'--file', u'/Users/dpage/e.sql',
> u'--host', u'localhost', u'--port', '5432', u'--username', u'postgres',
> u'--no-password', u'--verbose', u'--format=c', u'--blobs', u'\xe9']
> 2018-01-03 15:23:00,117: INFO werkzeug: 127.0.0.1 - - [03/Jan/2018
> 15:23:00] "POST /backup/job/1/object HTTP/1.1" 200 -
> Exception in thread Thread-6:
> Traceback (most recent call last):
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
> line 810, in __bootstrap_inner
> self.run()
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
> line 763, in run
> self.__target(*self.__args, **self.__kwargs)
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/
> lib/python2.7/SocketServer.py", line 602, in process_request_thread
> self.handle_error(request, client_address)
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/
> lib/python2.7/SocketServer.py", line 599, in process_request_thread
> self.finish_request(request, client_address)
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/
> lib/python2.7/SocketServer.py", line 334, in finish_request
> self.RequestHandlerClass(request, client_address, self)
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/
> lib/python2.7/SocketServer.py", line 655, in __init__
> self.handle()
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
> line 200, in handle
> rv = BaseHTTPRequestHandler.handle(self)
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/
> lib/python2.7/BaseHTTPServer.py", line 340, in handle
> self.handle_one_request()
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
> line 235, in handle_one_request
> return self.run_wsgi()
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
> line 177, in run_wsgi
> execute(self.server.app)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
> line 165, in execute
> application_iter = app(environ, start_response)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 2000, in __call__
> return self.wsgi_app(environ, start_response)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1991, in wsgi_app
> response = self.make_response(self.handle_exception(e))
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1567, in handle_exception
> reraise(exc_type, exc_value, tb)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1988, in wsgi_app
> response = self.full_dispatch_request()
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1641, in full_dispatch_request
> rv = self.handle_user_exception(e)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1544, in handle_user_exception
> reraise(exc_type, exc_value, tb)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1639, in full_dispatch_request
> rv = self.dispatch_request()
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
> line 1625, in dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
> line 792, in decorated_view
> return func(*args, **kwargs)
> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/__init__.py",
> line 62, in index
> return make_response(response=BatchProcess.list())
> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/processes.py",
> line 512, in list
> desc = loads(p.desc)
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
> line 1381, in loads
> file = StringIO(str)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in
> position 138: ordinal not in range(128)
>
> Fixed. Please find the attached updated patch.
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
Attachments:
[text/x-patch] RM_2963_ver1.patch (1.3K, 3-RM_2963_ver1.patch)
download | inline diff:
diff --git a/web/pgadmin/misc/bgprocess/processes.py b/web/pgadmin/misc/bgprocess/processes.py
index 39bb8e3..3f23a99 100644
--- a/web/pgadmin/misc/bgprocess/processes.py
+++ b/web/pgadmin/misc/bgprocess/processes.py
@@ -164,12 +164,13 @@ class BatchProcess(object):
csv_writer.writerow(_args)
args_val = args_csv_io.getvalue().strip(str('\r\n'))
+ desc_str = dumps(self.desc)
j = Process(
pid=int(id), command=_cmd,
- arguments=args_val.decode('utf-8', 'replace') if IS_PY2 and hasattr(args_val, 'decode') \
- else args_val,
- logdir=log_dir, desc=dumps(self.desc), user_id=current_user.id
+ arguments=args_val.decode('utf-8', 'replace') if IS_PY2 and hasattr(args_val, 'decode') else args_val,
+ logdir=log_dir, desc=desc_str.decode('utf-8', 'ignore') if IS_PY2 and hasattr(desc_str, 'decode')
+ else desc_str, user_id=current_user.id
)
db.session.add(j)
db.session.commit()
@@ -508,6 +509,7 @@ class BatchProcess(object):
etime = parser.parse(p.end_time or get_current_time())
execution_time = (etime - stime).total_seconds()
+ p.desc = p.desc.encode('utf-8', 'ignore') if IS_PY2 and hasattr(p.desc, 'decode') else p.desc
desc = loads(p.desc)
details = desc
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-01-12 09:54 Dave Page <[email protected]>
parent: Khushboo Vashi <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Dave Page @ 2018-01-12 09:54 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
Hi
On Wed, Jan 10, 2018 at 6:59 AM, Khushboo Vashi <
[email protected]> wrote:
>
>
> On Wed, Jan 3, 2018 at 8:54 PM, Dave Page <[email protected]> wrote:
>
>> Hi
>>
>> On Mon, Dec 25, 2017 at 11:03 AM, Khushboo Vashi <
>> [email protected]> wrote:
>>
>>> Hi,
>>>
>>> Please find the attached patch to fix RM #2963: Backup database, Restore
>>> database and Maintenance Database failed for é object.
>>>
>>
>> With the patch applied, I get:
>>
>> 2018-01-03 15:23:00,110: INFO pgadmin: Executing the process executor
>> with the arguments: ['/Users/dpage/.virtualenvs/pgadmin4/bin/python',
>> '/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/process_executor.py',
>> '/usr/local/pgsql/bin/pg_dump', u'--file', u'/Users/dpage/e.sql',
>> u'--host', u'localhost', u'--port', '5432', u'--username', u'postgres',
>> u'--no-password', u'--verbose', u'--format=c', u'--blobs', u'\xe9']
>> 2018-01-03 15:23:00,117: INFO werkzeug: 127.0.0.1 - - [03/Jan/2018
>> 15:23:00] "POST /backup/job/1/object HTTP/1.1" 200 -
>> Exception in thread Thread-6:
>> Traceback (most recent call last):
>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
>> line 810, in __bootstrap_inner
>> self.run()
>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
>> line 763, in run
>> self.__target(*self.__args, **self.__kwargs)
>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>> line 602, in process_request_thread
>> self.handle_error(request, client_address)
>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>> line 599, in process_request_thread
>> self.finish_request(request, client_address)
>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>> line 334, in finish_request
>> self.RequestHandlerClass(request, client_address, self)
>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>> line 655, in __init__
>> self.handle()
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>> line 200, in handle
>> rv = BaseHTTPRequestHandler.handle(self)
>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/li
>> b/python2.7/BaseHTTPServer.py", line 340, in handle
>> self.handle_one_request()
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>> line 235, in handle_one_request
>> return self.run_wsgi()
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>> line 177, in run_wsgi
>> execute(self.server.app)
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>> line 165, in execute
>> application_iter = app(environ, start_response)
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 2000, in __call__
>> return self.wsgi_app(environ, start_response)
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1991, in wsgi_app
>> response = self.make_response(self.handle_exception(e))
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1567, in handle_exception
>> reraise(exc_type, exc_value, tb)
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1988, in wsgi_app
>> response = self.full_dispatch_request()
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1641, in full_dispatch_request
>> rv = self.handle_user_exception(e)
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1544, in handle_user_exception
>> reraise(exc_type, exc_value, tb)
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1639, in full_dispatch_request
>> rv = self.dispatch_request()
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>> line 1625, in dispatch_request
>> return self.view_functions[rule.endpoint](**req.view_args)
>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
>> line 792, in decorated_view
>> return func(*args, **kwargs)
>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/__init__.py",
>> line 62, in index
>> return make_response(response=BatchProcess.list())
>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/processes.py",
>> line 512, in list
>> desc = loads(p.desc)
>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
>> line 1381, in loads
>> file = StringIO(str)
>> UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in
>> position 138: ordinal not in range(128)
>>
>> Fixed. Please find the attached updated patch.
>
>>
That's getting further, but still not right. Please see the attached
screenshot - the beginning of the executed command has been lost.
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[image/png] Screen Shot 2018-01-12 at 09.52.48.png (197.9K, 3-Screen%20Shot%202018-01-12%20at%2009.52.48.png)
download | view image
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-09 03:54 Khushboo Vashi <[email protected]>
parent: Dave Page <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Khushboo Vashi @ 2018-03-09 03:54 UTC (permalink / raw)
To: Dave Page <[email protected]>; +Cc: pgadmin-hackers
Hi,
Please find the attached patch to fix below issues:
1. #2963 - Backup database, Restore database and Maintenance Database
failed for é object
2. #3157 - Process viewer doesn't show complete command executed.
Test cases are not included for these fixes as we don't have test cases for
these modules (backup, restore, maintenance).
I will create one separate RM for the same which will cover this.
Thanks,
Khushboo
On Fri, Jan 12, 2018 at 3:24 PM, Dave Page <[email protected]> wrote:
> Hi
>
> On Wed, Jan 10, 2018 at 6:59 AM, Khushboo Vashi <
> [email protected]> wrote:
>
>>
>>
>> On Wed, Jan 3, 2018 at 8:54 PM, Dave Page <[email protected]> wrote:
>>
>>> Hi
>>>
>>> On Mon, Dec 25, 2017 at 11:03 AM, Khushboo Vashi <
>>> [email protected]> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find the attached patch to fix RM #2963: Backup database,
>>>> Restore database and Maintenance Database failed for é object.
>>>>
>>>
>>> With the patch applied, I get:
>>>
>>> 2018-01-03 15:23:00,110: INFO pgadmin: Executing the process executor
>>> with the arguments: ['/Users/dpage/.virtualenvs/pgadmin4/bin/python',
>>> '/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/process_executor.py',
>>> '/usr/local/pgsql/bin/pg_dump', u'--file', u'/Users/dpage/e.sql',
>>> u'--host', u'localhost', u'--port', '5432', u'--username', u'postgres',
>>> u'--no-password', u'--verbose', u'--format=c', u'--blobs', u'\xe9']
>>> 2018-01-03 15:23:00,117: INFO werkzeug: 127.0.0.1 - - [03/Jan/2018
>>> 15:23:00] "POST /backup/job/1/object HTTP/1.1" 200 -
>>> Exception in thread Thread-6:
>>> Traceback (most recent call last):
>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
>>> line 810, in __bootstrap_inner
>>> self.run()
>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
>>> line 763, in run
>>> self.__target(*self.__args, **self.__kwargs)
>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>>> line 602, in process_request_thread
>>> self.handle_error(request, client_address)
>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>>> line 599, in process_request_thread
>>> self.finish_request(request, client_address)
>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>>> line 334, in finish_request
>>> self.RequestHandlerClass(request, client_address, self)
>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py",
>>> line 655, in __init__
>>> self.handle()
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>>> line 200, in handle
>>> rv = BaseHTTPRequestHandler.handle(self)
>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/li
>>> b/python2.7/BaseHTTPServer.py", line 340, in handle
>>> self.handle_one_request()
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>>> line 235, in handle_one_request
>>> return self.run_wsgi()
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>>> line 177, in run_wsgi
>>> execute(self.server.app)
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/werkzeug/serving.py",
>>> line 165, in execute
>>> application_iter = app(environ, start_response)
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 2000, in __call__
>>> return self.wsgi_app(environ, start_response)
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1991, in wsgi_app
>>> response = self.make_response(self.handle_exception(e))
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1567, in handle_exception
>>> reraise(exc_type, exc_value, tb)
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1988, in wsgi_app
>>> response = self.full_dispatch_request()
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1641, in full_dispatch_request
>>> rv = self.handle_user_exception(e)
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1544, in handle_user_exception
>>> reraise(exc_type, exc_value, tb)
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1639, in full_dispatch_request
>>> rv = self.dispatch_request()
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask/app.py",
>>> line 1625, in dispatch_request
>>> return self.view_functions[rule.endpoint](**req.view_args)
>>> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/flask_login.py",
>>> line 792, in decorated_view
>>> return func(*args, **kwargs)
>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/__init__.py",
>>> line 62, in index
>>> return make_response(response=BatchProcess.list())
>>> File "/Users/dpage/git/pgadmin4/web/pgadmin/misc/bgprocess/processes.py",
>>> line 512, in list
>>> desc = loads(p.desc)
>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py",
>>> line 1381, in loads
>>> file = StringIO(str)
>>> UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in
>>> position 138: ordinal not in range(128)
>>>
>>> Fixed. Please find the attached updated patch.
>>
>>>
> That's getting further, but still not right. Please see the attached
> screenshot - the beginning of the executed command has been lost.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
Attachments:
[text/x-patch] RM_2963_3157.patch (5.1K, 3-RM_2963_3157.patch)
download | inline diff:
diff --git a/web/pgadmin/misc/bgprocess/processes.py b/web/pgadmin/misc/bgprocess/processes.py
index 02b953d..385b7bd 100644
--- a/web/pgadmin/misc/bgprocess/processes.py
+++ b/web/pgadmin/misc/bgprocess/processes.py
@@ -78,10 +78,20 @@ class BatchProcess(object):
_("Could not find a process with the specified ID.")
)
+ try:
+ tmp_desc = loads(p.desc.encode('latin-1')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except UnicodeDecodeError:
+ tmp_desc = loads(p.desc.encode('utf-8')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except Exception as e:
+ tmp_desc = loads(p.desc.encode('utf-8', 'ignore')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+
# ID
self.id = _id
# Description
- self.desc = loads(p.desc)
+ self.desc = tmp_desc
# Status Acknowledged time
self.atime = p.acknowledge
# Command
@@ -171,6 +181,16 @@ class BatchProcess(object):
csv_writer.writerow(_args)
args_val = args_csv_io.getvalue().strip(str('\r\n'))
+ tmp_desc = dumps(self.desc)
+ try:
+ tmp_desc =tmp_desc.decode('utf-8') if\
+ IS_PY2 and hasattr(tmp_desc, 'decode') else tmp_desc
+ except UnicodeDecodeError:
+ tmp_desc = tmp_desc.decode('latin-1') if \
+ IS_PY2 and hasattr(tmp_desc, 'decode') else tmp_desc
+ except Exception:
+ tmp_desc = tmp_desc.decode('utf-8', 'ignore') if \
+ IS_PY2 and hasattr(tmp_desc, 'decode') else tmp_desc
j = Process(
pid=int(id),
@@ -178,7 +198,7 @@ class BatchProcess(object):
arguments=args_val.decode('utf-8', 'replace')
if IS_PY2 and hasattr(args_val, 'decode') else args_val,
logdir=log_dir,
- desc=dumps(self.desc),
+ desc=tmp_desc,
user_id=current_user.id
)
db.session.add(j)
@@ -534,7 +554,17 @@ class BatchProcess(object):
etime = parser.parse(p.end_time or get_current_time())
execution_time = (etime - stime).total_seconds()
- desc = loads(p.desc)
+ desc = ""
+ try:
+ desc = loads(p.desc.encode('latin-1')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except UnicodeDecodeError:
+ desc = loads(p.desc.encode('utf-8')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except Exception:
+ desc = loads(p.desc.encode('utf-8', 'ignore')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+
details = desc
if isinstance(desc, IProcessDesc):
diff --git a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
index 5e30590..8ad2c65 100644
--- a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
+++ b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
@@ -248,7 +248,7 @@ define('misc.bgprocess', [
if (!self.notifier) {
var header = $('<div></div>', {
class: 'h5 pg-bg-notify-header',
- }).append($('<span></span>').text(self.desc)),
+ }).append($('<span></span>').text(_.escape(self.desc))),
content = $('<div class="pg-bg-bgprocess row"></div>').append(
header
).append(
@@ -366,7 +366,7 @@ define('misc.bgprocess', [
self.logs[0].scrollTop = self.logs[0].scrollHeight;
});
// set bgprocess detailed description
- $header.find('.bg-detailed-desc').html(self.detailed_desc);
+ $header.find('.bg-detailed-desc').html(_.unescape(self.detailed_desc));
}
// set bgprocess start time
@@ -562,4 +562,4 @@ define('misc.bgprocess', [
});
return pgBrowser.BackgroundProcessObsorver;
-});
\ No newline at end of file
+});
diff --git a/web/pgadmin/tools/backup/__init__.py b/web/pgadmin/tools/backup/__init__.py
index 715ff6e..9e70c4f 100644
--- a/web/pgadmin/tools/backup/__init__.py
+++ b/web/pgadmin/tools/backup/__init__.py
@@ -97,11 +97,9 @@ class BackupMessage(IProcessDesc):
def cmdArg(x):
if x:
- # x = html.safe_str(x)
x = x.replace('\\', '\\\\')
x = x.replace('"', '\\"')
x = x.replace('""', '\\"')
-
return ' "' + x + '"'
return ''
@@ -111,6 +109,7 @@ class BackupMessage(IProcessDesc):
else:
self.cmd += cmdArg(arg)
+
@property
def message(self):
# Fetch the server details like hostname, port, roles etc
@@ -189,7 +188,7 @@ class BackupMessage(IProcessDesc):
res += '</div><div class="h5">'
res += _("Running command:")
res += '</b><br><span class="pg-bg-cmd enable-selection">'
- res += html.safe_str(self.cmd)
+ res += html.safe_str(cmd + self.cmd)
res += '</span></div>'
return res
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-09 15:32 Dave Page <[email protected]>
parent: Khushboo Vashi <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Dave Page @ 2018-03-09 15:32 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
Hi
On Fri, Mar 9, 2018 at 3:54 AM, Khushboo Vashi <
[email protected]> wrote:
> Hi,
>
> Please find the attached patch to fix below issues:
>
> 1. #2963 - Backup database, Restore database and Maintenance Database
> failed for é object
> 2. #3157 - Process viewer doesn't show complete command executed.
>
> Test cases are not included for these fixes as we don't have test cases
> for these modules (backup, restore, maintenance).
> I will create one separate RM for the same which will cover this.
>
Interesting that you fix these together, as together they also exhibit
another bug :-). Backing up the é database displays the following command:
/usr/local/pgsql/bin/pg_dump --file "/Users/dpage/foo.bak" --host
"localhost" --port "5432" --username "postgres" --no-password --verbose
--format=c --blobs "é"
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-09 15:39 Dave Page <[email protected]>
parent: Dave Page <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Dave Page @ 2018-03-09 15:39 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
Hi
On Fri, Mar 9, 2018 at 3:32 PM, Dave Page <[email protected]> wrote:
> Hi
>
> On Fri, Mar 9, 2018 at 3:54 AM, Khushboo Vashi <
> [email protected]> wrote:
>
>> Hi,
>>
>> Please find the attached patch to fix below issues:
>>
>> 1. #2963 - Backup database, Restore database and Maintenance Database
>> failed for é object
>> 2. #3157 - Process viewer doesn't show complete command executed.
>>
>> Test cases are not included for these fixes as we don't have test cases
>> for these modules (backup, restore, maintenance).
>> I will create one separate RM for the same which will cover this.
>>
>
> Interesting that you fix these together, as together they also exhibit
> another bug :-). Backing up the é database displays the following command:
>
> /usr/local/pgsql/bin/pg_dump --file "/Users/dpage/foo.bak" --host
> "localhost" --port "5432" --username "postgres" --no-password --verbose
> --format=c --blobs "é"
>
Also, what tests can we add for backup/restore? We have nothing at all at
the moment, and it is pretty troublesome. I'd like to ensure that we can
backup and restore a database correctly, and ensure that the displayed
commands are what we expect and that we get valid output from
pg_dump/pg_restore (though, it may change from PG version to PG version, so
maybe we should just check for something small and generic). I guess this
might need some config parameters for the tests to specify the pg_* utility
paths for each server.
I'd suggest maybe having a feature test that opens the prefs, sets the
appropriate path, then runs a backup, waits for it to finish, checks the
process monitor output, then restores the same backup to a new database,
checking the process monitor output again, and then checking that the
restored database contains at least one object from the original database
(we don't need to check all of pg_dump/pg_restore, just that something
expected was restored). We should use a (partial) database name and backup
filename from the advanced test config file, and I think both should
default to some interesting non-ASCII strings to ensure quoting works.
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-12 06:00 Khushboo Vashi <[email protected]>
parent: Dave Page <[email protected]>
0 siblings, 2 replies; 19+ messages in thread
From: Khushboo Vashi @ 2018-03-12 06:00 UTC (permalink / raw)
To: Dave Page <[email protected]>; +Cc: pgadmin-hackers
Hi Dave,
On Fri, Mar 9, 2018 at 9:09 PM, Dave Page <[email protected]> wrote:
> Hi
>
> On Fri, Mar 9, 2018 at 3:32 PM, Dave Page <[email protected]> wrote:
>
>> Hi
>>
>> On Fri, Mar 9, 2018 at 3:54 AM, Khushboo Vashi <
>> [email protected]> wrote:
>>
>>> Hi,
>>>
>>> Please find the attached patch to fix below issues:
>>>
>>> 1. #2963 - Backup database, Restore database and Maintenance Database
>>> failed for é object
>>> 2. #3157 - Process viewer doesn't show complete command executed.
>>>
>>> Test cases are not included for these fixes as we don't have test cases
>>> for these modules (backup, restore, maintenance).
>>> I will create one separate RM for the same which will cover this.
>>>
>>
>> Interesting that you fix these together, as together they also exhibit
>> another bug :-). Backing up the é database displays the following
>> command:
>>
>> /usr/local/pgsql/bin/pg_dump --file "/Users/dpage/foo.bak" --host
>> "localhost" --port "5432" --username "postgres" --no-password --verbose
>> --format=c --blobs "é"
>>
>
> I can reproduce this issue only with notification dialogue (which I have
fixed in the attached patch) not with the details dialogue. Please refer
the screenshots for the same.
> Also, what tests can we add for backup/restore? We have nothing at all at
> the moment, and it is pretty troublesome. I'd like to ensure that we can
> backup and restore a database correctly, and ensure that the displayed
> commands are what we expect and that we get valid output from
> pg_dump/pg_restore (though, it may change from PG version to PG version, so
> maybe we should just check for something small and generic). I guess this
> might need some config parameters for the tests to specify the pg_* utility
> paths for each server.
>
> I'd suggest maybe having a feature test that opens the prefs, sets the
> appropriate path, then runs a backup, waits for it to finish, checks the
> process monitor output, then restores the same backup to a new database,
> checking the process monitor output again, and then checking that the
> restored database contains at least one object from the original database
> (we don't need to check all of pg_dump/pg_restore, just that something
> expected was restored). We should use a (partial) database name and backup
> filename from the advanced test config file, and I think both should
> default to some interesting non-ASCII strings to ensure quoting works.
>
I was thinking of writing the unit test cases for the processes.py file as
all the major functionalities for backup/restore/maintenance jobs done by
this file, but by this we can not achieve the front-end string validation
esp for non-ASCII strings.
So, I am thinking of writing feature tests (as you have suggested) first
and after that if needed I will write unit test cases.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
Attachments:
[text/x-patch] RM_2963_3157_ver1.patch (5.1K, 3-RM_2963_3157_ver1.patch)
download | inline diff:
diff --git a/web/pgadmin/misc/bgprocess/processes.py b/web/pgadmin/misc/bgprocess/processes.py
index 02b953d..385b7bd 100644
--- a/web/pgadmin/misc/bgprocess/processes.py
+++ b/web/pgadmin/misc/bgprocess/processes.py
@@ -78,10 +78,20 @@ class BatchProcess(object):
_("Could not find a process with the specified ID.")
)
+ try:
+ tmp_desc = loads(p.desc.encode('latin-1')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except UnicodeDecodeError:
+ tmp_desc = loads(p.desc.encode('utf-8')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except Exception as e:
+ tmp_desc = loads(p.desc.encode('utf-8', 'ignore')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+
# ID
self.id = _id
# Description
- self.desc = loads(p.desc)
+ self.desc = tmp_desc
# Status Acknowledged time
self.atime = p.acknowledge
# Command
@@ -171,6 +181,16 @@ class BatchProcess(object):
csv_writer.writerow(_args)
args_val = args_csv_io.getvalue().strip(str('\r\n'))
+ tmp_desc = dumps(self.desc)
+ try:
+ tmp_desc =tmp_desc.decode('utf-8') if\
+ IS_PY2 and hasattr(tmp_desc, 'decode') else tmp_desc
+ except UnicodeDecodeError:
+ tmp_desc = tmp_desc.decode('latin-1') if \
+ IS_PY2 and hasattr(tmp_desc, 'decode') else tmp_desc
+ except Exception:
+ tmp_desc = tmp_desc.decode('utf-8', 'ignore') if \
+ IS_PY2 and hasattr(tmp_desc, 'decode') else tmp_desc
j = Process(
pid=int(id),
@@ -178,7 +198,7 @@ class BatchProcess(object):
arguments=args_val.decode('utf-8', 'replace')
if IS_PY2 and hasattr(args_val, 'decode') else args_val,
logdir=log_dir,
- desc=dumps(self.desc),
+ desc=tmp_desc,
user_id=current_user.id
)
db.session.add(j)
@@ -534,7 +554,17 @@ class BatchProcess(object):
etime = parser.parse(p.end_time or get_current_time())
execution_time = (etime - stime).total_seconds()
- desc = loads(p.desc)
+ desc = ""
+ try:
+ desc = loads(p.desc.encode('latin-1')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except UnicodeDecodeError:
+ desc = loads(p.desc.encode('utf-8')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+ except Exception:
+ desc = loads(p.desc.encode('utf-8', 'ignore')) if \
+ IS_PY2 and hasattr(p.desc, 'encode') else loads(p.desc)
+
details = desc
if isinstance(desc, IProcessDesc):
diff --git a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
index 5e30590..0934ae4 100644
--- a/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
+++ b/web/pgadmin/misc/bgprocess/static/js/bgprocess.js
@@ -248,7 +248,7 @@ define('misc.bgprocess', [
if (!self.notifier) {
var header = $('<div></div>', {
class: 'h5 pg-bg-notify-header',
- }).append($('<span></span>').text(self.desc)),
+ }).append($('<span></span>').text(_.unescape(self.desc))),
content = $('<div class="pg-bg-bgprocess row"></div>').append(
header
).append(
@@ -366,7 +366,7 @@ define('misc.bgprocess', [
self.logs[0].scrollTop = self.logs[0].scrollHeight;
});
// set bgprocess detailed description
- $header.find('.bg-detailed-desc').html(self.detailed_desc);
+ $header.find('.bg-detailed-desc').html(_.unescape(self.detailed_desc));
}
// set bgprocess start time
@@ -562,4 +562,4 @@ define('misc.bgprocess', [
});
return pgBrowser.BackgroundProcessObsorver;
-});
\ No newline at end of file
+});
diff --git a/web/pgadmin/tools/backup/__init__.py b/web/pgadmin/tools/backup/__init__.py
index 715ff6e..9e70c4f 100644
--- a/web/pgadmin/tools/backup/__init__.py
+++ b/web/pgadmin/tools/backup/__init__.py
@@ -97,11 +97,9 @@ class BackupMessage(IProcessDesc):
def cmdArg(x):
if x:
- # x = html.safe_str(x)
x = x.replace('\\', '\\\\')
x = x.replace('"', '\\"')
x = x.replace('""', '\\"')
-
return ' "' + x + '"'
return ''
@@ -111,6 +109,7 @@ class BackupMessage(IProcessDesc):
else:
self.cmd += cmdArg(arg)
+
@property
def message(self):
# Fetch the server details like hostname, port, roles etc
@@ -189,7 +188,7 @@ class BackupMessage(IProcessDesc):
res += '</div><div class="h5">'
res += _("Running command:")
res += '</b><br><span class="pg-bg-cmd enable-selection">'
- res += html.safe_str(self.cmd)
+ res += html.safe_str(cmd + self.cmd)
res += '</span></div>'
return res
[image/png] detail_dialogue.png (366.6K, 4-detail_dialogue.png)
download | view image
[image/png] notification_dialogue.png (397.3K, 5-notification_dialogue.png)
download | view image
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-12 11:45 Dave Page <[email protected]>
parent: Khushboo Vashi <[email protected]>
1 sibling, 1 reply; 19+ messages in thread
From: Dave Page @ 2018-03-12 11:45 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
Hi
On Mon, Mar 12, 2018 at 2:00 AM, Khushboo Vashi <
[email protected]> wrote:
>
> Hi Dave,
>
> On Fri, Mar 9, 2018 at 9:09 PM, Dave Page <[email protected]> wrote:
>
>> Hi
>>
>> On Fri, Mar 9, 2018 at 3:32 PM, Dave Page <[email protected]> wrote:
>>
>>> Hi
>>>
>>> On Fri, Mar 9, 2018 at 3:54 AM, Khushboo Vashi <
>>> [email protected]> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find the attached patch to fix below issues:
>>>>
>>>> 1. #2963 - Backup database, Restore database and Maintenance Database
>>>> failed for é object
>>>> 2. #3157 - Process viewer doesn't show complete command executed.
>>>>
>>>> Test cases are not included for these fixes as we don't have test cases
>>>> for these modules (backup, restore, maintenance).
>>>> I will create one separate RM for the same which will cover this.
>>>>
>>>
>>> Interesting that you fix these together, as together they also exhibit
>>> another bug :-). Backing up the é database displays the following
>>> command:
>>>
>>> /usr/local/pgsql/bin/pg_dump --file "/Users/dpage/foo.bak" --host
>>> "localhost" --port "5432" --username "postgres" --no-password --verbose
>>> --format=c --blobs "é"
>>>
>>
>> I can reproduce this issue only with notification dialogue (which I have
> fixed in the attached patch) not with the details dialogue. Please refer
> the screenshots for the same.
>
Huh, interesting. What Python version were you using? I had 2.7.
> Also, what tests can we add for backup/restore? We have nothing at all at
>> the moment, and it is pretty troublesome. I'd like to ensure that we can
>> backup and restore a database correctly, and ensure that the displayed
>> commands are what we expect and that we get valid output from
>> pg_dump/pg_restore (though, it may change from PG version to PG version, so
>> maybe we should just check for something small and generic). I guess this
>> might need some config parameters for the tests to specify the pg_* utility
>> paths for each server.
>>
>> I'd suggest maybe having a feature test that opens the prefs, sets the
>> appropriate path, then runs a backup, waits for it to finish, checks the
>> process monitor output, then restores the same backup to a new database,
>> checking the process monitor output again, and then checking that the
>> restored database contains at least one object from the original database
>> (we don't need to check all of pg_dump/pg_restore, just that something
>> expected was restored). We should use a (partial) database name and backup
>> filename from the advanced test config file, and I think both should
>> default to some interesting non-ASCII strings to ensure quoting works.
>>
> I was thinking of writing the unit test cases for the processes.py file as
> all the major functionalities for backup/restore/maintenance jobs done by
> this file, but by this we can not achieve the front-end string validation
> esp for non-ASCII strings.
> So, I am thinking of writing feature tests (as you have suggested) first
> and after that if needed I will write unit test cases.
>
Sounds good. Let's try to keep the feature testing minimal, and expand the
coverage with unit tests.
Thanks!
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-12 11:47 Khushboo Vashi <[email protected]>
parent: Dave Page <[email protected]>
0 siblings, 0 replies; 19+ messages in thread
From: Khushboo Vashi @ 2018-03-12 11:47 UTC (permalink / raw)
To: Dave Page <[email protected]>; +Cc: pgadmin-hackers
On Mon, Mar 12, 2018 at 5:15 PM, Dave Page <[email protected]> wrote:
> Hi
>
> On Mon, Mar 12, 2018 at 2:00 AM, Khushboo Vashi <
> [email protected]> wrote:
>
>>
>> Hi Dave,
>>
>> On Fri, Mar 9, 2018 at 9:09 PM, Dave Page <[email protected]> wrote:
>>
>>> Hi
>>>
>>> On Fri, Mar 9, 2018 at 3:32 PM, Dave Page <[email protected]> wrote:
>>>
>>>> Hi
>>>>
>>>> On Fri, Mar 9, 2018 at 3:54 AM, Khushboo Vashi <
>>>> [email protected]> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Please find the attached patch to fix below issues:
>>>>>
>>>>> 1. #2963 - Backup database, Restore database and Maintenance Database
>>>>> failed for é object
>>>>> 2. #3157 - Process viewer doesn't show complete command executed.
>>>>>
>>>>> Test cases are not included for these fixes as we don't have test
>>>>> cases for these modules (backup, restore, maintenance).
>>>>> I will create one separate RM for the same which will cover this.
>>>>>
>>>>
>>>> Interesting that you fix these together, as together they also exhibit
>>>> another bug :-). Backing up the é database displays the following
>>>> command:
>>>>
>>>> /usr/local/pgsql/bin/pg_dump --file "/Users/dpage/foo.bak" --host
>>>> "localhost" --port "5432" --username "postgres" --no-password --verbose
>>>> --format=c --blobs "é"
>>>>
>>>
>>> I can reproduce this issue only with notification dialogue (which I have
>> fixed in the attached patch) not with the details dialogue. Please refer
>> the screenshots for the same.
>>
>
> Huh, interesting. What Python version were you using? I had 2.7.
>
>
PY 2.7 and PY 3.5
> Also, what tests can we add for backup/restore? We have nothing at all at
>>> the moment, and it is pretty troublesome. I'd like to ensure that we can
>>> backup and restore a database correctly, and ensure that the displayed
>>> commands are what we expect and that we get valid output from
>>> pg_dump/pg_restore (though, it may change from PG version to PG version, so
>>> maybe we should just check for something small and generic). I guess this
>>> might need some config parameters for the tests to specify the pg_* utility
>>> paths for each server.
>>>
>>> I'd suggest maybe having a feature test that opens the prefs, sets the
>>> appropriate path, then runs a backup, waits for it to finish, checks the
>>> process monitor output, then restores the same backup to a new database,
>>> checking the process monitor output again, and then checking that the
>>> restored database contains at least one object from the original database
>>> (we don't need to check all of pg_dump/pg_restore, just that something
>>> expected was restored). We should use a (partial) database name and backup
>>> filename from the advanced test config file, and I think both should
>>> default to some interesting non-ASCII strings to ensure quoting works.
>>>
>> I was thinking of writing the unit test cases for the processes.py file
>> as all the major functionalities for backup/restore/maintenance jobs done
>> by this file, but by this we can not achieve the front-end string
>> validation esp for non-ASCII strings.
>> So, I am thinking of writing feature tests (as you have suggested) first
>> and after that if needed I will write unit test cases.
>>
>
> Sounds good. Let's try to keep the feature testing minimal, and expand the
> coverage with unit tests.
>
> Sure.
> Thanks!
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-12 20:59 Dave Page <[email protected]>
parent: Khushboo Vashi <[email protected]>
1 sibling, 1 reply; 19+ messages in thread
From: Dave Page @ 2018-03-12 20:59 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
So I was trying to test this, and every time I try to run a backup, I'm
getting the following, with or without your patch:
(sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless you
use a text_factory that can interpret 8-bit bytestrings (like text_factory
= str). It is highly recommended that you instead just switch your
application to Unicode strings. [SQL: u'INSERT INTO process (pid, user_id,
command, "desc", arguments, logdir, start_time, end_time, exit_code,
acknowledge) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters:
(180312205250107339, 1, u'/Library/PostgreSQL/9.4/bin/pg_dump',
'ccopy_reg\n_reconstructor\np0\n(cpgadmin.tools.backup\nBackupMessage\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS\'cmd\'\np6\nV
--file "/Users/dpage/foo.dmp" --host "127.0.0.1" --port "5432" --username
"postgres" --no-password --verbose --format=c --blobs
"\xe9"\np7\nsS\'backup_type\'\np8\nI3\nsS\'database\'\np9\nV\xe9\np10\nsS\'bfile\'\np11\nS\'foo.dmp\'\np12\nsS\'sid\'\np13\nI1\nsb.',
u'--file,/Users/dpage/foo.dmp,--host,127.0.0.1,--port,5432,--username,postgres,--no-password,--verbose,--format=c,--blobs,\xe9',
'/var/lib/pgadmin/sessions/process_logs/180312205250107339', None, None,
None, None)]
Any thoughts as to what's going on? I wasn't getting this on my other
laptop, and I can't think what else we would have changed to cause this.
On Mon, Mar 12, 2018 at 2:00 AM, Khushboo Vashi <
[email protected]> wrote:
>
> Hi Dave,
>
> On Fri, Mar 9, 2018 at 9:09 PM, Dave Page <[email protected]> wrote:
>
>> Hi
>>
>> On Fri, Mar 9, 2018 at 3:32 PM, Dave Page <[email protected]> wrote:
>>
>>> Hi
>>>
>>> On Fri, Mar 9, 2018 at 3:54 AM, Khushboo Vashi <
>>> [email protected]> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please find the attached patch to fix below issues:
>>>>
>>>> 1. #2963 - Backup database, Restore database and Maintenance Database
>>>> failed for é object
>>>> 2. #3157 - Process viewer doesn't show complete command executed.
>>>>
>>>> Test cases are not included for these fixes as we don't have test cases
>>>> for these modules (backup, restore, maintenance).
>>>> I will create one separate RM for the same which will cover this.
>>>>
>>>
>>> Interesting that you fix these together, as together they also exhibit
>>> another bug :-). Backing up the é database displays the following
>>> command:
>>>
>>> /usr/local/pgsql/bin/pg_dump --file "/Users/dpage/foo.bak" --host
>>> "localhost" --port "5432" --username "postgres" --no-password --verbose
>>> --format=c --blobs "é"
>>>
>>
>> I can reproduce this issue only with notification dialogue (which I have
> fixed in the attached patch) not with the details dialogue. Please refer
> the screenshots for the same.
>
>> Also, what tests can we add for backup/restore? We have nothing at all at
>> the moment, and it is pretty troublesome. I'd like to ensure that we can
>> backup and restore a database correctly, and ensure that the displayed
>> commands are what we expect and that we get valid output from
>> pg_dump/pg_restore (though, it may change from PG version to PG version, so
>> maybe we should just check for something small and generic). I guess this
>> might need some config parameters for the tests to specify the pg_* utility
>> paths for each server.
>>
>> I'd suggest maybe having a feature test that opens the prefs, sets the
>> appropriate path, then runs a backup, waits for it to finish, checks the
>> process monitor output, then restores the same backup to a new database,
>> checking the process monitor output again, and then checking that the
>> restored database contains at least one object from the original database
>> (we don't need to check all of pg_dump/pg_restore, just that something
>> expected was restored). We should use a (partial) database name and backup
>> filename from the advanced test config file, and I think both should
>> default to some interesting non-ASCII strings to ensure quoting works.
>>
> I was thinking of writing the unit test cases for the processes.py file as
> all the major functionalities for backup/restore/maintenance jobs done by
> this file, but by this we can not achieve the front-end string validation
> esp for non-ASCII strings.
> So, I am thinking of writing feature tests (as you have suggested) first
> and after that if needed I will write unit test cases.
>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-13 03:12 Khushboo Vashi <[email protected]>
parent: Dave Page <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Khushboo Vashi @ 2018-03-13 03:12 UTC (permalink / raw)
To: Dave Page <[email protected]>; +Cc: pgadmin-hackers
On Tue, Mar 13, 2018 at 2:29 AM, Dave Page <[email protected]> wrote:
> So I was trying to test this, and every time I try to run a backup, I'm
> getting the following, with or without your patch:
>
> (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless you
> use a text_factory that can interpret 8-bit bytestrings (like text_factory
> = str). It is highly recommended that you instead just switch your
> application to Unicode strings. [SQL: u'INSERT INTO process (pid, user_id,
> command, "desc", arguments, logdir, start_time, end_time, exit_code,
> acknowledge) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters:
> (180312205250107339, 1, u'/Library/PostgreSQL/9.4/bin/pg_dump',
> 'ccopy_reg\n_reconstructor\np0\n(cpgadmin.tools.backup\
> nBackupMessage\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS\'cmd\'\np6\nV
> --file "/Users/dpage/foo.dmp" --host "127.0.0.1" --port "5432" --username
> "postgres" --no-password --verbose --format=c --blobs
> "\xe9"\np7\nsS\'backup_type\'\np8\nI3\nsS\'database\'\np9\
> nV\xe9\np10\nsS\'bfile\'\np11\nS\'foo.dmp\'\np12\nsS\'sid\'\np13\nI1\nsb.',
> u'--file,/Users/dpage/foo.dmp,--host,127.0.0.1,--port,5432,-
> -username,postgres,--no-password,--verbose,--format=c,--blobs,\xe9',
> '/var/lib/pgadmin/sessions/process_logs/180312205250107339', None, None,
> None, None)]
>
> Any thoughts as to what's going on? I wasn't getting this on my other
> laptop, and I can't think what else we would have changed to cause this.
>
> Deleting all the records from the process table from SQLITE will solve
this problem.
There were few issues related to encoding-decoding in my old patches, you
may have applied those.
> On Mon, Mar 12, 2018 at 2:00 AM, Khushboo Vashi <
> [email protected]> wrote:
>
>>
>> Hi Dave,
>>
>> On Fri, Mar 9, 2018 at 9:09 PM, Dave Page <[email protected]> wrote:
>>
>>> Hi
>>>
>>> On Fri, Mar 9, 2018 at 3:32 PM, Dave Page <[email protected]> wrote:
>>>
>>>> Hi
>>>>
>>>> On Fri, Mar 9, 2018 at 3:54 AM, Khushboo Vashi <
>>>> [email protected]> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Please find the attached patch to fix below issues:
>>>>>
>>>>> 1. #2963 - Backup database, Restore database and Maintenance Database
>>>>> failed for é object
>>>>> 2. #3157 - Process viewer doesn't show complete command executed.
>>>>>
>>>>> Test cases are not included for these fixes as we don't have test
>>>>> cases for these modules (backup, restore, maintenance).
>>>>> I will create one separate RM for the same which will cover this.
>>>>>
>>>>
>>>> Interesting that you fix these together, as together they also exhibit
>>>> another bug :-). Backing up the é database displays the following
>>>> command:
>>>>
>>>> /usr/local/pgsql/bin/pg_dump --file "/Users/dpage/foo.bak" --host
>>>> "localhost" --port "5432" --username "postgres" --no-password --verbose
>>>> --format=c --blobs "é"
>>>>
>>>
>>> I can reproduce this issue only with notification dialogue (which I have
>> fixed in the attached patch) not with the details dialogue. Please refer
>> the screenshots for the same.
>>
>>> Also, what tests can we add for backup/restore? We have nothing at all
>>> at the moment, and it is pretty troublesome. I'd like to ensure that we can
>>> backup and restore a database correctly, and ensure that the displayed
>>> commands are what we expect and that we get valid output from
>>> pg_dump/pg_restore (though, it may change from PG version to PG version, so
>>> maybe we should just check for something small and generic). I guess this
>>> might need some config parameters for the tests to specify the pg_* utility
>>> paths for each server.
>>>
>>> I'd suggest maybe having a feature test that opens the prefs, sets the
>>> appropriate path, then runs a backup, waits for it to finish, checks the
>>> process monitor output, then restores the same backup to a new database,
>>> checking the process monitor output again, and then checking that the
>>> restored database contains at least one object from the original database
>>> (we don't need to check all of pg_dump/pg_restore, just that something
>>> expected was restored). We should use a (partial) database name and backup
>>> filename from the advanced test config file, and I think both should
>>> default to some interesting non-ASCII strings to ensure quoting works.
>>>
>> I was thinking of writing the unit test cases for the processes.py file
>> as all the major functionalities for backup/restore/maintenance jobs done
>> by this file, but by this we can not achieve the front-end string
>> validation esp for non-ASCII strings.
>> So, I am thinking of writing feature tests (as you have suggested) first
>> and after that if needed I will write unit test cases.
>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-13 04:09 Dave Page <[email protected]>
parent: Khushboo Vashi <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Dave Page @ 2018-03-13 04:09 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
> On 12 Mar 2018, at 23:12, Khushboo Vashi <[email protected]> wrote:
>
>
>
>> On Tue, Mar 13, 2018 at 2:29 AM, Dave Page <[email protected]> wrote:
>> So I was trying to test this, and every time I try to run a backup, I'm getting the following, with or without your patch:
>>
>> (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. [SQL: u'INSERT INTO process (pid, user_id, command, "desc", arguments, logdir, start_time, end_time, exit_code, acknowledge) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (180312205250107339, 1, u'/Library/PostgreSQL/9.4/bin/pg_dump', 'ccopy_reg\n_reconstructor\np0\n(cpgadmin.tools.backup\nBackupMessage\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS\'cmd\'\np6\nV --file "/Users/dpage/foo.dmp" --host "127.0.0.1" --port "5432" --username "postgres" --no-password --verbose --format=c --blobs "\xe9"\np7\nsS\'backup_type\'\np8\nI3\nsS\'database\'\np9\nV\xe9\np10\nsS\'bfile\'\np11\nS\'foo.dmp\'\np12\nsS\'sid\'\np13\nI1\nsb.', u'--file,/Users/dpage/foo.dmp,--host,127.0.0.1,--port,5432,--username,postgres,--no-password,--verbose,--format=c,--blobs,\xe9', '/var/lib/pgadmin/sessions/process_logs/180312205250107339', None, None, None, None)]
>>
>> Any thoughts as to what's going on? I wasn't getting this on my other laptop, and I can't think what else we would have changed to cause this.
>>
> Deleting all the records from the process table from SQLITE will solve this problem.
> There were few issues related to encoding-decoding in my old patches, you may have applied those.
I deleted the database entirely, and still saw the problem.
>>> On Mon, Mar 12, 2018 at 2:00 AM, Khushboo Vashi <[email protected]> wrote:
>>>
>>> Hi Dave,
>>>
>>>> On Fri, Mar 9, 2018 at 9:09 PM, Dave Page <[email protected]> wrote:
>>>> Hi
>>>>
>>>>> On Fri, Mar 9, 2018 at 3:32 PM, Dave Page <[email protected]> wrote:
>>>>> Hi
>>>>>
>>>>>> On Fri, Mar 9, 2018 at 3:54 AM, Khushboo Vashi <[email protected]> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Please find the attached patch to fix below issues:
>>>>>>
>>>>>> 1. #2963 - Backup database, Restore database and Maintenance Database failed for é object
>>>>>> 2. #3157 - Process viewer doesn't show complete command executed.
>>>>>>
>>>>>> Test cases are not included for these fixes as we don't have test cases for these modules (backup, restore, maintenance).
>>>>>> I will create one separate RM for the same which will cover this.
>>>>>
>>>>> Interesting that you fix these together, as together they also exhibit another bug :-). Backing up the é database displays the following command:
>>>>>
>>>>> /usr/local/pgsql/bin/pg_dump --file "/Users/dpage/foo.bak" --host "localhost" --port "5432" --username "postgres" --no-password --verbose --format=c --blobs "é"
>>>>
>>>
>>> I can reproduce this issue only with notification dialogue (which I have fixed in the attached patch) not with the details dialogue. Please refer the screenshots for the same.
>>>> Also, what tests can we add for backup/restore? We have nothing at all at the moment, and it is pretty troublesome. I'd like to ensure that we can backup and restore a database correctly, and ensure that the displayed commands are what we expect and that we get valid output from pg_dump/pg_restore (though, it may change from PG version to PG version, so maybe we should just check for something small and generic). I guess this might need some config parameters for the tests to specify the pg_* utility paths for each server.
>>>>
>>>> I'd suggest maybe having a feature test that opens the prefs, sets the appropriate path, then runs a backup, waits for it to finish, checks the process monitor output, then restores the same backup to a new database, checking the process monitor output again, and then checking that the restored database contains at least one object from the original database (we don't need to check all of pg_dump/pg_restore, just that something expected was restored). We should use a (partial) database name and backup filename from the advanced test config file, and I think both should default to some interesting non-ASCII strings to ensure quoting works.
>>>
>>> I was thinking of writing the unit test cases for the processes.py file as all the major functionalities for backup/restore/maintenance jobs done by this file, but by this we can not achieve the front-end string validation esp for non-ASCII strings.
>>> So, I am thinking of writing feature tests (as you have suggested) first and after that if needed I will write unit test cases.
>>>>
>>>> --
>>>> Dave Page
>>>> Blog: http://pgsnake.blogspot.com
>>>> Twitter: @pgsnake
>>>>
>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>> The Enterprise PostgreSQL Company
>>>
>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-13 04:46 Khushboo Vashi <[email protected]>
parent: Dave Page <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Khushboo Vashi @ 2018-03-13 04:46 UTC (permalink / raw)
To: Dave Page <[email protected]>; +Cc: pgadmin-hackers
On Tue, Mar 13, 2018 at 9:39 AM, Dave Page <[email protected]> wrote:
>
>
> On 12 Mar 2018, at 23:12, Khushboo Vashi <[email protected]>
> wrote:
>
>
>
> On Tue, Mar 13, 2018 at 2:29 AM, Dave Page <[email protected]> wrote:
>
>> So I was trying to test this, and every time I try to run a backup, I'm
>> getting the following, with or without your patch:
>>
>> (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless you
>> use a text_factory that can interpret 8-bit bytestrings (like text_factory
>> = str). It is highly recommended that you instead just switch your
>> application to Unicode strings. [SQL: u'INSERT INTO process (pid, user_id,
>> command, "desc", arguments, logdir, start_time, end_time, exit_code,
>> acknowledge) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters:
>> (180312205250107339, 1, u'/Library/PostgreSQL/9.4/bin/pg_dump',
>> 'ccopy_reg\n_reconstructor\np0\n(cpgadmin.tools.backup\nBack
>> upMessage\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS\'cmd\'\np6\nV
>> --file "/Users/dpage/foo.dmp" --host "127.0.0.1" --port "5432" --username
>> "postgres" --no-password --verbose --format=c --blobs
>> "\xe9"\np7\nsS\'backup_type\'\np8\nI3\nsS\'database\'\np9\nV
>> \xe9\np10\nsS\'bfile\'\np11\nS\'foo.dmp\'\np12\nsS\'sid\'\np13\nI1\nsb.',
>> u'--file,/Users/dpage/foo.dmp,--host,127.0.0.1,--port,5432,-
>> -username,postgres,--no-password,--verbose,--format=c,--blobs,\xe9',
>> '/var/lib/pgadmin/sessions/process_logs/180312205250107339', None, None,
>> None, None)]
>>
>> Any thoughts as to what's going on? I wasn't getting this on my other
>> laptop, and I can't think what else we would have changed to cause this.
>>
>> Deleting all the records from the process table from SQLITE will solve
> this problem.
> There were few issues related to encoding-decoding in my old patches, you
> may have applied those.
>
>
> I deleted the database entirely, and still saw the problem.
>
> I have tried many things to reproduce this but couldn't. I faced this
issue when I was fixing the issue but not now.
Please make sure to delete the old session and a process table (or
database) and apply my latest patch. (of course you do this :) )
I have PY - 2.7.12, psycopg2 - 2.7.4 and SQLAlhemy -1.0.14 to be more
specific.
> On Mon, Mar 12, 2018 at 2:00 AM, Khushboo Vashi <
>> [email protected]> wrote:
>>
>>>
>>> Hi Dave,
>>>
>>> On Fri, Mar 9, 2018 at 9:09 PM, Dave Page <[email protected]> wrote:
>>>
>>>> Hi
>>>>
>>>> On Fri, Mar 9, 2018 at 3:32 PM, Dave Page <[email protected]> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> On Fri, Mar 9, 2018 at 3:54 AM, Khushboo Vashi <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please find the attached patch to fix below issues:
>>>>>>
>>>>>> 1. #2963 - Backup database, Restore database and Maintenance Database
>>>>>> failed for é object
>>>>>> 2. #3157 - Process viewer doesn't show complete command executed.
>>>>>>
>>>>>> Test cases are not included for these fixes as we don't have test
>>>>>> cases for these modules (backup, restore, maintenance).
>>>>>> I will create one separate RM for the same which will cover this.
>>>>>>
>>>>>
>>>>> Interesting that you fix these together, as together they also exhibit
>>>>> another bug :-). Backing up the é database displays the following
>>>>> command:
>>>>>
>>>>> /usr/local/pgsql/bin/pg_dump --file "/Users/dpage/foo.bak" --host
>>>>> "localhost" --port "5432" --username "postgres" --no-password --verbose
>>>>> --format=c --blobs "é"
>>>>>
>>>>
>>>> I can reproduce this issue only with notification dialogue (which I
>>> have fixed in the attached patch) not with the details dialogue. Please
>>> refer the screenshots for the same.
>>>
>>>> Also, what tests can we add for backup/restore? We have nothing at all
>>>> at the moment, and it is pretty troublesome. I'd like to ensure that we can
>>>> backup and restore a database correctly, and ensure that the displayed
>>>> commands are what we expect and that we get valid output from
>>>> pg_dump/pg_restore (though, it may change from PG version to PG version, so
>>>> maybe we should just check for something small and generic). I guess this
>>>> might need some config parameters for the tests to specify the pg_* utility
>>>> paths for each server.
>>>>
>>>> I'd suggest maybe having a feature test that opens the prefs, sets the
>>>> appropriate path, then runs a backup, waits for it to finish, checks the
>>>> process monitor output, then restores the same backup to a new database,
>>>> checking the process monitor output again, and then checking that the
>>>> restored database contains at least one object from the original database
>>>> (we don't need to check all of pg_dump/pg_restore, just that something
>>>> expected was restored). We should use a (partial) database name and backup
>>>> filename from the advanced test config file, and I think both should
>>>> default to some interesting non-ASCII strings to ensure quoting works.
>>>>
>>> I was thinking of writing the unit test cases for the processes.py file
>>> as all the major functionalities for backup/restore/maintenance jobs done
>>> by this file, but by this we can not achieve the front-end string
>>> validation esp for non-ASCII strings.
>>> So, I am thinking of writing feature tests (as you have suggested) first
>>> and after that if needed I will write unit test cases.
>>>
>>>>
>>>> --
>>>> Dave Page
>>>> Blog: http://pgsnake.blogspot.com
>>>> Twitter: @pgsnake
>>>>
>>>> EnterpriseDB UK: http://www.enterprisedb.com
>>>> The Enterprise PostgreSQL Company
>>>>
>>>
>>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-13 20:48 Dave Page <[email protected]>
parent: Khushboo Vashi <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Dave Page @ 2018-03-13 20:48 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
On Tue, Mar 13, 2018 at 12:46 AM, Khushboo Vashi <
[email protected]> wrote:
>
>
> On Tue, Mar 13, 2018 at 9:39 AM, Dave Page <[email protected]> wrote:
>
>>
>>
>> On 12 Mar 2018, at 23:12, Khushboo Vashi <[email protected]>
>> wrote:
>>
>>
>>
>> On Tue, Mar 13, 2018 at 2:29 AM, Dave Page <[email protected]> wrote:
>>
>>> So I was trying to test this, and every time I try to run a backup, I'm
>>> getting the following, with or without your patch:
>>>
>>> (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless you
>>> use a text_factory that can interpret 8-bit bytestrings (like text_factory
>>> = str). It is highly recommended that you instead just switch your
>>> application to Unicode strings. [SQL: u'INSERT INTO process (pid, user_id,
>>> command, "desc", arguments, logdir, start_time, end_time, exit_code,
>>> acknowledge) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters:
>>> (180312205250107339, 1, u'/Library/PostgreSQL/9.4/bin/pg_dump',
>>> 'ccopy_reg\n_reconstructor\np0\n(cpgadmin.tools.backup\nBack
>>> upMessage\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS\'cmd\'\np6\nV
>>> --file "/Users/dpage/foo.dmp" --host "127.0.0.1" --port "5432" --username
>>> "postgres" --no-password --verbose --format=c --blobs
>>> "\xe9"\np7\nsS\'backup_type\'\np8\nI3\nsS\'database\'\np9\nV
>>> \xe9\np10\nsS\'bfile\'\np11\nS\'foo.dmp\'\np12\nsS\'sid\'\np13\nI1\nsb.',
>>> u'--file,/Users/dpage/foo.dmp,--host,127.0.0.1,--port,5432,-
>>> -username,postgres,--no-password,--verbose,--format=c,--blobs,\xe9',
>>> '/var/lib/pgadmin/sessions/process_logs/180312205250107339', None,
>>> None, None, None)]
>>>
>>> Any thoughts as to what's going on? I wasn't getting this on my other
>>> laptop, and I can't think what else we would have changed to cause this.
>>>
>>> Deleting all the records from the process table from SQLITE will solve
>> this problem.
>> There were few issues related to encoding-decoding in my old patches, you
>> may have applied those.
>>
>>
>> I deleted the database entirely, and still saw the problem.
>>
>> I have tried many things to reproduce this but couldn't. I faced this
> issue when I was fixing the issue but not now.
> Please make sure to delete the old session and a process table (or
> database) and apply my latest patch. (of course you do this :) )
> I have PY - 2.7.12, psycopg2 - 2.7.4 and SQLAlhemy -1.0.14 to be more
> specific.
>
>>
Well I eventually got this to work (basically recreated most of my dev
environment), so I committed the patch as it's clearly an improvement.
I can still reproduce the display issue I mentioned though - see the
attached screenshot which shows é in a couple of places. I wonder if
it's because the database name is just a single character in my tests,
whilst you had some other unicode characters in the string?
Can you take a look please?
Thanks.
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[image/png] Screen Shot 2018-03-13 at 16.45.25.png (213.4K, 3-Screen%20Shot%202018-03-13%20at%2016.45.25.png)
download | view image
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-14 03:18 Khushboo Vashi <[email protected]>
parent: Dave Page <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Khushboo Vashi @ 2018-03-14 03:18 UTC (permalink / raw)
To: Dave Page <[email protected]>; +Cc: pgadmin-hackers
On Wed, Mar 14, 2018 at 2:18 AM, Dave Page <[email protected]> wrote:
>
>
> On Tue, Mar 13, 2018 at 12:46 AM, Khushboo Vashi <
> [email protected]> wrote:
>
>>
>>
>> On Tue, Mar 13, 2018 at 9:39 AM, Dave Page <[email protected]> wrote:
>>
>>>
>>>
>>> On 12 Mar 2018, at 23:12, Khushboo Vashi <[email protected]
>>> om> wrote:
>>>
>>>
>>>
>>> On Tue, Mar 13, 2018 at 2:29 AM, Dave Page <[email protected]> wrote:
>>>
>>>> So I was trying to test this, and every time I try to run a backup, I'm
>>>> getting the following, with or without your patch:
>>>>
>>>> (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless
>>>> you use a text_factory that can interpret 8-bit bytestrings (like
>>>> text_factory = str). It is highly recommended that you instead just switch
>>>> your application to Unicode strings. [SQL: u'INSERT INTO process (pid,
>>>> user_id, command, "desc", arguments, logdir, start_time, end_time,
>>>> exit_code, acknowledge) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)']
>>>> [parameters: (180312205250107339, 1, u'/Library/PostgreSQL/9.4/bin/pg_dump',
>>>> 'ccopy_reg\n_reconstructor\np0\n(cpgadmin.tools.backup\nBack
>>>> upMessage\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS\'cmd\'\np6\nV
>>>> --file "/Users/dpage/foo.dmp" --host "127.0.0.1" --port "5432" --username
>>>> "postgres" --no-password --verbose --format=c --blobs
>>>> "\xe9"\np7\nsS\'backup_type\'\np8\nI3\nsS\'database\'\np9\nV
>>>> \xe9\np10\nsS\'bfile\'\np11\nS\'foo.dmp\'\np12\nsS\'sid\'\np13\nI1\nsb.',
>>>> u'--file,/Users/dpage/foo.dmp,--host,127.0.0.1,--port,5432,-
>>>> -username,postgres,--no-password,--verbose,--format=c,--blobs,\xe9',
>>>> '/var/lib/pgadmin/sessions/process_logs/180312205250107339', None,
>>>> None, None, None)]
>>>>
>>>> Any thoughts as to what's going on? I wasn't getting this on my other
>>>> laptop, and I can't think what else we would have changed to cause this.
>>>>
>>>> Deleting all the records from the process table from SQLITE will solve
>>> this problem.
>>> There were few issues related to encoding-decoding in my old patches,
>>> you may have applied those.
>>>
>>>
>>> I deleted the database entirely, and still saw the problem.
>>>
>>> I have tried many things to reproduce this but couldn't. I faced this
>> issue when I was fixing the issue but not now.
>> Please make sure to delete the old session and a process table (or
>> database) and apply my latest patch. (of course you do this :) )
>> I have PY - 2.7.12, psycopg2 - 2.7.4 and SQLAlhemy -1.0.14 to be more
>> specific.
>>
>>>
> Well I eventually got this to work (basically recreated most of my dev
> environment), so I committed the patch as it's clearly an improvement.
>
> Thanks.
> I can still reproduce the display issue I mentioned though - see the
> attached screenshot which shows é in a couple of places. I wonder if
> it's because the database name is just a single character in my tests,
> whilst you had some other unicode characters in the string?
>
> I have tested it with a single character also but couldn't reproduce this.
Please find the attached screen-shot for the same.
Can you please take a complete screen shot of the screen (with left side
tree and properties panel of the database) and send it?
> Can you take a look please?
>
> Thanks.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
Attachments:
[image/png] Screen Shot 2018-03-14 at 8.41.44 am.png (572.0K, 3-Screen%20Shot%202018-03-14%20at%208.41.44%20am.png)
download | view image
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-14 21:43 Dave Page <[email protected]>
parent: Khushboo Vashi <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Dave Page @ 2018-03-14 21:43 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
Hi
On Tue, Mar 13, 2018 at 11:18 PM, Khushboo Vashi <
[email protected]> wrote:
>
>
> On Wed, Mar 14, 2018 at 2:18 AM, Dave Page <[email protected]> wrote:
>
>>
>>
>> On Tue, Mar 13, 2018 at 12:46 AM, Khushboo Vashi <
>> [email protected]> wrote:
>>
>>>
>>>
>>> On Tue, Mar 13, 2018 at 9:39 AM, Dave Page <[email protected]> wrote:
>>>
>>>>
>>>>
>>>> On 12 Mar 2018, at 23:12, Khushboo Vashi <[email protected]
>>>> om> wrote:
>>>>
>>>>
>>>>
>>>> On Tue, Mar 13, 2018 at 2:29 AM, Dave Page <[email protected]> wrote:
>>>>
>>>>> So I was trying to test this, and every time I try to run a backup,
>>>>> I'm getting the following, with or without your patch:
>>>>>
>>>>> (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless
>>>>> you use a text_factory that can interpret 8-bit bytestrings (like
>>>>> text_factory = str). It is highly recommended that you instead just switch
>>>>> your application to Unicode strings. [SQL: u'INSERT INTO process (pid,
>>>>> user_id, command, "desc", arguments, logdir, start_time, end_time,
>>>>> exit_code, acknowledge) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)']
>>>>> [parameters: (180312205250107339, 1, u'/Library/PostgreSQL/9.4/bin/pg_dump',
>>>>> 'ccopy_reg\n_reconstructor\np0\n(cpgadmin.tools.backup\nBack
>>>>> upMessage\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS\'cmd\'\np6\nV
>>>>> --file "/Users/dpage/foo.dmp" --host "127.0.0.1" --port "5432" --username
>>>>> "postgres" --no-password --verbose --format=c --blobs
>>>>> "\xe9"\np7\nsS\'backup_type\'\np8\nI3\nsS\'database\'\np9\nV
>>>>> \xe9\np10\nsS\'bfile\'\np11\nS\'foo.dmp\'\np12\nsS\'sid\'\np13\nI1\nsb.',
>>>>> u'--file,/Users/dpage/foo.dmp,--host,127.0.0.1,--port,5432,-
>>>>> -username,postgres,--no-password,--verbose,--format=c,--blobs,\xe9',
>>>>> '/var/lib/pgadmin/sessions/process_logs/180312205250107339', None,
>>>>> None, None, None)]
>>>>>
>>>>> Any thoughts as to what's going on? I wasn't getting this on my other
>>>>> laptop, and I can't think what else we would have changed to cause this.
>>>>>
>>>>> Deleting all the records from the process table from SQLITE will solve
>>>> this problem.
>>>> There were few issues related to encoding-decoding in my old patches,
>>>> you may have applied those.
>>>>
>>>>
>>>> I deleted the database entirely, and still saw the problem.
>>>>
>>>> I have tried many things to reproduce this but couldn't. I faced this
>>> issue when I was fixing the issue but not now.
>>> Please make sure to delete the old session and a process table (or
>>> database) and apply my latest patch. (of course you do this :) )
>>> I have PY - 2.7.12, psycopg2 - 2.7.4 and SQLAlhemy -1.0.14 to be more
>>> specific.
>>>
>>>>
>> Well I eventually got this to work (basically recreated most of my dev
>> environment), so I committed the patch as it's clearly an improvement.
>>
>> Thanks.
>
>> I can still reproduce the display issue I mentioned though - see the
>> attached screenshot which shows é in a couple of places. I wonder if
>> it's because the database name is just a single character in my tests,
>> whilst you had some other unicode characters in the string?
>>
>> I have tested it with a single character also but couldn't reproduce
> this. Please find the attached screen-shot for the same.
> Can you please take a complete screen shot of the screen (with left side
> tree and properties panel of the database) and send it?
>
Sure - attached.
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[image/png] Screen Shot 2018-03-14 at 17.42.19.png (626.3K, 3-Screen%20Shot%202018-03-14%20at%2017.42.19.png)
download | view image
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-15 12:10 Khushboo Vashi <[email protected]>
parent: Dave Page <[email protected]>
0 siblings, 1 reply; 19+ messages in thread
From: Khushboo Vashi @ 2018-03-15 12:10 UTC (permalink / raw)
To: Dave Page <[email protected]>; +Cc: pgadmin-hackers
On Thu, Mar 15, 2018 at 3:13 AM, Dave Page <[email protected]> wrote:
> Hi
>
> On Tue, Mar 13, 2018 at 11:18 PM, Khushboo Vashi <
> [email protected]> wrote:
>
>>
>>
>> On Wed, Mar 14, 2018 at 2:18 AM, Dave Page <[email protected]> wrote:
>>
>>>
>>>
>>> On Tue, Mar 13, 2018 at 12:46 AM, Khushboo Vashi <
>>> [email protected]> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Mar 13, 2018 at 9:39 AM, Dave Page <[email protected]> wrote:
>>>>
>>>>>
>>>>>
>>>>> On 12 Mar 2018, at 23:12, Khushboo Vashi <
>>>>> [email protected]> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 13, 2018 at 2:29 AM, Dave Page <[email protected]> wrote:
>>>>>
>>>>>> So I was trying to test this, and every time I try to run a backup,
>>>>>> I'm getting the following, with or without your patch:
>>>>>>
>>>>>> (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless
>>>>>> you use a text_factory that can interpret 8-bit bytestrings (like
>>>>>> text_factory = str). It is highly recommended that you instead just switch
>>>>>> your application to Unicode strings. [SQL: u'INSERT INTO process (pid,
>>>>>> user_id, command, "desc", arguments, logdir, start_time, end_time,
>>>>>> exit_code, acknowledge) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)']
>>>>>> [parameters: (180312205250107339, 1, u'/Library/PostgreSQL/9.4/bin/pg_dump',
>>>>>> 'ccopy_reg\n_reconstructor\np0\n(cpgadmin.tools.backup\nBack
>>>>>> upMessage\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS\'cmd\'\np6\nV
>>>>>> --file "/Users/dpage/foo.dmp" --host "127.0.0.1" --port "5432" --username
>>>>>> "postgres" --no-password --verbose --format=c --blobs
>>>>>> "\xe9"\np7\nsS\'backup_type\'\np8\nI3\nsS\'database\'\np9\nV
>>>>>> \xe9\np10\nsS\'bfile\'\np11\nS\'foo.dmp\'\np12\nsS\'sid\'\np13\nI1\nsb.',
>>>>>> u'--file,/Users/dpage/foo.dmp,--host,127.0.0.1,--port,5432,-
>>>>>> -username,postgres,--no-password,--verbose,--format=c,--blobs,\xe9',
>>>>>> '/var/lib/pgadmin/sessions/process_logs/180312205250107339', None,
>>>>>> None, None, None)]
>>>>>>
>>>>>> Any thoughts as to what's going on? I wasn't getting this on my other
>>>>>> laptop, and I can't think what else we would have changed to cause this.
>>>>>>
>>>>>> Deleting all the records from the process table from SQLITE will
>>>>> solve this problem.
>>>>> There were few issues related to encoding-decoding in my old patches,
>>>>> you may have applied those.
>>>>>
>>>>>
>>>>> I deleted the database entirely, and still saw the problem.
>>>>>
>>>>> I have tried many things to reproduce this but couldn't. I faced this
>>>> issue when I was fixing the issue but not now.
>>>> Please make sure to delete the old session and a process table (or
>>>> database) and apply my latest patch. (of course you do this :) )
>>>> I have PY - 2.7.12, psycopg2 - 2.7.4 and SQLAlhemy -1.0.14 to be more
>>>> specific.
>>>>
>>>>>
>>> Well I eventually got this to work (basically recreated most of my dev
>>> environment), so I committed the patch as it's clearly an improvement.
>>>
>>> Thanks.
>>
>>> I can still reproduce the display issue I mentioned though - see the
>>> attached screenshot which shows é in a couple of places. I wonder if
>>> it's because the database name is just a single character in my tests,
>>> whilst you had some other unicode characters in the string?
>>>
>>> I have tested it with a single character also but couldn't reproduce
>> this. Please find the attached screen-shot for the same.
>> Can you please take a complete screen shot of the screen (with left side
>> tree and properties panel of the database) and send it?
>>
>
> Sure - attached.
>
>
I didn't find any root cause as the below line responsible for un-escaping
the html and it is clearly shown that it is not happening in your case.
$header.find('.bg-detailed-desc').html(_.unescape(self.detailed_desc));
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object.
@ 2018-03-15 19:57 Dave Page <[email protected]>
parent: Khushboo Vashi <[email protected]>
0 siblings, 0 replies; 19+ messages in thread
From: Dave Page @ 2018-03-15 19:57 UTC (permalink / raw)
To: Khushboo Vashi <[email protected]>; +Cc: pgadmin-hackers
Hi
On Thu, Mar 15, 2018 at 8:10 AM, Khushboo Vashi <
[email protected]> wrote:
>
>
> On Thu, Mar 15, 2018 at 3:13 AM, Dave Page <[email protected]> wrote:
>
>> Hi
>>
>> On Tue, Mar 13, 2018 at 11:18 PM, Khushboo Vashi <
>> [email protected]> wrote:
>>
>>>
>>>
>>> On Wed, Mar 14, 2018 at 2:18 AM, Dave Page <[email protected]> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Mar 13, 2018 at 12:46 AM, Khushboo Vashi <
>>>> [email protected]> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 13, 2018 at 9:39 AM, Dave Page <[email protected]> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 12 Mar 2018, at 23:12, Khushboo Vashi <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Mar 13, 2018 at 2:29 AM, Dave Page <[email protected]> wrote:
>>>>>>
>>>>>>> So I was trying to test this, and every time I try to run a backup,
>>>>>>> I'm getting the following, with or without your patch:
>>>>>>>
>>>>>>> (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless
>>>>>>> you use a text_factory that can interpret 8-bit bytestrings (like
>>>>>>> text_factory = str). It is highly recommended that you instead just switch
>>>>>>> your application to Unicode strings. [SQL: u'INSERT INTO process (pid,
>>>>>>> user_id, command, "desc", arguments, logdir, start_time, end_time,
>>>>>>> exit_code, acknowledge) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)']
>>>>>>> [parameters: (180312205250107339, 1, u'/Library/PostgreSQL/9.4/bin/pg_dump',
>>>>>>> 'ccopy_reg\n_reconstructor\np0\n(cpgadmin.tools.backup\nBack
>>>>>>> upMessage\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS\'cmd\'\np6\nV
>>>>>>> --file "/Users/dpage/foo.dmp" --host "127.0.0.1" --port "5432" --username
>>>>>>> "postgres" --no-password --verbose --format=c --blobs
>>>>>>> "\xe9"\np7\nsS\'backup_type\'\np8\nI3\nsS\'database\'\np9\nV
>>>>>>> \xe9\np10\nsS\'bfile\'\np11\nS\'foo.dmp\'\np12\nsS\'sid\'\np13\nI1\nsb.',
>>>>>>> u'--file,/Users/dpage/foo.dmp,--host,127.0.0.1,--port,5432,-
>>>>>>> -username,postgres,--no-password,--verbose,--format=c,--blobs,\xe9',
>>>>>>> '/var/lib/pgadmin/sessions/process_logs/180312205250107339', None,
>>>>>>> None, None, None)]
>>>>>>>
>>>>>>> Any thoughts as to what's going on? I wasn't getting this on my
>>>>>>> other laptop, and I can't think what else we would have changed to cause
>>>>>>> this.
>>>>>>>
>>>>>>> Deleting all the records from the process table from SQLITE will
>>>>>> solve this problem.
>>>>>> There were few issues related to encoding-decoding in my old patches,
>>>>>> you may have applied those.
>>>>>>
>>>>>>
>>>>>> I deleted the database entirely, and still saw the problem.
>>>>>>
>>>>>> I have tried many things to reproduce this but couldn't. I faced
>>>>> this issue when I was fixing the issue but not now.
>>>>> Please make sure to delete the old session and a process table (or
>>>>> database) and apply my latest patch. (of course you do this :) )
>>>>> I have PY - 2.7.12, psycopg2 - 2.7.4 and SQLAlhemy -1.0.14 to be more
>>>>> specific.
>>>>>
>>>>>>
>>>> Well I eventually got this to work (basically recreated most of my dev
>>>> environment), so I committed the patch as it's clearly an improvement.
>>>>
>>>> Thanks.
>>>
>>>> I can still reproduce the display issue I mentioned though - see the
>>>> attached screenshot which shows é in a couple of places. I wonder if
>>>> it's because the database name is just a single character in my tests,
>>>> whilst you had some other unicode characters in the string?
>>>>
>>>> I have tested it with a single character also but couldn't reproduce
>>> this. Please find the attached screen-shot for the same.
>>> Can you please take a complete screen shot of the screen (with left side
>>> tree and properties panel of the database) and send it?
>>>
>>
>> Sure - attached.
>>
>>
> I didn't find any root cause as the below line responsible for un-escaping
> the html and it is clearly shown that it is not happening in your case.
> $header.find('.bg-detailed-desc').html(_.unescape(self.detailed_desc));
>
Well this is weird - I just tried it again with the dev tools enabled (to
look for console errors), and it worked. Tried again without the dev tools,
and it's still working.
I have no idea what was going on, but it's working now.
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 19+ messages in thread
end of thread, other threads:[~2018-03-15 19:57 UTC | newest]
Thread overview: 19+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2017-12-25 11:03 [pgAdmin4][Patch]: RM #2963 - Backup database, Restore database and Maintenance Database failed for é object. Khushboo Vashi <[email protected]>
2018-01-03 15:24 ` Dave Page <[email protected]>
2018-01-10 06:59 ` Khushboo Vashi <[email protected]>
2018-01-12 09:54 ` Dave Page <[email protected]>
2018-03-09 03:54 ` Khushboo Vashi <[email protected]>
2018-03-09 15:32 ` Dave Page <[email protected]>
2018-03-09 15:39 ` Dave Page <[email protected]>
2018-03-12 06:00 ` Khushboo Vashi <[email protected]>
2018-03-12 11:45 ` Dave Page <[email protected]>
2018-03-12 11:47 ` Khushboo Vashi <[email protected]>
2018-03-12 20:59 ` Dave Page <[email protected]>
2018-03-13 03:12 ` Khushboo Vashi <[email protected]>
2018-03-13 04:09 ` Dave Page <[email protected]>
2018-03-13 04:46 ` Khushboo Vashi <[email protected]>
2018-03-13 20:48 ` Dave Page <[email protected]>
2018-03-14 03:18 ` Khushboo Vashi <[email protected]>
2018-03-14 21:43 ` Dave Page <[email protected]>
2018-03-15 12:10 ` Khushboo Vashi <[email protected]>
2018-03-15 19:57 ` Dave Page <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox