public inbox for [email protected]
help / color / mirror / Atom feedFrom: Khushboo Vashi <[email protected]>
To: Victoria Henry <[email protected]>
Cc: Dave Page <[email protected]>
Cc: Joao De Almeida Pereira <[email protected]>
Cc: pgadmin-hackers <[email protected]>
Subject: Re: [pgadmin4][Patch]: Test cases for the backup module
Date: Wed, 6 Jun 2018 10:42:38 +0530
Message-ID: <CAFOhELc6zT5pLHJTWd68q=jn3rhBjROUAKi5DneURdd7=B0VqA@mail.gmail.com> (raw)
In-Reply-To: <CAFOhELfcq29y-kU-dmSQFXKE8mkwHo9_OGT4wLCPhH=+6D5RAg@mail.gmail.com>
References: <CAFOhELe450RFJANE3F4wJKeOOCz6zYdfOBcT+kfT4EEBvVXbgg@mail.gmail.com>
<CAE+jjamrgDd0Z=5q=-Ps_4vBgZdM8XtGgu3=3ASvVhm5jtKe1Q@mail.gmail.com>
<CAFOhELdfV2Rb8Ov+brvjAcvz_+gqQtaMGca51c=xzv8EQuP_+w@mail.gmail.com>
<CA+OCxox8SRuCcymh+BLYmRpSPORfdQZ3or+gxxKqZHjRj0M-AA@mail.gmail.com>
<CAFOhELdssVVcqaO4r8Lrg+n4=bdPhzrq4ORL8TLRvqNYFTvigw@mail.gmail.com>
<CAFOhELfpDc9Cae++nY_wXLQo4CZH-_7VD5ra45LQaPoWD3rRBA@mail.gmail.com>
<CANxYE3J=OgdXcZ_0EGjWtRcKCJZ-vNmQeobm+dwziHjEnAQ-gg@mail.gmail.com>
<CAFOhELfTcN=QbAOLT+KPimC6rCiFfbjPn6SFz7tmVb9PQz2fGQ@mail.gmail.com>
<CA+OCxozWa=MBvousDs6X0gCoDS0A6QOzD5SGCSRAq5hoKsUhKA@mail.gmail.com>
<CAFOhELfsi56NT4V+oAfeL9Lv9MsaurjFBUz08moT0nWJvLns8w@mail.gmail.com>
<CAE+jjamv3VAxJpG_-NQjS5cOvMKV5X4aD2t1DaJKVBzg5R-jUw@mail.gmail.com>
<CAFOhELeqBi0ZLM+0LiOp2pxwCy2WR-gxVT+vxprfysr5XrUKBg@mail.gmail.com>
<CA+OCxoy=pmB36=c7CBvgL1rj11fan+XRSA4t92WJcfLXUyfdnA@mail.gmail.com>
<CAFOhELcF-5N5ysxHWmvoNXhKi4H3z4+JacQ2OVvPPJF6pK6cYg@mail.gmail.com>
<CA+OCxozcLr90+E4bDooQdR5PciJFwaeX_cHfHjYFKz_49-vqEA@mail.gmail.com>
<CAFOhELdU4Q+jgL-ZZOoHkCEFN0chJ1Tz3sVSia2EmATOj6UpQQ@mail.gmail.com>
<CANxYE3LLgXWHkVG6fvSF070tUEi=tbr5cHOka536BdLaDL-HvA@mail.gmail.com>
<CAFOhELfcq29y-kU-dmSQFXKE8mkwHo9_OGT4wLCPhH=+6D5RAg@mail.gmail.com>
Hi Dave,
As per our discussion I have added the code to clean up the generated files.
Please find the attached updated patch.
Thanks,
Khushboo
On Wed, Jun 6, 2018 at 9:37 AM, Khushboo Vashi <
[email protected]> wrote:
> Hi Victoria,
>
> As per the logs, Restore job is failing only for GPDB. As I don't have
> setup for the greenplum database, can you please check this functionality
> works well in pgAdmin4 with GPDB?
>
> Thanks,
> Khushboo
>
> On Wed, Jun 6, 2018 at 5:13 AM, Victoria Henry <[email protected]> wrote:
>
>> Hi Khushboo
>>
>> The tests are still failing and seems flaky:
>> https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines
>> /pgadmin-patch/jobs/run-tests/builds/113
>>
>> Sincerely,
>>
>> Victoria
>>
>> On Tue, Jun 5, 2018 at 4:50 AM Khushboo Vashi <
>> [email protected]> wrote:
>>
>>>
>>>
>>> On Tue, Jun 5, 2018 at 2:09 PM, Dave Page <[email protected]> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Jun 5, 2018 at 9:37 AM, Khushboo Vashi <
>>>> [email protected]> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Jun 5, 2018 at 1:36 PM, Dave Page <[email protected]> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> On Tue, Jun 5, 2018 at 4:39 AM, Khushboo Vashi <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 4, 2018 at 8:41 PM, Joao De Almeida Pereira <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> Hi Khushboo,
>>>>>>>>
>>>>>>>> Some tests are failing in greenplum:
>>>>>>>> https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines
>>>>>>>> /pgadmin-patch/jobs/run-tests/builds/108
>>>>>>>> The piece of code responsible for the error is:
>>>>>>>>
>>>>>>>> if server['default_binary_paths'] is not None:
>>>>>>>> test_utils.set_preference(server['default_binary_paths'])
>>>>>>>>
>>>>>>>> config.DEFAULT_BINARY_PATHS = {
>>>>>>>> "pg": str(server['default_binary_paths']['pg']),
>>>>>>>> "ppas": str(server['default_binary_paths']['ppas']),
>>>>>>>> "gpdb": ""
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> Can you send me the test_config.json file? The above code sets the
>>>>>>> paths to the SQLite database and through the logs couldn't figure out the
>>>>>>> exact failure.
>>>>>>>
>>>>>>
>>>>>> It seems clear from the code shown that it's not setting the binary
>>>>>> paths for gpdb database servers. Shouldn't it be something like:
>>>>>>
>>>>>> config.DEFAULT_BINARY_PATHS = {
>>>>>> "pg": str(server['default_binary_paths']['pg']),
>>>>>> "ppas": str(server['default_binary_paths']['ppas']),
>>>>>> "gpdb": str(server['default_binary_paths']['gpdb'])
>>>>>> }
>>>>>>
>>>>>> Without this code, the test cases should work as I already set paths
>>>>> through below code.
>>>>>
>>>>> test_utils.set_preference(server['default_binary_paths'])
>>>>>
>>>>>
>>>> In that case, why is the code above required at all?
>>>>
>>>> My bad. Removed this code and also updated set_preference function for
>>> greenplum database.
>>> Please find the attached updated patch.
>>>
>>>>
>>>>
>>>>>
>>>>>>
>>>>>>> test_backup_utils.py file name is misleading, these are not tests,
>>>>>>>> are helpers.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Victoria & Joao
>>>>>>>>
>>>>>>>> On Mon, Jun 4, 2018 at 1:36 AM Khushboo Vashi <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Jun 2, 2018 at 3:01 AM, Dave Page <[email protected]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> This looks good, except that it's leaving the
>>>>>>>>>> test_restore_database behind. Can we clean that up please?
>>>>>>>>>>
>>>>>>>>>> PFA updated patch.
>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>> On Fri, Jun 1, 2018 at 7:06 AM, Khushboo Vashi <
>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Victoria,
>>>>>>>>>>>
>>>>>>>>>>> Thanks for reviewing the patch.
>>>>>>>>>>> The tests were failing due to the latest commit
>>>>>>>>>>> #2b4605a9d390cb44e5dfe9967c3adf2b28d04f1f - Ensure
>>>>>>>>>>> backup/restore/maintenance work via SSH tunnels. Fixes #3355
>>>>>>>>>>>
>>>>>>>>>>> I have fixed the issues and attached the updated patch.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Khushboo
>>>>>>>>>>>
>>>>>>>>>>> On Thu, May 31, 2018 at 10:00 PM, Victoria Henry <
>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi there,
>>>>>>>>>>>>
>>>>>>>>>>>> We've been noticing some issues with the tests on both our CI
>>>>>>>>>>>> and local Mac workstations.
>>>>>>>>>>>>
>>>>>>>>>>>> 1. When the following code blocks are invoked - we get
>>>>>>>>>>>> plenty of app.context() issues. It must not be valid when
>>>>>>>>>>>> running tests.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> from pgadmin.utils.driver import get_driver
>>>>>>>>>>>> driver = get_driver(PG_DEFAULT_DRIVER)
>>>>>>>>>>>> manager = driver.connection_manager(self.sid)
>>>>>>>>>>>>
>>>>>>>>>>>> host = manager.local_bind_host if manager.use_ssh_tunnel else s.host
>>>>>>>>>>>> port = manager.local_bind_port if manager.use_ssh_tunnel else s.port
>>>>>>>>>>>>
>>>>>>>>>>>> 2. When we finally enable
>>>>>>>>>>>>
>>>>>>>>>>>> "default_binary_paths": {
>>>>>>>>>>>>
>>>>>>>>>>>> in our test_config, we get more failing tests that look like:
>>>>>>>>>>>>
>>>>>>>>>>>> ======================================================================
>>>>>>>>>>>> FAIL: runTest (pgadmin.tools.restore.tests.test_restore_create_job_unit_test.RestoreCreateJobTest)
>>>>>>>>>>>> When restore object with option - Miscellaneous
>>>>>>>>>>>> ----------------------------------------------------------------------
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>> File "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/unittest/mock.py", line 1179, in patched
>>>>>>>>>>>> return func(*args, **keywargs)
>>>>>>>>>>>> File "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/tests/test_restore_create_job_unit_test.py", line 295, in runTest
>>>>>>>>>>>> self.assertEquals(response.status_code, 200)
>>>>>>>>>>>> AssertionError: 410 != 200
>>>>>>>>>>>>
>>>>>>>>>>>> And
>>>>>>>>>>>>
>>>>>>>>>>>> When restore object with the sections options ... 2018-05-31 12:24:42,988: ERROR pgadmin: illegal environment variable name
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>> File "/Users/pivotal/workspace/pgadmin4/web/pgadmin/tools/restore/__init__.py", line 352, in create_restore_job
>>>>>>>>>>>> manager.export_password_env(p.id)
>>>>>>>>>>>> File "/Users/pivotal/workspace/pgadmin4/web/pgadmin/utils/driver/psycopg2/server_manager.py", line 365, in export_password_env
>>>>>>>>>>>> os.environ[str(env)] = password
>>>>>>>>>>>> File "/Users/pivotal/.pyenv/versions/3.6.5/lib/python3.6/os.py", line 675, in __setitem__
>>>>>>>>>>>> self.putenv(key, value)
>>>>>>>>>>>> ValueError: illegal environment variable name
>>>>>>>>>>>> FAIL
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Sincerely,
>>>>>>>>>>>>
>>>>>>>>>>>> Victoria && Anthony
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, May 31, 2018 at 1:16 AM Khushboo Vashi <
>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please find the attached updated patch with the fixes.
>>>>>>>>>>>>> The test cases were only failing on MAC not on Linux.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Khushboo
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, May 30, 2018 at 10:13 AM, Khushboo Vashi <
>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, May 30, 2018 at 1:05 AM, Dave Page <[email protected]
>>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, May 28, 2018 at 8:09 AM, Khushboo Vashi <
>>>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> please find the attached updated patch for the test cases
>>>>>>>>>>>>>>>> of Backup, Restore and Maintenance modules which includes:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. Unit test cases
>>>>>>>>>>>>>>>> 2. End to end regression test cases
>>>>>>>>>>>>>>>> 3. Feature test cases
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks. I've yet to be able to run the feature tests
>>>>>>>>>>>>>>> successfully. Here's what I've found so far:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) DEFAULT_BINARY_PATHS should be default_binary_paths in
>>>>>>>>>>>>>>> the JSON config file.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Will do.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) I've hit screensize related issues:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ============================================================
>>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ERROR: runTest (pgadmin.feature_tests.pg_util
>>>>>>>>>>>>>>> ities_maintenance_test.PGUtilitiesMaintenanceFeatureTest)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Test for PG maintenance: database
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>>>>>> ----------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web
>>>>>>>>>>>>>>> /pgadmin/feature_tests/pg_utilities_maintenance_test.py",
>>>>>>>>>>>>>>> line 56, in runTest
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> self._open_maintenance_dialogue()
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web
>>>>>>>>>>>>>>> /pgadmin/feature_tests/pg_utilities_maintenance_test.py",
>>>>>>>>>>>>>>> line 75, in _open_maintenance_dialogue
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> "*[.='" + self.table_name +
>>>>>>>>>>>>>>> "']/../*[@class='aciTreeItem'"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga
>>>>>>>>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>>>>>>>>>>>> line 80, in click
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> self._execute(Command.CLICK_ELEMENT)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga
>>>>>>>>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
>>>>>>>>>>>>>>> line 628, in _execute
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> return self._parent.execute(command, params)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga
>>>>>>>>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>>>>>>>>>>>>>> line 312, in execute
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> self.error_handler.check_response(response)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga
>>>>>>>>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>>>>>>>>>>>>>> line 242, in check_response
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> raise exception_class(message, screen, stacktrace)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> WebDriverException: Message: unknown error: Element <span
>>>>>>>>>>>>>>> class="aciTreeItem">...</span> is not clickable at point (223, 604). Other
>>>>>>>>>>>>>>> element would receive the click: <div class="wcFrameCenter
>>>>>>>>>>>>>>> wcPanelBackground wcScrollableX wcScrollableY" style="left: 0px; right:
>>>>>>>>>>>>>>> 0px; bottom: 0px;">...</div>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (Session info: chrome=66.0.3359.181)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (Driver info: chromedriver=2.38.552518
>>>>>>>>>>>>>>> (183d19265345f54ce39cbb94cf81ba5f15905011),platform=Mac OS
>>>>>>>>>>>>>>> X 10.12.6 x86_64)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 3) One time the test did start, but then I saw this failure:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ============================================================
>>>>>>>>>>>>>>> ==========
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ERROR: runTest (pgadmin.feature_tests.pg_util
>>>>>>>>>>>>>>> ities_backup_restore_test.PGUtilitiesBackupFeatureTest)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Test for PG utilities - Backup and Restore
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>>>>>> ----------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web
>>>>>>>>>>>>>>> /pgadmin/feature_tests/pg_utilities_backup_restore_test.py",
>>>>>>>>>>>>>>> line 93, in runTest
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> self.page.fill_input_by_field_name("file",
>>>>>>>>>>>>>>> "test_backup_file")
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web
>>>>>>>>>>>>>>> /regression/feature_utils/pgadmin_page.py", line 211, in
>>>>>>>>>>>>>>> fill_input_by_field_name
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> self.wait_for_input_field_content(field_name,
>>>>>>>>>>>>>>> field_content)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web
>>>>>>>>>>>>>>> /regression/feature_utils/pgadmin_page.py", line 251, in
>>>>>>>>>>>>>>> wait_for_input_field_content
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> "field to contain '" + str(content) + "'",
>>>>>>>>>>>>>>> input_field_has_content
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/git/pgadmin4/web
>>>>>>>>>>>>>>> /regression/feature_utils/pgadmin_page.py", line 337, in
>>>>>>>>>>>>>>> _wait_for
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> "Timed out waiting for " + waiting_for_message
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> File "/Users/dpage/.virtualenvs/pga
>>>>>>>>>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/support/wait.py",
>>>>>>>>>>>>>>> line 80, in until
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> raise TimeoutException(message, screen, stacktrace)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> TimeoutException: Message: Timed out waiting for field to
>>>>>>>>>>>>>>> contain 'test_backup_file'
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (with screenshot attached)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have ran the feature tests with multiple servers many
>>>>>>>>>>>>>> times but didn't get a single failure.
>>>>>>>>>>>>>> I have asked Akshay to run on his machine, let see what he
>>>>>>>>>>>>>> gets.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Khushboo
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Apr 25, 2018 at 9:40 PM, Joao De Almeida Pereira <
>>>>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Khushboo,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We reviewed the patch and it is very nice to see some more
>>>>>>>>>>>>>>>>> coverage in this area. Good job :D
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We passed the tests through our CI the feature tests are
>>>>>>>>>>>>>>>>> not passing, but the linter fails:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:37: [E501] line too long (91 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:53: [E501] line too long (104 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:59: [E501] line too long (85 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:62: [E501] line too long (96 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:63: [E501] line too long (91 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/feature_tests/pg_utilities_backup_test.py:70: [E501] line too long (118 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:37: [E121] continuation line under-indented for hanging indent
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:48: [E122] continuation line missing indentation or outdented
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:49: [E251] unexpected spaces around keyword / parameter equals
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:49: [E251] unexpected spaces around keyword / parameter equals
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:51: [E501] line too long (91 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:52: [E501] line too long (94 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:53: [E501] line too long (108 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:81: [E501] line too long (113 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:82: [E501] line too long (94 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:83: [E501] line too long (108 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:111: [E501] line too long (100 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:113: [E501] line too long (94 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:114: [E501] line too long (108 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_backup_message.py:147: [E501] line too long (93 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:40: [E121] continuation line under-indented for hanging indent
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:51: [E122] continuation line missing indentation or outdented
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:135: [E501] line too long (80 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:137: [E501] line too long (83 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:138: [E122] continuation line missing indentation or outdented
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:139: [E122] continuation line missing indentation or outdented
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:140: [E122] continuation line missing indentation or outdented
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:191: [E501] line too long (81 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:203: [E501] line too long (80 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:204: [E128] continuation line under-indented for visual indent
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:204: [E501] line too long (94 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:205: [E128] continuation line under-indented for visual indent
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:205: [E501] line too long (94 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_batch_process.py:216: [W391] blank line at end of file
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_create_backup_job.py:296: [E501] line too long (97 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_create_backup_job.py:317: [E303] too many blank lines (2)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_create_backup_job.py:336: [E501] line too long (84 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> ./pgadmin/tools/backup/tests/test_create_backup_job.py:371: [W391] blank line at end of file
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> 2 E121 continuation line under-indented for hanging indent
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> 5 E122 continuation line missing indentation or outdented
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> 2 E128 continuation line under-indented for visual indent
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> 2 E251 unexpected spaces around keyword / parameter equals
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> 1 E303 too many blank lines (2)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> 24 E501 line too long (91 > 79 characters)
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> 2 W391 blank line at end of file
>>>>>>>>>>>>>>>>> <https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/jobs/run-linter/builds/17...;
>>>>>>>>>>>>>>>>> 38
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> For the feature tests, we realized we had to update the
>>>>>>>>>>>>>>>>> configuration, and we did that, but we get the following error attached. We
>>>>>>>>>>>>>>>>> spent some time trying to understand the problem but we were not successful.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Codewise:
>>>>>>>>>>>>>>>>> - We just found some One Letter Variables in the code...
>>>>>>>>>>>>>>>>> - Looks like there is a bug report in this area of the
>>>>>>>>>>>>>>>>> code and we do not have coverage for it:
>>>>>>>>>>>>>>>>> https://redmine.postgresql.org/issues/3232
>>>>>>>>>>>>>>>>> Looks like in some of the unit tests we only have happy
>>>>>>>>>>>>>>>>> path tests, maybe we should see if there are any sad paths that also need
>>>>>>>>>>>>>>>>> coverage.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The configuration change, maybe need to be updated. When
>>>>>>>>>>>>>>>>> we install multiple versions of postgres the binaries live in
>>>>>>>>>>>>>>>>> `/usr/lib/postgresql/{{db_version}}/bin`, which makes us
>>>>>>>>>>>>>>>>> think that this configuration should live near the server configuration,
>>>>>>>>>>>>>>>>> maybe? Also to maintain coherency on the naming maybe we should make it all
>>>>>>>>>>>>>>>>> lower case.
>>>>>>>>>>>>>>>>> Just as an aside, you can add the gpdb configuration as
>>>>>>>>>>>>>>>>> well in you patch.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>> Victoria & Joao
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Apr 25, 2018 at 5:20 AM Khushboo Vashi <
>>>>>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please find the attached patch which covers test cases
>>>>>>>>>>>>>>>>>> for the backup module (RM #3206).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. Unit test cases
>>>>>>>>>>>>>>>>>> 2. End to end regression test cases
>>>>>>>>>>>>>>>>>> 3. Feature test cases
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Khushboo
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>
Attachments:
[application/octet-stream] RM_3206_ver5.patch (108.2K, 3-RM_3206_ver5.patch)
download | inline diff:
diff --git a/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py b/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py
new file mode 100644
index 0000000..65b5708
--- /dev/null
+++ b/web/pgadmin/feature_tests/pg_utilities_backup_restore_test.py
@@ -0,0 +1,151 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+
+import time
+import os
+
+from selenium.webdriver.support.ui import WebDriverWait
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+from regression.feature_utils.base_feature_test import BaseFeatureTest
+from regression.python_test_utils import test_utils
+
+import config
+
+
+class PGUtilitiesBackupFeatureTest(BaseFeatureTest):
+ """ This class test PG utilities - Backup and Restore test scenarios """
+
+ scenarios = [
+ ("Test for PG utilities - Backup and Restore", dict())
+ ]
+
+ def before(self):
+ if self.server['default_binary_paths'] is None:
+ self.skipTest(
+ "default_binary_paths is not set for the server {0}".format(
+ self.server['name']
+ )
+ )
+ connection = test_utils.get_db_connection(
+ self.server['db'],
+ self.server['username'],
+ self.server['db_password'],
+ self.server['host'],
+ self.server['port'],
+ self.server['sslmode']
+ )
+ test_utils.drop_database(connection, "pg_utility_test_db")
+
+ test_utils.create_database(self.server, "pg_utility_test_db")
+ self.page.add_server(self.server)
+
+ self.wait = WebDriverWait(self.page.driver, 20)
+
+ def runTest(self):
+ self.page.toggle_open_server(self.server['name'])
+ self.page.toggle_open_tree_item('Databases')
+ self.page.toggle_open_tree_item('pg_utility_test_db')
+ self.driver.find_element_by_link_text("Tools").click()
+
+ self.page.find_by_partial_link_text("Backup...").click()
+
+ self.wait.until(EC.presence_of_element_located(
+ (
+ By.XPATH,
+ "//label[contains(string(), 'Filename')]"
+ )
+ ))
+
+ self.wait.until(EC.element_to_be_clickable(
+ (By.CSS_SELECTOR, ".browse_file_input"))).click()
+
+ self.page.fill_input_by_field_name("file", "test_backup")
+
+ self.page.find_by_xpath("//button[contains(@class,'fa-save') "
+ "and contains(.,'Backup')]").click()
+
+ self.page.wait_for_element_to_disappear(
+ lambda driver: driver.find_element_by_css_selector(".ajs-modal")
+ )
+
+ status = self.page.find_by_xpath("//div[contains(@class,"
+ "'bg-success')]").text
+
+ self.assertEquals(status, "Successfully completed.")
+ self.page.find_by_xpath("//span[contains(string(), "
+ "'Click here for details.')]").click()
+ command = self.page.find_by_xpath("//p[contains(@class, "
+ "'bg-detailed-desc')]").text
+
+ self.assertIn(self.server['name'], str(command))
+ self.assertIn("from database 'pg_utility_test_db'", str(command))
+ self.assertIn("test_backup", str(command))
+ self.assertIn("pg_dump", str(command))
+
+ backup_file = None
+ if command:
+ backup_file = command[int(command.find('--file')) + 8:
+ int(command.find('--host')) - 2]
+
+ self.page.find_by_xpath("//div[contains(@class,'wcFloatingFocus')"
+ "]//div[contains(@class,'fa-close')]").click()
+
+ self.driver.find_element_by_link_text("Tools").click()
+ self.page.find_by_partial_link_text("Restore...").click()
+
+ self.wait.until(EC.presence_of_element_located(
+ (
+ By.XPATH,
+ "//label[contains(string(), 'Filename')]"
+ )
+ ))
+
+ self.wait.until(EC.element_to_be_clickable(
+ (By.CSS_SELECTOR, ".browse_file_input"))).click()
+
+ self.page.fill_input_by_field_name("file", "test_backup")
+ self.page.find_by_xpath("//button[contains(@class,'fa-upload')"
+ " and contains(.,'Restore')]").click()
+
+ self.page.wait_for_element_to_disappear(
+ lambda driver: driver.find_element_by_css_selector(".ajs-modal")
+ )
+
+ status = self.page.find_by_xpath("//div[contains(@class,"
+ "'bg-success')]").text
+
+ self.assertEquals(status, "Successfully completed.")
+ self.page.find_by_xpath("//span[contains(string(),"
+ " 'Click here for details.')]").click()
+ command = self.page.find_by_xpath("//p[contains(@class,"
+ " 'bg-detailed-desc')]").text
+
+ self.assertIn(self.server['name'], str(command))
+ self.assertIn("test_backup", str(command))
+ self.assertIn("pg_restore", str(command))
+
+ self.page.find_by_xpath("//div[contains(@class,'wcFloatingFocus')]"
+ "//div[contains(@class,'fa-close')]").click()
+
+ if backup_file is not None:
+ if os.path.isfile(backup_file):
+ os.remove(backup_file)
+
+ def after(self):
+ self.page.remove_server(self.server)
+ connection = test_utils.get_db_connection(
+ self.server['db'],
+ self.server['username'],
+ self.server['db_password'],
+ self.server['host'],
+ self.server['port'],
+ self.server['sslmode']
+ )
+ test_utils.drop_database(connection, "pg_utility_test_db")
diff --git a/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py b/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py
new file mode 100644
index 0000000..7806268
--- /dev/null
+++ b/web/pgadmin/feature_tests/pg_utilities_maintenance_test.py
@@ -0,0 +1,112 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+# _*_ coding: utf-8 _*_
+import time
+from selenium.webdriver.support.ui import WebDriverWait
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+from regression.feature_utils.base_feature_test import BaseFeatureTest
+from regression.python_test_utils import test_utils
+
+
+class PGUtilitiesMaintenanceFeatureTest(BaseFeatureTest):
+ """ This class test PG utilities test scenarios """
+
+ scenarios = [
+ ("Test for PG maintenance: database pg_maintenance", dict(
+ database_name='pg_maintenance',
+ table_name='pg_maintenance_table',
+ test_level='database'
+ )),
+ ("Test for PG maintenance: database", dict(
+ database_name='pg_maintenance',
+ table_name='pg_maintenance_table',
+ test_level='table'
+ )),
+ ]
+
+ def before(self):
+ if self.server['default_binary_paths'] is None:
+ self.skipTest(
+ "default_binary_paths is not set for the server {0}".format(
+ self.server['name']
+ )
+ )
+ connection = test_utils.get_db_connection(
+ self.server['db'],
+ self.server['username'],
+ self.server['db_password'],
+ self.server['host'],
+ self.server['port'],
+ self.server['sslmode']
+ )
+ test_utils.create_database(self.server, self.database_name)
+ test_utils.create_table(self.server, self.database_name,
+ self.table_name)
+ self.page.add_server(self.server)
+ self.wait = WebDriverWait(self.page.driver, 20)
+
+ def runTest(self):
+ self._open_maintenance_dialogue()
+ # time.sleep
+ self.page.find_by_xpath("//button[contains(@class,'fa-save') and"
+ " contains(.,'OK')]").click()
+ self.page.wait_for_element_to_disappear(
+ lambda driver: driver.find_element_by_css_selector(".ajs-modal")
+ )
+ self._verify_command()
+
+ def _open_maintenance_dialogue(self):
+ self.page.toggle_open_server(self.server['name'])
+ self.page.toggle_open_tree_item('Databases')
+ self.page.toggle_open_tree_item(self.database_name)
+ if self.test_level == 'table':
+ self.page.toggle_open_tree_item('Schemas')
+ self.page.toggle_open_tree_item('public')
+ self.page.toggle_open_tree_item('Tables')
+ self.page.find_by_xpath(
+ "//*[@id='tree']//"
+ "*[.='" + self.table_name + "']/../*[@class='aciTreeItem'"
+ "]").click()
+ self.driver.find_element_by_link_text("Tools").click()
+ self.page.find_by_partial_link_text("Maintenance...").click()
+ time.sleep(0.5)
+
+ def _verify_command(self):
+ status = self.page.find_by_xpath("//div[contains(@class,"
+ "'bg-success')]").text
+ self.assertEquals(status, "Successfully completed.")
+ self.page.find_by_xpath("//span[contains(string(),"
+ " 'Click here for details.')]").click()
+ command = self.page.find_by_xpath("//p[contains(@class,"
+ " 'bg-detailed-desc')]").text
+ if self.test_level == 'database':
+ self.assertEquals(command, "VACUUM "
+ "(VERBOSE)\nRunning Query:"
+ "\nVACUUM VERBOSE;")
+ else:
+ self.assertEquals(command, "VACUUM "
+ "(VERBOSE)\nRunning Query:"
+ "\nVACUUM VERBOSE"
+ " public." + self.table_name + ";")
+
+ self.page.find_by_xpath("//div[contains(@class,'wcFloatingFocus')]//"
+ "div[contains(@class,'fa-close')]").click()
+
+ def after(self):
+ self.page.remove_server(self.server)
+ connection = test_utils.get_db_connection(
+ self.server['db'],
+ self.server['username'],
+ self.server['db_password'],
+ self.server['host'],
+ self.server['port'],
+ self.server['sslmode']
+ )
+ test_utils.drop_database(connection, self.database_name)
diff --git a/web/pgadmin/tools/backup/__init__.py b/web/pgadmin/tools/backup/__init__.py
index 125db80..0513365 100644
--- a/web/pgadmin/tools/backup/__init__.py
+++ b/web/pgadmin/tools/backup/__init__.py
@@ -109,8 +109,7 @@ class BackupMessage(IProcessDesc):
else:
self.cmd += cmdArg(arg)
- @property
- def message(self):
+ def get_server_details(self):
# Fetch the server details like hostname, port, roles etc
s = Server.query.filter_by(
id=self.sid, user_id=current_user.id
@@ -123,13 +122,19 @@ class BackupMessage(IProcessDesc):
host = manager.local_bind_host if manager.use_ssh_tunnel else s.host
port = manager.local_bind_port if manager.use_ssh_tunnel else s.port
+ return s.name, host, port
+
+ @property
+ def message(self):
+ name, host, port = self.get_server_details()
+
if self.backup_type == BACKUP.OBJECT:
return _(
"Backing up an object on the server '{0}' "
"from database '{1}'..."
).format(
"{0} ({1}:{2})".format(
- s.name, host, port
+ name, host, port
),
self.database
)
@@ -137,13 +142,13 @@ class BackupMessage(IProcessDesc):
return _("Backing up the global objects on "
"the server '{0}'...").format(
"{0} ({1}:{2})".format(
- s.name, host, port
+ name, host, port
)
)
elif self.backup_type == BACKUP.SERVER:
return _("Backing up the server '{0}'...").format(
"{0} ({1}:{2})".format(
- s.name, host, port
+ name, host, port
)
)
else:
@@ -151,17 +156,7 @@ class BackupMessage(IProcessDesc):
return "Unknown Backup"
def details(self, cmd, args):
- # Fetch the server details like hostname, port, roles etc
- s = Server.query.filter_by(
- id=self.sid, user_id=current_user.id
- ).first()
-
- from pgadmin.utils.driver import get_driver
- driver = get_driver(PG_DEFAULT_DRIVER)
- manager = driver.connection_manager(self.sid)
-
- host = manager.local_bind_host if manager.use_ssh_tunnel else s.host
- port = manager.local_bind_port if manager.use_ssh_tunnel else s.port
+ name, host, port = self.get_server_details()
res = '<div class="h5">'
@@ -171,7 +166,7 @@ class BackupMessage(IProcessDesc):
"from database '{1}'..."
).format(
"{0} ({1}:{2})".format(
- html.safe_str(s.name),
+ html.safe_str(name),
html.safe_str(host),
html.safe_str(port),
),
@@ -181,7 +176,7 @@ class BackupMessage(IProcessDesc):
res += _("Backing up the global objects on "
"the server '{0}'...").format(
"{0} ({1}:{2})".format(
- html.safe_str(s.name),
+ html.safe_str(name),
html.safe_str(host),
html.safe_str(port)
)
@@ -189,7 +184,7 @@ class BackupMessage(IProcessDesc):
elif self.backup_type == BACKUP.SERVER:
res += _("Backing up the server '{0}'...").format(
"{0} ({1}:{2})".format(
- html.safe_str(s.name),
+ html.safe_str(name),
html.safe_str(host),
html.safe_str(port)
)
diff --git a/web/pgadmin/tools/backup/tests/__init__.py b/web/pgadmin/tools/backup/tests/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/web/pgadmin/tools/backup/tests/test_backup_create_job_unit_test.py b/web/pgadmin/tools/backup/tests/test_backup_create_job_unit_test.py
new file mode 100644
index 0000000..a376a3b
--- /dev/null
+++ b/web/pgadmin/tools/backup/tests/test_backup_create_job_unit_test.py
@@ -0,0 +1,463 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+
+
+import sys
+import simplejson as json
+
+from pgadmin.misc.bgprocess.processes import BatchProcess
+from pgadmin.tools.backup import BackupMessage
+from pgadmin.utils.route import BaseTestGenerator
+from regression import parent_node_dict
+from regression.python_test_utils import test_utils as utils
+from pgadmin.utils import server_utils as server_utils
+from pgadmin.browser.server_groups.servers.databases.tests import utils as \
+ database_utils
+
+
+if sys.version_info < (3, 3):
+ from mock import patch, MagicMock
+else:
+ from unittest.mock import patch, MagicMock
+
+
+class BackupCreateJobTest(BaseTestGenerator):
+ """Test the BackupCreateJob class"""
+ scenarios = [
+ ('When backup object with default options',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='custom',
+ verbose=True,
+ blobs=True,
+ schemas=[],
+ tables=[],
+ database='postgres'
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--verbose', '--format=c', '--blobs'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with option sections to all data',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='custom',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ data=True,
+ pre_data=True,
+ post_data=True
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--verbose', '--format=c',
+ '--section=pre-data', '--section=data',
+ '--section=post-data'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with option only_data',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='plain',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ only_data=True,
+ only_schema=False
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--verbose', '--format=p', '--data-only'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with option only_data',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='plain',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ only_data=True,
+ only_schema=True,
+ dns_owner=True
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--verbose', '--format=p', '--data-only'],
+ not_expected_cmd_opts=['--schema-only', '--no-owner'],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with option only_schema',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='plain',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ only_data=False,
+ only_schema=True
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--verbose', '--format=p', '--schema-only'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with option - format plain and dns_owner',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='plain',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ dns_owner=True
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--verbose', '--format=p', '--no-owner'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with option - Do not save privilege,'
+ ' tablespace, unlogged table data',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='custom',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ dns_privilege=True,
+ dns_unlogged_tbl_data=True,
+ dns_tablespace=True
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--no-privileges',
+ '--no-tablespaces',
+ '--no-unlogged-table-data'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with option - all queries',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='plain',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ use_column_inserts=True,
+ include_create_database=True,
+ use_insert_commands=True,
+ include_drop_database=True
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--create', '--clean', '--inserts',
+ '--column-inserts'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with option - all queries and format custom',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='custom',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ use_column_inserts=True,
+ include_create_database=True,
+ use_insert_commands=True,
+ include_drop_database=True
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--inserts',
+ '--column-inserts'],
+ not_expected_cmd_opts=['--create', '--clean'],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with option - miscellaneous',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='custom',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ disable_quoting=True,
+ use_set_session_auth=True,
+ with_oids=True,
+ dqoute=True
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--verbose', '--quote-all-identifiers',
+ '--disable-dollar-quoting', '--oids',
+ '--use-set-session-authorization'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the object with format tar',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='tar',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ blobs=True,
+ ),
+ url='/backup/job/{0}/object',
+ expected_cmd_opts=['--verbose',
+ '--blobs',
+ '--format=t'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup the server',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_server_file',
+ dqoute=False,
+ verbose=True,
+ type='server'
+ ),
+ url='/backup/job/{0}',
+ expected_cmd_opts=['--verbose'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When backup globals',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_global_file',
+ dqoute=False,
+ verbose=True,
+ type='globals'
+ ),
+ url='/backup/job/{0}',
+ expected_cmd_opts=['--verbose'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ ))
+ ]
+
+ def setUp(self):
+ if self.server['default_binary_paths'] is None:
+ self.skipTest(
+ "default_binary_paths is not set for the server {0}".format(
+ self.server['name']
+ )
+ )
+
+ @patch('pgadmin.tools.backup.Server')
+ @patch('pgadmin.tools.backup.current_user')
+ @patch('pgadmin.tools.backup.BackupMessage')
+ @patch('pgadmin.tools.backup.filename_with_file_manager_path')
+ @patch('pgadmin.tools.backup.BatchProcess')
+ @patch('pgadmin.utils.driver.psycopg2.server_manager.ServerManager.'
+ 'export_password_env')
+ def runTest(self, export_password_env_mock, batch_process_mock,
+ filename_mock, backup_message_mock,
+ current_user_mock, server_mock):
+ class TestMockServer():
+ def __init__(self, name, host, port, id, username,
+ maintenance_db):
+ self.name = name
+ self.host = host
+ self.port = port
+ self.id = id
+ self.username = username
+ self.maintenance_db = maintenance_db
+
+ self.db_name = ''
+ self.server_id = parent_node_dict["server"][-1]["server_id"]
+ mock_obj = TestMockServer(self.class_params['name'],
+ self.class_params['host'],
+ self.class_params['port'],
+ self.server_id,
+ self.class_params['username'],
+ self.class_params['database']
+ )
+ mock_result = server_mock.query.filter_by.return_value
+ mock_result.first.return_value = mock_obj
+
+ filename_mock.return_value = self.params['file']
+
+ batch_process_mock.set_env_variables = MagicMock(
+ return_value=True
+ )
+ batch_process_mock.start = MagicMock(
+ return_value=True
+ )
+
+ export_password_env_mock.return_value = True
+
+ server_response = server_utils.connect_server(self, self.server_id)
+ if server_response["info"] == "Server connected.":
+ db_owner = server_response['data']['user']['name']
+ self.data = database_utils.get_db_data(db_owner)
+ self.db_name = self.data['name']
+
+ url = self.url.format(self.server_id)
+
+ # Create the backup job
+ response = self.tester.post(url,
+ data=json.dumps(self.params),
+ content_type='html/json')
+ self.assertEquals(response.status_code, 200)
+ response_data = json.loads(response.data.decode('utf-8'))
+ job_id = response_data['data']['job_id']
+
+ assert backup_message_mock.called
+ assert batch_process_mock.called
+
+ if self.expected_cmd_opts:
+ for opt in self.expected_cmd_opts:
+ self.assertIn(
+ opt,
+ batch_process_mock.call_args_list[0][1]['args']
+ )
+ if self.not_expected_cmd_opts:
+ for opt in self.not_expected_cmd_opts:
+ self.assertNotIn(
+ opt,
+ batch_process_mock.call_args_list[0][1]['args']
+ )
diff --git a/web/pgadmin/tools/backup/tests/test_backup_message.py b/web/pgadmin/tools/backup/tests/test_backup_message.py
new file mode 100644
index 0000000..34eacc9
--- /dev/null
+++ b/web/pgadmin/tools/backup/tests/test_backup_message.py
@@ -0,0 +1,149 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+import sys
+
+from flask import Response
+import simplejson as json
+
+from pgadmin.tools.backup import BackupMessage, BACKUP
+from pgadmin.utils.route import BaseTestGenerator
+
+if sys.version_info < (3, 3):
+ from mock import patch, MagicMock
+else:
+ from unittest.mock import patch, MagicMock
+
+
+class BackupMessageTest(BaseTestGenerator):
+ """Test the BackupMessage class"""
+ scenarios = [
+ ('When Backup server',
+ dict(
+ class_params=dict(
+ type=BACKUP.SERVER,
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_restore',
+ args=[
+ '--file',
+ "backup_file",
+ '--host',
+ "localhost",
+ '--port',
+ "5444",
+ '--username',
+ "postgres",
+ '--no-password',
+ '--database',
+ "postgres"
+ ],
+ cmd="/test_path/pg_dump"
+ ),
+ extected_msg="Backing up the server"
+ " 'test_backup_server (localhost:5444)'...",
+ expetced_details_cmd='/test_path/pg_dump --file '
+ '"backup_file" --host "localhost" '
+ '--port "5444" --username "postgres" '
+ '--no-password --database "postgres"'
+
+ )),
+ ('When Backup global',
+ dict(
+ class_params=dict(
+ type=BACKUP.GLOBALS,
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ args=[
+ '--file',
+ 'backup_file',
+ '--host',
+ 'localhost',
+ '--port',
+ '5444',
+ '--username',
+ 'postgres',
+ '--no-password',
+ '--database',
+ 'postgres'
+ ],
+ cmd="/test_path/pg_dump"
+ ),
+ extected_msg="Backing up the global objects on the server "
+ "'test_backup_server (localhost:5444)'...",
+ expetced_details_cmd='/test_path/pg_dump --file "backup_file" '
+ '--host "localhost"'
+ ' --port "5444" --username "postgres" '
+ '--no-password --database "postgres"'
+
+ )),
+ ('When backup object',
+ dict(
+ class_params=dict(
+ type=BACKUP.OBJECT,
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_backup',
+ args=[
+ '--file',
+ 'backup_file',
+ '--host',
+ 'localhost',
+ '--port',
+ '5444',
+ '--username',
+ 'postgres',
+ '--no-password',
+ '--database',
+ 'postgres'
+ ],
+ cmd="/test_path/pg_dump"
+ ),
+ extected_msg="Backing up an object on the server "
+ "'test_backup_server (localhost:5444)'"
+ " from database 'postgres'...",
+ expetced_details_cmd='/test_path/pg_dump --file "backup_file" '
+ '--host "localhost" '
+ '--port "5444" --username "postgres" '
+ '--no-password --database "postgres"'
+
+ ))
+ ]
+
+ @patch('pgadmin.tools.backup.BackupMessage.get_server_details')
+ def runTest(self, get_server_details_mock):
+ get_server_details_mock.return_value = \
+ self.class_params['name'],\
+ self.class_params['host'],\
+ self.class_params['port']
+
+ backup_obj = BackupMessage(
+ self.class_params['type'],
+ self.class_params['sid'],
+ self.class_params['bfile'],
+ *self.class_params['args'],
+ **{'database': self.class_params['database']}
+ )
+
+ # Check the expected message returned
+ assert backup_obj.message == self.extected_msg
+
+ # Check the command
+ obj_details = backup_obj.details(self.class_params['cmd'],
+ self.class_params['args'])
+ self.assertIn(self.expetced_details_cmd, obj_details)
diff --git a/web/pgadmin/tools/backup/tests/test_backup_utils.py b/web/pgadmin/tools/backup/tests/test_backup_utils.py
new file mode 100644
index 0000000..c4432b6
--- /dev/null
+++ b/web/pgadmin/tools/backup/tests/test_backup_utils.py
@@ -0,0 +1,119 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+
+import time
+import random
+import simplejson as json
+
+
+def create_backup_job(tester, url, params):
+ # Create the backup job
+ response = tester.post(url,
+ data=json.dumps(params),
+ content_type='html/json')
+ assert response.status_code == 200
+ response_data = json.loads(response.data.decode('utf-8'))
+ job_id = response_data['data']['job_id']
+ return job_id
+
+
+def run_backup_job(tester, job_id, expected_params, assertIn, assertNotIn):
+ cnt = 0
+ while 1:
+ if cnt > 1:
+ break
+ # Check the process list
+ response1 = tester.get('/misc/bgprocess/?_='.format(
+ random.randint(1, 9999999)))
+ assert response1.status_code == 200
+ process_list = json.loads(response1.data.decode('utf-8'))
+
+ if len(process_list) > 0 and 'execution_time' in process_list[0]:
+ break
+ time.sleep(0.5)
+ cnt += 1
+
+ assert 'execution_time' in process_list[0]
+ assert 'stime' in process_list[0]
+ assert 'exit_code' in process_list[0]
+ assert process_list[0]['exit_code'] in expected_params[
+ 'expected_exit_code'
+ ]
+
+ backup_file = None
+ if 'details' in process_list[0]:
+ backup_det = process_list[0]['details']
+ backup_file = backup_det[int(backup_det.find('--file')) + 8:
+ int(backup_det.find('--host')) - 2]
+
+ if expected_params['expected_cmd_opts']:
+ for opt in expected_params['expected_cmd_opts']:
+ assertIn(opt, process_list[0]['details'])
+ if expected_params['not_expected_cmd_opts']:
+ for opt in expected_params['not_expected_cmd_opts']:
+ assertNotIn(opt, process_list[0]['details'])
+
+ # Check the process details
+ p_details = tester.get('/misc/bgprocess/{0}?_='.format(
+ job_id, random.randint(1, 9999999))
+ )
+ assert p_details.status_code == 200
+ p_details_data = json.loads(p_details.data.decode('utf-8'))
+
+ p_details = tester.get('/misc/bgprocess/{0}/{1}/{2}/?_='.format(
+ job_id, 0, 0, random.randint(1, 9999999))
+ )
+ assert p_details.status_code == 200
+ p_details_data = json.loads(p_details.data.decode('utf-8'))
+
+ cnt = 0
+ # Retrieve the backup job process logs
+ while 1:
+ out, err, status = get_params(p_details_data)
+ if status or cnt >= 10:
+ break
+
+ p_details = tester.get(
+ '/misc/bgprocess/{0}/{1}/{2}/?_={3}'.format(
+ job_id, out, err, random.randint(1, 9999999))
+ )
+ assert p_details.status_code == 200
+ p_details_data = json.loads(p_details.data.decode('utf-8'))
+
+ cnt += 1
+ time.sleep(1)
+
+ # Check the job is complete.
+ backup_ack = tester.put('/misc/bgprocess/{0}'.format(job_id))
+ assert backup_ack.status_code == 200
+ backup_ack_res = json.loads(backup_ack.data.decode('utf-8'))
+
+ assert backup_ack_res['success'] == 1
+
+ return backup_file
+
+
+def get_params(data):
+ out = 0
+ out_done = False
+ err = 0
+ err_done = False
+ if 'out' in data:
+ out = data['out'] and data['out']['pos']
+
+ if 'done' in data['out']:
+ out_done = data['out']['done']
+
+ if 'err' in data:
+ err = data['err'] and data['err']['pos']
+
+ if 'done' in data['err']:
+ err_done = data['err']['done']
+
+ return out, err, (out_done and err_done)
diff --git a/web/pgadmin/tools/backup/tests/test_batch_process.py b/web/pgadmin/tools/backup/tests/test_batch_process.py
new file mode 100644
index 0000000..c074ca5
--- /dev/null
+++ b/web/pgadmin/tools/backup/tests/test_batch_process.py
@@ -0,0 +1,212 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+import sys
+import simplejson as json
+
+from pgadmin.misc.bgprocess.processes import BatchProcess, IProcessDesc
+from pgadmin.tools.backup import BackupMessage, BACKUP
+from pgadmin.utils.route import BaseTestGenerator
+from pickle import dumps, loads
+
+if sys.version_info < (3, 3):
+ from mock import patch
+else:
+ from unittest.mock import patch
+
+
+class BatchProcessTest(BaseTestGenerator):
+ """Test the BatchProcess class"""
+ scenarios = [
+ ('When backup server',
+ dict(
+ class_params=dict(
+ type=BACKUP.SERVER,
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ username='postgres',
+ bfile='test_backup',
+ args=[
+ '--file',
+ "backup_file",
+ '--host',
+ "localhost",
+ '--port',
+ "5444",
+ '--username',
+ "postgres",
+ '--no-password',
+ '--database',
+ "postgres"
+ ],
+ cmd='backup_server'
+ )
+ )),
+ ('When backup globals',
+ dict(
+ class_params=dict(
+ type=BACKUP.GLOBALS,
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ username='postgres',
+ bfile='test_backup',
+ args=[
+ '--file',
+ "backup_file",
+ '--host',
+ "localhost",
+ '--port',
+ "5444",
+ '--username',
+ "postgres",
+ '--no-password',
+ '--database',
+ "postgres"
+ ],
+ cmd='backup'
+ )
+ )),
+ ('When backup object',
+ dict(
+ class_params=dict(
+ type=BACKUP.OBJECT,
+ sid=1,
+ name='test_backup_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ username='postgres',
+ bfile='test_backup',
+ args=[
+ '--file',
+ "backup_file",
+ '--host',
+ "localhost",
+ '--port',
+ "5444",
+ '--username',
+ "postgres",
+ '--no-password',
+ '--database',
+ "postgres"
+ ],
+ cmd='backup'
+ )
+ ))
+ ]
+
+ @patch('pgadmin.tools.backup.BackupMessage.get_server_details')
+ @patch('pgadmin.misc.bgprocess.processes.Popen')
+ @patch('pgadmin.misc.bgprocess.processes.current_app')
+ @patch('pgadmin.misc.bgprocess.processes.db')
+ @patch('pgadmin.tools.backup.current_user')
+ @patch('pgadmin.misc.bgprocess.processes.current_user')
+ def runTest(self, current_user_mock, current_user, db_mock,
+ current_app_mock, popen_mock, get_server_details_mock):
+ current_user.id = 1
+ current_user_mock.id = 1
+ current_app_mock.PGADMIN_RUNTIME = False
+
+ def db_session_add_mock(j):
+ cmd_obj = loads(j.desc)
+ assert isinstance(cmd_obj, IProcessDesc)
+ self.assertEquals(cmd_obj.backup_type, self.class_params['type'])
+ self.assertEquals(cmd_obj.bfile, self.class_params['bfile'])
+ self.assertEquals(cmd_obj.database, self.class_params['database'])
+ self.assertEquals(cmd_obj.cmd,
+ ' --file "backup_file" '
+ '--host "{0}" '
+ '--port "{1}" '
+ '--username "{2}" '
+ '--no-password '
+ '--database "{3}"'.format(
+ self.class_params['host'],
+ self.class_params['port'],
+ self.class_params['username'],
+ self.class_params['database']
+ ))
+
+ db_mock.session.add.side_effect = db_session_add_mock
+
+ get_server_details_mock.return_value = \
+ self.class_params['name'],\
+ self.class_params['host'],\
+ self.class_params['port']
+
+ backup_obj = BackupMessage(
+ self.class_params['type'],
+ self.class_params['sid'],
+ self.class_params['bfile'],
+ *self.class_params['args'],
+ **{'database': self.class_params['database']}
+ )
+
+ p = BatchProcess(
+ desc=backup_obj,
+ cmd=self.class_params['cmd'],
+ args=self.class_params['args']
+ )
+
+ # Check that _create_process has been called
+ assert db_mock.session.add.called
+
+ # Check start method
+ self._check_start(popen_mock, p)
+
+ # Check list method
+ self._check_list(p, backup_obj)
+
+ def _check_start(self, popen_mock, p):
+ cmd_test = self.class_params['cmd']
+
+ class popenMockSideEffect():
+ def __init__(self, cmd, **kwargs):
+ assert cmd_test in cmd
+ assert 'env' in kwargs
+
+ def poll(self):
+ pass
+
+ popen_mock.side_effect = popenMockSideEffect
+ p.start()
+
+ assert popen_mock.called
+
+ @patch('pgadmin.misc.bgprocess.processes.Process')
+ @patch('pgadmin.misc.bgprocess.processes.BatchProcess.'
+ 'update_process_info')
+ def _check_list(self, p, backup_obj, update_process_info_mock,
+ process_mock):
+ class TestMockProcess():
+ def __init__(self, desc, args, cmd):
+ self.pid = 1
+ self.exit_code = 1
+ self.start_time = '2018-04-17 06:18:56.315445 +0000'
+ self.end_time = None
+ self.desc = dumps(desc)
+ self.arguments = " ".join(args)
+ self.command = cmd
+ self.acknowledge = None
+
+ process_mock.query.filter_by.return_value = [
+ TestMockProcess(backup_obj,
+ self.class_params['args'],
+ self.class_params['cmd'])]
+
+ update_process_info_mock.return_value = [True, True]
+
+ ret_value = p.list()
+ self.assertEqual(1, len(ret_value))
+ assert 'details' in ret_value[0]
+ assert 'desc' in ret_value[0]
diff --git a/web/pgadmin/tools/backup/tests/test_create_backup_job.py b/web/pgadmin/tools/backup/tests/test_create_backup_job.py
new file mode 100644
index 0000000..ed48c58
--- /dev/null
+++ b/web/pgadmin/tools/backup/tests/test_create_backup_job.py
@@ -0,0 +1,62 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+
+import os
+from pgadmin.utils.route import BaseTestGenerator
+from regression import parent_node_dict
+from regression.python_test_utils import test_utils as utils
+import pgadmin.tools.backup.tests.test_backup_utils as backup_utils
+
+class BackupJobTest(BaseTestGenerator):
+ """Backup api test cases"""
+ scenarios = [
+ ('When backup the object with the default options',
+ dict(
+ params=dict(
+ file='test_backup',
+ format='custom',
+ verbose=True,
+ blobs=True,
+ schemas=[],
+ tables=[],
+ database='postgres'
+ ),
+ url='/backup/job/{0}/object',
+ expected_params=dict(
+ expected_cmd_opts=['--verbose', '--format=c', '--blobs'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )
+ ))
+ ]
+
+ def setUp(self):
+ if self.server['default_binary_paths'] is None:
+ self.skipTest(
+ "default_binary_paths is not set for the server {0}".format(
+ self.server['name']
+ )
+ )
+
+ def runTest(self):
+ self.server_id = parent_node_dict["server"][-1]["server_id"]
+ url = self.url.format(self.server_id)
+
+ # Create the backup job
+ job_id = backup_utils.create_backup_job(self.tester, url, self.params)
+ backup_file = backup_utils.run_backup_job(self.tester,
+ job_id,
+ self.expected_params,
+ self.assertIn,
+ self.assertNotIn
+ )
+
+ if backup_file is not None:
+ if os.path.isfile(backup_file):
+ os.remove(backup_file)
diff --git a/web/pgadmin/tools/maintenance/tests/__init__.py b/web/pgadmin/tools/maintenance/tests/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/web/pgadmin/tools/maintenance/tests/test_batch_process_maintenance.py b/web/pgadmin/tools/maintenance/tests/test_batch_process_maintenance.py
new file mode 100644
index 0000000..55c3db8
--- /dev/null
+++ b/web/pgadmin/tools/maintenance/tests/test_batch_process_maintenance.py
@@ -0,0 +1,153 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+import sys
+
+from pgadmin.misc.bgprocess.processes import BatchProcess, IProcessDesc
+from pgadmin.tools.maintenance import Message
+from pgadmin.utils.route import BaseTestGenerator
+from pickle import dumps, loads
+
+if sys.version_info < (3, 3):
+ from mock import patch
+else:
+ from unittest.mock import patch
+
+
+class BatchProcessTest(BaseTestGenerator):
+ """Test the BatchProcess class"""
+ scenarios = [
+ ('When maintained server',
+ dict(
+ class_params=dict(
+ sid=1,
+ host='localhost',
+ port=5444,
+ username='postgres',
+ args=[
+ '--host',
+ "localhost",
+ '--port',
+ "5444",
+ '--username',
+ '--dbname',
+ "postgres",
+ '--command',
+ "VACUUM VERBOSE;\n"
+ ],
+ data={
+ 'database': 'postgres',
+ 'op': 'VACUUM',
+ 'vacuum_analyze': False,
+ 'vacuum_freeze': False,
+ 'vacuum_full': False,
+ 'verbose': True
+ },
+ cmd="VACUUM VERBOSE;\n"
+ ),
+ expected_msg="Maintenance (Vacuum)",
+ expetced_details_cmd='VACUUM VERBOSE;'
+ ))
+ ]
+
+ @patch('pgadmin.misc.bgprocess.processes.Popen')
+ @patch('pgadmin.misc.bgprocess.processes.current_app')
+ @patch('pgadmin.misc.bgprocess.processes.db')
+ @patch('pgadmin.tools.maintenance.Server')
+ @patch('pgadmin.misc.bgprocess.processes.current_user')
+ def runTest(self, current_user_mock, server_mock, db_mock,
+ current_app_mock, popen_mock):
+ current_user_mock.id = 1
+ current_app_mock.PGADMIN_RUNTIME = False
+
+ class TestMockServer():
+ def __init__(self, name, host, port):
+ self.name = name
+ self.host = host
+ self.port = port
+
+ def db_session_add_mock(j):
+ cmd_obj = loads(j.desc)
+ assert isinstance(cmd_obj, IProcessDesc)
+ self.assertEquals(cmd_obj.query, self.class_params['cmd'])
+ self.assertEquals(cmd_obj.message, self.expected_msg)
+ self.assertEquals(cmd_obj.data, self.class_params['data'])
+
+ mock_obj = TestMockServer(self.class_params['username'],
+ self.class_params['host'],
+ self.class_params['port'])
+ mock_result = server_mock.query.filter_by.return_value
+ mock_result.first.return_value = mock_obj
+
+ db_mock.session.add.side_effect = db_session_add_mock
+
+ maintenance_obj = Message(
+ self.class_params['sid'],
+ self.class_params['data'],
+ self.class_params['cmd']
+ )
+
+ p = BatchProcess(
+ desc=maintenance_obj,
+ cmd=self.class_params['cmd'],
+ args=self.class_params['args']
+ )
+
+ # Check that _create_process has been called
+ assert db_mock.session.add.called
+
+ # Check start method
+ self._check_start(popen_mock, p)
+
+ # Check list method
+ self._check_list(p, maintenance_obj)
+
+ def _check_start(self, popen_mock, p):
+ cmd_test = self.class_params['cmd']
+
+ class popenMockSideEffect():
+ def __init__(self, cmd, **kwargs):
+ assert cmd_test in cmd
+ assert 'env' in kwargs
+
+ def poll(self):
+ pass
+
+ popen_mock.side_effect = popenMockSideEffect
+ p.start()
+
+ assert popen_mock.called
+
+ @patch('pgadmin.misc.bgprocess.processes.Process')
+ @patch('pgadmin.misc.bgprocess.processes.BatchProcess.'
+ 'update_process_info')
+ def _check_list(self, p, maintenance_obj, update_process_info_mock,
+ process_mock):
+ class TestMockProcess():
+ def __init__(self, desc, args, cmd):
+ self.pid = 1
+ self.exit_code = 1
+ self.start_time = '2018-04-17 06:18:56.315445 +0000'
+ self.end_time = None
+ self.desc = dumps(desc)
+ self.arguments = " ".join(args)
+ self.command = cmd
+ self.acknowledge = None
+
+ process_mock.query.filter_by.return_value = [
+ TestMockProcess(maintenance_obj,
+ self.class_params['args'],
+ self.class_params['cmd'])
+ ]
+
+ update_process_info_mock.return_value = [True, True]
+
+ ret_value = p.list()
+ self.assertEqual(1, len(ret_value))
+ assert 'details' in ret_value[0]
+ assert 'desc' in ret_value[0]
diff --git a/web/pgadmin/tools/maintenance/tests/test_create_maintenance_job.py b/web/pgadmin/tools/maintenance/tests/test_create_maintenance_job.py
new file mode 100644
index 0000000..1e01f1d
--- /dev/null
+++ b/web/pgadmin/tools/maintenance/tests/test_create_maintenance_job.py
@@ -0,0 +1,140 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+import time
+import random
+import simplejson as json
+
+from pgadmin.utils.route import BaseTestGenerator
+from regression import parent_node_dict
+from regression.python_test_utils import test_utils as utils
+from pgadmin.utils import server_utils as server_utils
+from pgadmin.browser.server_groups.servers.databases.tests import utils as \
+ database_utils
+
+
+class MaintenanceJobTest(BaseTestGenerator):
+ """Maintenance api test cases"""
+ scenarios = [
+ ('When maintenance the object with the default options',
+ dict(
+ params=dict(
+ data={
+ 'database': 'postgres',
+ 'op': 'VACUUM',
+ 'vacuum_analyze': False,
+ 'vacuum_freeze': False,
+ 'vacuum_full': False,
+ 'verbose': True
+ },
+ cmd="VACUUM VERBOSE;\n"
+ ),
+ url='/maintenance/job/{0}/{1}',
+ expected_cmd='VACUUM VERBOSE',
+ expected_exit_code=[0, None]
+ ))
+ ]
+
+ def setUp(self):
+ if self.server['default_binary_paths'] is None:
+ self.skipTest(
+ "default_binary_paths is not set for the server {0}".format(
+ self.server['name']
+ )
+ )
+
+ def runTest(self):
+ self.db_name = ''
+ self.server_id = parent_node_dict["database"][-1]["server_id"]
+ self.db_id = parent_node_dict["database"][-1]["db_id"]
+ url = self.url.format(self.server_id, self.db_id)
+
+ # Create the backup job
+ response = self.tester.post(url,
+ data=json.dumps(self.params['data']),
+ content_type='html/json')
+ self.assertEquals(response.status_code, 200)
+ response_data = json.loads(response.data.decode('utf-8'))
+ job_id = response_data['data']['job_id']
+
+ cnt = 0
+ while 1:
+ if cnt > 1:
+ break
+ # Check the process list
+ response1 = self.tester.get('/misc/bgprocess/?_='.format(
+ random.randint(1, 9999999)))
+ self.assertEquals(response1.status_code, 200)
+ process_list = json.loads(response1.data.decode('utf-8'))
+
+ if len(process_list) > 0 and 'execution_time' in process_list[0]:
+ break
+ time.sleep(0.5)
+ cnt += 1
+
+ assert 'execution_time' in process_list[0]
+ assert 'stime' in process_list[0]
+ assert 'exit_code' in process_list[0]
+ assert process_list[0]['exit_code'] in self.expected_exit_code
+
+ self.assertIn(self.expected_cmd, process_list[0]['details'])
+
+ # Check the process details
+ p_details = self.tester.get('/misc/bgprocess/{0}?_='.format(
+ job_id, random.randint(1, 9999999))
+ )
+ self.assertEquals(p_details.status_code, 200)
+ p_details_data = json.loads(p_details.data.decode('utf-8'))
+
+ p_details = self.tester.get('/misc/bgprocess/{0}/{1}/{2}/?_='.format(
+ job_id, 0, 0, random.randint(1, 9999999))
+ )
+ self.assertEquals(p_details.status_code, 200)
+ p_details_data = json.loads(p_details.data.decode('utf-8'))
+
+ # Retrieve the backup job process logs
+ while 1:
+ out, err, status = MaintenanceJobTest.get_params(p_details_data)
+ if status:
+ break
+
+ p_details = self.tester.get(
+ '/misc/bgprocess/{0}/{1}/{2}/?_={3}'.format(
+ job_id, out, err, random.randint(1, 9999999))
+ )
+ self.assertEquals(p_details.status_code, 200)
+ p_details_data = json.loads(p_details.data.decode('utf-8'))
+
+ time.sleep(1)
+
+ # Check the job is complete.
+ backup_ack = self.tester.put('/misc/bgprocess/{0}'.format(job_id))
+ self.assertEquals(backup_ack.status_code, 200)
+ backup_ack_res = json.loads(backup_ack.data.decode('utf-8'))
+
+ self.assertEquals(backup_ack_res['success'], 1)
+
+ @staticmethod
+ def get_params(data):
+ out = 0
+ out_done = False
+ err = 0
+ err_done = False
+ if 'out' in data:
+ out = data['out'] and data['out']['pos']
+
+ if 'done' in data['out']:
+ out_done = data['out']['done']
+
+ if 'err' in data:
+ err = data['err'] and data['err']['pos']
+
+ if 'done' in data['err']:
+ err_done = data['err']['done']
+
+ return out, err, (out_done and err_done)
diff --git a/web/pgadmin/tools/maintenance/tests/test_maintenance_create_job_unit_test.py b/web/pgadmin/tools/maintenance/tests/test_maintenance_create_job_unit_test.py
new file mode 100644
index 0000000..58aef5c
--- /dev/null
+++ b/web/pgadmin/tools/maintenance/tests/test_maintenance_create_job_unit_test.py
@@ -0,0 +1,198 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+
+
+import sys
+import simplejson as json
+
+
+from pgadmin.misc.bgprocess.processes import BatchProcess
+from pgadmin.tools.maintenance import Message
+from pgadmin.utils.route import BaseTestGenerator
+from regression import parent_node_dict
+from regression.python_test_utils import test_utils as utils
+from pgadmin.utils import server_utils as server_utils
+from pgadmin.browser.server_groups.servers.databases.tests import utils as \
+ database_utils
+
+
+if sys.version_info < (3, 3):
+ from mock import patch, MagicMock
+else:
+ from unittest.mock import patch, MagicMock
+
+
+class MaintenanceCreateJobTest(BaseTestGenerator):
+ """Test the BackupCreateJob class"""
+ scenarios = [
+ ('When maintenance object with default options',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_maintenance_server',
+ port=5444,
+ host='localhost',
+ username='postgres'
+ ),
+ params=dict(
+ database='postgres',
+ op='VACUUM',
+ vacuum_analyze=False,
+ vacuum_freeze=False,
+ vacuum_full=False,
+ verbose=True
+ ),
+ url='/maintenance/job/{0}/{1}',
+ expected_cmd_opts=['VACUUM VERBOSE;\n'],
+ )),
+ ('When maintenance object with VACUUM FULL',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_maintenance_server',
+ port=5444,
+ host='localhost',
+ username='postgres'
+ ),
+ params=dict(
+ database='postgres',
+ op='VACUUM',
+ vacuum_analyze=False,
+ vacuum_freeze=False,
+ vacuum_full=True,
+ verbose=True
+ ),
+ url='/maintenance/job/{0}/{1}',
+ expected_cmd_opts=['VACUUM FULL VERBOSE;\n'],
+ )),
+ ('When maintenance object with the ANALYZE',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_maintenance_server',
+ port=5444,
+ host='localhost',
+ username='postgres'
+ ),
+ params=dict(
+ database='postgres',
+ op='ANALYZE',
+ vacuum_analyze=True,
+ vacuum_freeze=False,
+ vacuum_full=False,
+ verbose=True
+ ),
+ url='/maintenance/job/{0}/{1}',
+ expected_cmd_opts=['ANALYZE VERBOSE;\n'],
+ )),
+ ('When maintenance the object with the REINDEX',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_maintenance_server',
+ port=5444,
+ host='localhost',
+ username='postgres'
+ ),
+ params=dict(
+ database='postgres',
+ op='REINDEX',
+ vacuum_analyze=False,
+ vacuum_freeze=False,
+ vacuum_full=False,
+ verbose=False
+ ),
+ url='/maintenance/job/{0}/{1}',
+ expected_cmd_opts=['REINDEX DATABASE postgres;\n'],
+ )),
+ ('When maintenance the object with the CLUSTER',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_maintenance_server',
+ port=5444,
+ host='localhost',
+ username='postgres'
+ ),
+ params=dict(
+ database='postgres',
+ op='CLUSTER',
+ vacuum_analyze=False,
+ vacuum_freeze=False,
+ vacuum_full=False,
+ verbose=False
+ ),
+ url='/maintenance/job/{0}/{1}',
+ expected_cmd_opts=['CLUSTER;\n'],
+ ))
+ ]
+
+ def setUp(self):
+ if self.server['default_binary_paths'] is None:
+ self.skipTest(
+ "default_binary_paths is not set for the server {0}".format(
+ self.server['name']
+ )
+ )
+
+ @patch('pgadmin.tools.maintenance.Server')
+ @patch('pgadmin.tools.maintenance.Message')
+ @patch('pgadmin.tools.maintenance.BatchProcess')
+ @patch('pgadmin.utils.driver.psycopg2.server_manager.ServerManager.'
+ 'export_password_env')
+ def runTest(self, export_password_env_mock,
+ batch_process_mock, message_mock, server_mock):
+ self.server_id = parent_node_dict["database"][-1]["server_id"]
+ self.db_id = parent_node_dict["database"][-1]["db_id"]
+ url = self.url.format(self.server_id, self.db_id)
+
+ class TestMockServer():
+ def __init__(self, host, port, id, username):
+ self.host = host
+ self.port = port
+ self.id = id
+ self.username = username
+
+ mock_obj = TestMockServer(self.class_params['host'],
+ self.class_params['port'],
+ self.server_id,
+ self.class_params['username']
+ )
+ mock_result = server_mock.query.filter_by.return_value
+ mock_result.first.return_value = mock_obj
+
+ batch_process_mock.set_env_variables = MagicMock(
+ return_value=True
+ )
+ batch_process_mock.start = MagicMock(
+ return_value=True
+ )
+ export_password_env_mock.return_value = True
+
+ server_response = server_utils.connect_server(self, self.server_id)
+ if server_response["info"] == "Server connected.":
+ db_owner = server_response['data']['user']['name']
+ self.data = database_utils.get_db_data(db_owner)
+ self.db_name = self.data['name']
+
+ # Create the backup job
+ response = self.tester.post(url,
+ data=json.dumps(self.params),
+ content_type='html/json')
+ self.assertEquals(response.status_code, 200)
+ response_data = json.loads(response.data.decode('utf-8'))
+ job_id = response_data['data']['job_id']
+
+ assert message_mock.called
+ assert batch_process_mock.called
+
+ if self.expected_cmd_opts:
+ for opt in self.expected_cmd_opts:
+ self.assertIn(opt,
+ batch_process_mock.call_args_list[0][1]['args'])
diff --git a/web/pgadmin/tools/maintenance/tests/test_maintenance_message.py b/web/pgadmin/tools/maintenance/tests/test_maintenance_message.py
new file mode 100644
index 0000000..4cb89ed
--- /dev/null
+++ b/web/pgadmin/tools/maintenance/tests/test_maintenance_message.py
@@ -0,0 +1,124 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+
+from flask import Response
+import simplejson as json
+
+from pgadmin.tools.maintenance import Message
+from pgadmin.utils.route import BaseTestGenerator
+
+
+class MaintenanceMessageTest(BaseTestGenerator):
+ """Test the Maintenance Message class"""
+ scenarios = [
+ ('When maintained the server',
+ dict(
+ class_params=dict(
+ sid=1,
+ data={
+ 'database': 'postgres',
+ 'op': 'VACUUM',
+ 'vacuum_analyze': False,
+ 'vacuum_freeze': False,
+ 'vacuum_full': False,
+ 'verbose': True
+ },
+ cmd="VACUUM VERBOSE;\n"
+ ),
+ extected_msg="Maintenance (Vacuum)",
+ expetced_details_cmd='VACUUM VERBOSE;'
+
+ )),
+ ('When maintained the server with FULL VERBOSE options',
+ dict(
+ class_params=dict(
+ sid=1,
+ data={
+ 'database': 'postgres',
+ 'op': 'VACUUM',
+ 'vacuum_analyze': False,
+ 'vacuum_freeze': False,
+ 'vacuum_full': True,
+ 'verbose': True
+ },
+ cmd="VACUUM FULL VERBOSE;\n"
+ ),
+ extected_msg="Maintenance (Vacuum)",
+ expetced_details_cmd='VACUUM FULL VERBOSE;'
+
+ )),
+ ('When maintained the server with ANALYZE',
+ dict(
+ class_params=dict(
+ sid=1,
+ data={
+ 'database': 'postgres',
+ 'op': 'ANALYZE',
+ 'vacuum_analyze': False,
+ 'vacuum_freeze': False,
+ 'vacuum_full': False,
+ 'verbose': True
+ },
+ cmd="ANALYZE VERBOSE;\n"
+ ),
+ extected_msg="Maintenance (Analyze)",
+ expetced_details_cmd='ANALYZE VERBOSE;'
+
+ )),
+ ('When maintained the server with REINDEX',
+ dict(
+ class_params=dict(
+ sid=1,
+ data={
+ 'database': 'postgres',
+ 'op': 'REINDEX',
+ 'vacuum_analyze': False,
+ 'vacuum_freeze': False,
+ 'vacuum_full': False,
+ 'verbose': False
+ },
+ cmd="REINDEX;\n"
+ ),
+ extected_msg="Maintenance (Reindex)",
+ expetced_details_cmd='REINDEX;'
+
+ )),
+ ('When maintained the server with CLUSTER',
+ dict(
+ class_params=dict(
+ sid=1,
+ data={
+ 'database': 'postgres',
+ 'op': 'CLUSTER',
+ 'vacuum_analyze': False,
+ 'vacuum_freeze': False,
+ 'vacuum_full': False,
+ 'verbose': True
+ },
+ cmd="CLUSTER VERBOSE;\n"
+ ),
+ extected_msg="Maintenance (Cluster)",
+ expetced_details_cmd='CLUSTER VERBOSE;'
+
+ )),
+ ]
+
+ def runTest(self):
+ maintenance_obj = Message(
+ self.class_params['sid'],
+ self.class_params['data'],
+ self.class_params['cmd']
+ )
+
+ # Check the expected message returned
+ assert maintenance_obj.message == self.extected_msg
+
+ # Check the command
+ obj_details = maintenance_obj.details(self.class_params['cmd'], None)
+ self.assertIn(self.expetced_details_cmd, obj_details)
diff --git a/web/pgadmin/tools/restore/__init__.py b/web/pgadmin/tools/restore/__init__.py
index 45d3816..58bc251 100644
--- a/web/pgadmin/tools/restore/__init__.py
+++ b/web/pgadmin/tools/restore/__init__.py
@@ -86,8 +86,7 @@ class RestoreMessage(IProcessDesc):
else:
self.cmd += cmdArg(arg)
- @property
- def message(self):
+ def get_server_details(self):
# Fetch the server details like hostname, port, roles etc
s = Server.query.filter_by(
id=self.sid, user_id=current_user.id
@@ -100,30 +99,25 @@ class RestoreMessage(IProcessDesc):
host = manager.local_bind_host if manager.use_ssh_tunnel else s.host
port = manager.local_bind_port if manager.use_ssh_tunnel else s.port
+ return s.name, host, port
+
+ @property
+ def message(self):
+ name, host, port = self.get_server_details()
+
return _("Restoring backup on the server '{0}'...").format(
- "{0} ({1}:{2})".format(s.name, host, port),
+ "{0} ({1}:{2})".format(name, host, port),
)
def details(self, cmd, args):
- # Fetch the server details like hostname, port, roles etc
- s = Server.query.filter_by(
- id=self.sid, user_id=current_user.id
- ).first()
-
- from pgadmin.utils.driver import get_driver
- driver = get_driver(PG_DEFAULT_DRIVER)
- manager = driver.connection_manager(self.sid)
-
- host = manager.local_bind_host if manager.use_ssh_tunnel else s.host
- port = manager.local_bind_port if manager.use_ssh_tunnel else s.port
-
+ name, host, port = self.get_server_details()
res = '<div class="h5">'
res += html.safe_str(
_(
"Restoring backup on the server '{0}'..."
).format(
- "{0} ({1}:{2})".format(s.name, host, port)
+ "{0} ({1}:{2})".format(name, host, port)
)
)
@@ -206,6 +200,7 @@ def create_restore_job(sid):
if _file is None:
return make_json_response(
+ status=410,
success=0,
errormsg=_("File could not be found.")
)
diff --git a/web/pgadmin/tools/restore/tests/__init__.py b/web/pgadmin/tools/restore/tests/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/web/pgadmin/tools/restore/tests/test_batch_process.py b/web/pgadmin/tools/restore/tests/test_batch_process.py
new file mode 100644
index 0000000..28d692a
--- /dev/null
+++ b/web/pgadmin/tools/restore/tests/test_batch_process.py
@@ -0,0 +1,154 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+import sys
+
+from pgadmin.misc.bgprocess.processes import BatchProcess, IProcessDesc
+from pgadmin.tools.restore import RestoreMessage
+from pgadmin.utils.route import BaseTestGenerator
+from pickle import dumps, loads
+
+if sys.version_info < (3, 3):
+ from mock import patch
+else:
+ from unittest.mock import patch
+
+
+class BatchProcessTest(BaseTestGenerator):
+ """Test the BatchProcess class"""
+ scenarios = [
+ ('When restore server',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_restore_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ username='postgres',
+ bfile='test_restore',
+ args=[
+ '--file',
+ "restore_file",
+ '--host',
+ "localhost",
+ '--port',
+ "5444",
+ '--username',
+ "postgres",
+ '--no-password',
+ '--database',
+ "postgres"
+ ],
+ cmd='restore_server'
+ )
+ ))
+ ]
+
+ @patch('pgadmin.tools.restore.RestoreMessage.get_server_details')
+ @patch('pgadmin.misc.bgprocess.processes.Popen')
+ @patch('pgadmin.misc.bgprocess.processes.current_app')
+ @patch('pgadmin.misc.bgprocess.processes.db')
+ @patch('pgadmin.tools.restore.current_user')
+ @patch('pgadmin.misc.bgprocess.processes.current_user')
+ def runTest(self, current_user_mock, current_user, db_mock,
+ current_app_mock, popen_mock, get_server_details_mock):
+ current_user.id = 1
+ current_user_mock.id = 1
+ current_app_mock.PGADMIN_RUNTIME = False
+
+ def db_session_add_mock(j):
+ cmd_obj = loads(j.desc)
+ assert isinstance(cmd_obj, IProcessDesc)
+ print(cmd_obj)
+ self.assertEquals(cmd_obj.bfile, self.class_params['bfile'])
+ self.assertEquals(cmd_obj.cmd,
+ ' --file "restore_file" '
+ '--host "{0}" '
+ '--port "{1}" '
+ '--username "{2}" '
+ '--no-password '
+ '--database "{3}"'.format(
+ self.class_params['host'],
+ self.class_params['port'],
+ self.class_params['username'],
+ self.class_params['database']
+ ))
+
+ get_server_details_mock.return_value = \
+ self.class_params['name'],\
+ self.class_params['host'],\
+ self.class_params['port']
+
+ db_mock.session.add.side_effect = db_session_add_mock
+
+ restore_obj = RestoreMessage(
+ self.class_params['sid'],
+ self.class_params['bfile'],
+ *self.class_params['args']
+ )
+
+ p = BatchProcess(
+ desc=restore_obj,
+ cmd=self.class_params['cmd'],
+ args=self.class_params['args']
+ )
+
+ # Check that _create_process has been called
+ assert db_mock.session.add.called
+
+ # Check start method
+ self._check_start(popen_mock, p)
+
+ # Check list method
+ self._check_list(p, restore_obj)
+
+ def _check_start(self, popen_mock, p):
+ cmd_test = self.class_params['cmd']
+
+ class popenMockSideEffect():
+ def __init__(self, cmd, **kwargs):
+ assert cmd_test in cmd
+ assert 'env' in kwargs
+
+ def poll(self):
+ pass
+
+ popen_mock.side_effect = popenMockSideEffect
+ p.start()
+
+ assert popen_mock.called
+
+ @patch('pgadmin.misc.bgprocess.processes.Process')
+ @patch('pgadmin.misc.bgprocess.processes.BatchProcess.'
+ 'update_process_info')
+ def _check_list(self, p, restore_obj, update_process_info_mock,
+ process_mock):
+ class TestMockProcess():
+ def __init__(self, desc, args, cmd):
+ self.pid = 1
+ self.exit_code = 1
+ self.start_time = '2018-04-17 06:18:56.315445 +0000'
+ self.end_time = None
+ self.desc = dumps(desc)
+ self.arguments = " ".join(args)
+ self.command = cmd
+ self.acknowledge = None
+
+ process_mock.query.filter_by.return_value = [
+ TestMockProcess(restore_obj,
+ self.class_params['args'],
+ self.class_params['cmd'])
+ ]
+
+ update_process_info_mock.return_value = [True, True]
+
+ ret_value = p.list()
+ self.assertEqual(1, len(ret_value))
+ assert 'details' in ret_value[0]
+ assert 'desc' in ret_value[0]
diff --git a/web/pgadmin/tools/restore/tests/test_create_restore_job.py b/web/pgadmin/tools/restore/tests/test_create_restore_job.py
new file mode 100644
index 0000000..ff81bf6
--- /dev/null
+++ b/web/pgadmin/tools/restore/tests/test_create_restore_job.py
@@ -0,0 +1,199 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+import sys
+import time
+import random
+import os
+
+import simplejson as json
+
+from pgadmin.utils.route import BaseTestGenerator
+from regression import parent_node_dict
+from regression.python_test_utils import test_utils as utils
+from pgadmin.utils import server_utils as server_utils
+import pgadmin.tools.backup.tests.test_backup_utils as backup_utils
+
+
+if sys.version_info < (3, 3):
+ from mock import patch, MagicMock
+else:
+ from unittest.mock import patch, MagicMock
+
+
+class RestoreJobTest(BaseTestGenerator):
+ """Backup api test cases"""
+ scenarios = [
+ ('When restore the object with the default options',
+ dict(
+ params=dict(
+ file='test_restore_file',
+ format='custom',
+ custom=False,
+ verbose=True,
+ blobs=True,
+ schemas=[],
+ tables=[],
+ database='test_restore_database'
+ ),
+ url='/restore/job/{0}',
+ expected_cmd_opts=['--verbose'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None],
+ backup_options=dict(
+ params=dict(
+ file='test_restore_file',
+ format='custom',
+ verbose=True,
+ blobs=True,
+ schemas=[],
+ tables=[],
+ database='test_restore_database'
+ ),
+ url='/backup/job/{0}/object',
+ expected_params=dict(
+ expected_cmd_opts=['--verbose', '--format=c', '--blobs'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )
+
+ )
+ ))
+ ]
+
+ def setUp(self):
+ if self.server['default_binary_paths'] is None:
+ self.skipTest(
+ "default_binary_paths is not set for the server {0}".format(
+ self.server['name']
+ )
+ )
+
+ def create_backup(self):
+ url = self.backup_options['url'].format(self.server_id)
+ job_id = backup_utils.create_backup_job(self.tester, url,
+ self.backup_options['params'])
+ self.backup_file = backup_utils.run_backup_job(self.tester, job_id,
+ self.backup_options['expected_params'],
+ self.assertIn, self.assertNotIn)
+
+ def runTest(self):
+ self.db_name = ''
+ self.server_id = parent_node_dict["server"][-1]["server_id"]
+ server_response = server_utils.connect_server(self, self.server_id)
+ db_id = utils.create_database(self.server, self.params['database'])
+
+ self.create_backup()
+ url = self.url.format(self.server_id)
+
+ # Create the restore job
+ response = self.tester.post(url,
+ data=json.dumps(self.params),
+ content_type='html/json')
+ self.assertEquals(response.status_code, 200)
+ response_data = json.loads(response.data.decode('utf-8'))
+ job_id = response_data['data']['job_id']
+
+ cnt = 0
+ while 1:
+ if cnt > 1:
+ break
+ # Check the process list
+ response1 = self.tester.get('/misc/bgprocess/?_='.format(
+ random.randint(1, 9999999)))
+ self.assertEquals(response1.status_code, 200)
+ process_list = json.loads(response1.data.decode('utf-8'))
+
+ if len(process_list) > 0 and 'execution_time' in process_list[0]:
+ break
+ time.sleep(0.5)
+ cnt += 1
+
+ assert 'execution_time' in process_list[0]
+ assert 'stime' in process_list[0]
+ assert 'exit_code' in process_list[0]
+ assert process_list[0]['exit_code'] in self.expected_exit_code
+
+ if self.expected_cmd_opts:
+ for opt in self.expected_cmd_opts:
+ self.assertIn(opt, process_list[0]['details'])
+ if self.not_expected_cmd_opts:
+ for opt in self.not_expected_cmd_opts:
+ self.assertNotIn(opt, process_list[0]['details'])
+
+ # Check the process details
+ p_details = self.tester.get('/misc/bgprocess/{0}?_='.format(
+ job_id, random.randint(1, 9999999))
+ )
+ self.assertEquals(p_details.status_code, 200)
+ p_details_data = json.loads(p_details.data.decode('utf-8'))
+
+ p_details = self.tester.get('/misc/bgprocess/{0}/{1}/{2}/?_='.format(
+ job_id, 0, 0, random.randint(1, 9999999))
+ )
+ self.assertEquals(p_details.status_code, 200)
+ p_details_data = json.loads(p_details.data.decode('utf-8'))
+
+ # Retrieve the restore job process logs
+ cnt = 0
+ while 1:
+ out, err, status = RestoreJobTest.get_params(p_details_data)
+ if status or cnt >= 10:
+ break
+
+ p_details = self.tester.get(
+ '/misc/bgprocess/{0}/{1}/{2}/?_={3}'.format(
+ job_id, out, err, random.randint(1, 9999999))
+ )
+ self.assertEquals(p_details.status_code, 200)
+ p_details_data = json.loads(p_details.data.decode('utf-8'))
+
+ cnt += 1
+ time.sleep(1)
+
+ # Check the job is complete.
+ restore_ack = self.tester.put('/misc/bgprocess/{0}'.format(job_id))
+ self.assertEquals(restore_ack.status_code, 200)
+ restore_ack_res = json.loads(restore_ack.data.decode('utf-8'))
+
+ self.assertEquals(restore_ack_res['success'], 1)
+
+ if self.backup_file is not None:
+ if os.path.isfile(self.backup_file):
+ os.remove(self.backup_file)
+
+ @staticmethod
+ def get_params(data):
+ out = 0
+ out_done = False
+ err = 0
+ err_done = False
+ if 'out' in data:
+ out = data['out'] and data['out']['pos']
+
+ if 'done' in data['out']:
+ out_done = data['out']['done']
+
+ if 'err' in data:
+ err = data['err'] and data['err']['pos']
+
+ if 'done' in data['err']:
+ err_done = data['err']['done']
+
+ return out, err, (out_done and err_done)
+
+ def tearDown(self):
+ connection = utils.get_db_connection(
+ self.server['db'],
+ self.server['username'],
+ self.server['db_password'],
+ self.server['host'],
+ self.server['port'],
+ self.server['sslmode']
+ )
+ utils.drop_database(connection, self.params['database'])
diff --git a/web/pgadmin/tools/restore/tests/test_restore_create_job_unit_test.py b/web/pgadmin/tools/restore/tests/test_restore_create_job_unit_test.py
new file mode 100644
index 0000000..2829cd8
--- /dev/null
+++ b/web/pgadmin/tools/restore/tests/test_restore_create_job_unit_test.py
@@ -0,0 +1,318 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+
+import sys
+import simplejson as json
+
+from pgadmin.tools.restore import RestoreMessage
+from pgadmin.utils.route import BaseTestGenerator
+from regression import parent_node_dict
+from regression.python_test_utils import test_utils as utils
+from pgadmin.utils import server_utils as server_utils
+from pgadmin.browser.server_groups.servers.databases.tests import utils as \
+ database_utils
+
+if sys.version_info < (3, 3):
+ from mock import patch, MagicMock
+else:
+ from unittest.mock import patch, MagicMock
+
+
+class RestoreCreateJobTest(BaseTestGenerator):
+ """Test the RestoreCreateJob class"""
+ scenarios = [
+ ('When restore object with default options',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_restore_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_restore',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_restore_file',
+ format='custom',
+ custom=False,
+ verbose=True,
+ blobs=True,
+ schemas=[],
+ tables=[],
+ database='postgres'
+ ),
+ url='/restore/job/{0}',
+ expected_cmd_opts=['--verbose'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When restore object with the sections options',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_restore_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_restore',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_restore_file',
+ format='custom',
+ no_of_jobs='2',
+ custom=False,
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ data=True,
+ pre_data=True,
+ post_data=True,
+ only_data=True,
+ only_schema=True
+ ),
+ url='/restore/job/{0}',
+ # Please include sections data here, right now this is a bug
+ expected_cmd_opts=['--verbose', '--jobs', '2'],
+ not_expected_cmd_opts=[],
+ # Below options should be enabled once we fix the issue #3368
+ # not_expected_cmd_opts=['--data-only', '--schema-only'],
+ expected_exit_code=[0, None],
+ )),
+ ('When restore the object with Type of objects',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_restore_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_restore',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_restore_file',
+ format='custom',
+ no_of_jobs='2',
+ custom=False,
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ only_data=True,
+ only_schema=True,
+ dns_owner=True
+ ),
+ url='/restore/job/{0}',
+ expected_cmd_opts=['--verbose', '--data-only'],
+ not_expected_cmd_opts=[],
+ # Below options should be enabled once we fix the issue #3368
+ # not_expected_cmd_opts=['--schema-only', '--no-owner'],
+ expected_exit_code=[0, None],
+ )),
+ ('When restore object with option - Do not save',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_restore_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_restore',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_restore_file',
+ format='custom',
+ verbose=True,
+ custom=False,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ dns_owner=True,
+ dns_privilege=True,
+ dns_tablespace=True,
+ only_data=False
+ ),
+ url='/restore/job/{0}',
+ # Add '--no-privileges' to the expected_cmd once #3363 fixed
+ expected_cmd_opts=['--no-owner',
+ '--no-tablespaces'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When restore object with option - Queries',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_restore_file',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_restore',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='custom',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ clean=True,
+ include_create_database=True,
+ single_transaction=True,
+ ),
+ url='/restore/job/{0}',
+ expected_cmd_opts=['--create', '--clean',
+ '--single-transaction'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When restore object with option - Disbale',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_restore_file',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_restore',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='custom',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ disable_trigger=True,
+ no_data_fail_table=True,
+ only_schema=False
+ ),
+ url='/restore/job/{0}',
+ # Add '--no-data-for-failed-tables' into
+ # expected_cmd_opts once #3363 fixed
+ expected_cmd_opts=['--disable-triggers'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ('When restore object with option - Miscellaneous',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_restore_file',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_restore',
+ username='postgres'
+ ),
+ params=dict(
+ file='test_backup_file',
+ format='custom',
+ verbose=True,
+ schemas=[],
+ tables=[],
+ database='postgres',
+ use_set_session_auth=True,
+ exit_on_error=True,
+ ),
+ url='/restore/job/{0}',
+ # Add '--use_set_session_auth' into
+ # expected_cmd_opts once #3363 fixed
+ expected_cmd_opts=['--exit-on-error'],
+ not_expected_cmd_opts=[],
+ expected_exit_code=[0, None]
+ )),
+ ]
+
+ def setUp(self):
+ if self.server['default_binary_paths'] is None:
+ self.skipTest(
+ "default_binary_paths is not set for the server {0}".format(
+ self.server['name']
+ )
+ )
+
+ @patch('pgadmin.tools.restore.Server')
+ @patch('pgadmin.tools.restore.current_user')
+ @patch('pgadmin.tools.restore.RestoreMessage')
+ @patch('pgadmin.tools.restore.filename_with_file_manager_path')
+ @patch('pgadmin.tools.restore.BatchProcess')
+ @patch('pgadmin.utils.driver.psycopg2.server_manager.ServerManager.'
+ 'export_password_env')
+ def runTest(self, export_password_env_mock, batch_process_mock,
+ filename_mock, restore_message_mock,
+ current_user_mock, server_mock):
+ class TestMockServer():
+ def __init__(self, name, host, port, id, username):
+ self.name = name
+ self.host = host
+ self.port = port
+ self.id = id
+ self.username = username
+
+ self.db_name = ''
+ self.server_id = parent_node_dict["server"][-1]["server_id"]
+
+ mock_obj = TestMockServer(self.class_params['name'],
+ self.class_params['host'],
+ self.class_params['port'],
+ self.server_id,
+ self.class_params['username']
+ )
+ mock_result = server_mock.query.filter_by.return_value
+ mock_result.first.return_value = mock_obj
+
+ filename_mock.return_value = self.params['file']
+
+ batch_process_mock.set_env_variables = MagicMock(
+ return_value=True
+ )
+ batch_process_mock.start = MagicMock(
+ return_value=True
+ )
+
+ export_password_env_mock.return_value = True
+
+ server_response = server_utils.connect_server(self, self.server_id)
+ if server_response["info"] == "Server connected.":
+ db_owner = server_response['data']['user']['name']
+ self.data = database_utils.get_db_data(db_owner)
+ self.db_name = self.data['name']
+
+ url = self.url.format(self.server_id)
+
+ # Create the restore job
+ response = self.tester.post(url,
+ data=json.dumps(self.params),
+ content_type='html/json')
+ self.assertEquals(response.status_code, 200)
+ response_data = json.loads(response.data.decode('utf-8'))
+ job_id = response_data['data']['job_id']
+
+ assert restore_message_mock.called
+ assert batch_process_mock.called
+
+ if self.expected_cmd_opts:
+ for opt in self.expected_cmd_opts:
+ self.assertIn(
+ opt,
+ batch_process_mock.call_args_list[0][1]['args']
+ )
+ if self.not_expected_cmd_opts:
+ for opt in self.not_expected_cmd_opts:
+ self.assertNotIn(
+ opt,
+ batch_process_mock.call_args_list[0][1]['args']
+ )
diff --git a/web/pgadmin/tools/restore/tests/test_restore_message.py b/web/pgadmin/tools/restore/tests/test_restore_message.py
new file mode 100644
index 0000000..bb45286
--- /dev/null
+++ b/web/pgadmin/tools/restore/tests/test_restore_message.py
@@ -0,0 +1,76 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2018, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+import sys
+
+from pgadmin.tools.restore import RestoreMessage
+from pgadmin.utils.route import BaseTestGenerator
+
+if sys.version_info < (3, 3):
+ from mock import patch
+else:
+ from unittest.mock import patch
+
+
+class RestoreMessageTest(BaseTestGenerator):
+ """Test the RestoreMessage class"""
+ scenarios = [
+ ('When restore object',
+ dict(
+ class_params=dict(
+ sid=1,
+ name='test_restore_server',
+ port=5444,
+ host='localhost',
+ database='postgres',
+ bfile='test_restore',
+ args=[
+ '--file',
+ 'restore_file',
+ '--host',
+ 'localhost',
+ '--port',
+ '5444',
+ '--username',
+ 'postgres',
+ '--no-password',
+ '--database',
+ 'postgres'
+ ],
+ cmd="/test_path/pg_restore"
+ ),
+ extected_msg="Restoring backup on the server "
+ "'test_restore_server (localhost:5444)'...",
+ expetced_details_cmd='/test_path/pg_restore --file '
+ '"restore_file" --host "localhost"'
+ ' --port "5444" --username "postgres" '
+ '--no-password --database "postgres"'
+
+ ))
+ ]
+
+ @patch('pgadmin.tools.restore.RestoreMessage.get_server_details')
+ def runTest(self, get_server_details_mock):
+ get_server_details_mock.return_value = \
+ self.class_params['name'],\
+ self.class_params['host'],\
+ self.class_params['port']
+
+ restore_obj = RestoreMessage(
+ self.class_params['sid'],
+ self.class_params['bfile'],
+ *self.class_params['args']
+ )
+
+ # Check the expected message returned
+ assert restore_obj.message == self.extected_msg
+
+ # Check the command
+ obj_details = restore_obj.details(self.class_params['cmd'],
+ self.class_params['args'])
+ self.assertIn(self.expetced_details_cmd, obj_details)
diff --git a/web/regression/python_test_utils/test_utils.py b/web/regression/python_test_utils/test_utils.py
index 3e517b6..6f57c67 100644
--- a/web/regression/python_test_utils/test_utils.py
+++ b/web/regression/python_test_utils/test_utils.py
@@ -21,6 +21,8 @@ import config
import regression
from regression import test_setup
+from pgadmin.utils.preferences import Preferences
+
SERVER_GROUP = test_setup.config_data['server_group']
file_name = os.path.realpath(__file__)
@@ -86,7 +88,8 @@ def get_config_data():
"db_password": srv['db_password'],
"role": "",
"sslmode": srv['sslmode'],
- "tablespace_path": srv.get('tablespace_path', None)
+ "tablespace_path": srv.get('tablespace_path', None),
+ "default_binary_paths": srv.get('default_binary_paths', None)
}
if 'db_type' in srv:
data['db_type'] = srv['db_type']
@@ -445,6 +448,13 @@ def delete_server_with_api(tester, sid):
url = '/browser/server/obj/' + str(SERVER_GROUP) + "/"
# Call API to delete the server
response = tester.delete(url + str(sid))
+
+ cnt = 0
+ for s in regression.parent_node_dict["server"]:
+ if s['server_id'] == int(sid):
+ del regression.parent_node_dict["server"][cnt]
+ cnt += 1
+
except Exception:
traceback.print_exc(file=sys.stderr)
@@ -596,6 +606,64 @@ def get_db_server(sid):
return connection
+def set_preference(default_binary_path):
+ conn = sqlite3.connect(config.TEST_SQLITE_PATH)
+ cur = conn.cursor()
+
+ perf = Preferences.module('paths')
+ pg_path_pref = perf.preference('pg_bin_dir')
+
+ user_pref = cur.execute(
+ 'SELECT pid, uid FROM user_preferences where pid=%s' % pg_path_pref.pid
+ )
+ user_pref = user_pref.fetchone()
+
+ if user_pref:
+ cur.execute('UPDATE user_preferences SET value = ? WHERE pid = ?',
+ (default_binary_path['pg'], pg_path_pref.pid))
+ else:
+ pg_pref_details = (pg_path_pref.pid, 1,
+ default_binary_path['pg'])
+ cur.execute('INSERT INTO user_preferences(pid, uid, value)'
+ ' VALUES (?,?,?)', pg_pref_details)
+
+ ppas_path_pref = perf.preference('ppas_bin_dir')
+
+ user_pref = cur.execute(
+ 'SELECT pid, uid FROM user_preferences where pid=%s' %
+ ppas_path_pref.pid
+ )
+ user_pref = user_pref.fetchone()
+
+ if user_pref:
+ cur.execute('UPDATE user_preferences SET value = ? WHERE pid = ? ',
+ (default_binary_path['ppas'], ppas_path_pref.pid))
+ else:
+ ppas_pref_details = (ppas_path_pref.pid, 1,
+ default_binary_path['ppas'])
+ cur.execute('INSERT INTO user_preferences(pid, uid, value)'
+ ' VALUES (?,?,?)', ppas_pref_details)
+
+ gpdb_path_pref = perf.preference('gpdb_bin_dir')
+
+ user_pref = cur.execute(
+ 'SELECT pid, uid FROM user_preferences where pid=%s' %
+ gpdb_path_pref.pid
+ )
+ user_pref = user_pref.fetchone()
+
+ if user_pref:
+ cur.execute('UPDATE user_preferences SET value = ? WHERE pid = ? ',
+ (default_binary_path['gpdb'], gpdb_path_pref.pid))
+ else:
+ gpdb_pref_details = (gpdb_path_pref.pid, 1,
+ default_binary_path['gpdb'])
+ cur.execute('INSERT INTO user_preferences(pid, uid, value)'
+ ' VALUES (?,?,?)', gpdb_pref_details)
+
+ conn.commit()
+
+
def remove_db_file():
"""This function use to remove SQLite DB file"""
if os.path.isfile(config.TEST_SQLITE_PATH):
diff --git a/web/regression/runtests.py b/web/regression/runtests.py
index d786692..26b25c7 100644
--- a/web/regression/runtests.py
+++ b/web/regression/runtests.py
@@ -114,6 +114,9 @@ test_client = app.test_client()
driver = None
app_starter = None
handle_cleanup = None
+app.PGADMIN_RUNTIME = True
+if config.SERVER_MODE is True:
+ app.PGADMIN_RUNTIME = False
setattr(unit_test.result.TestResult, "passed", [])
@@ -234,7 +237,6 @@ def get_test_modules(arguments):
# Sort module list so that test suite executes the test cases sequentially
module_list = TestsGeneratorRegistry.registry.items()
module_list = sorted(module_list, key=lambda module_tuple: module_tuple[0])
-
return module_list
@@ -393,6 +395,9 @@ if __name__ == '__main__':
# Create test server
server_information = test_utils.create_parent_server_node(server)
+ if server['default_binary_paths'] is not None:
+ test_utils.set_preference(server['default_binary_paths'])
+
suite = get_suite(test_module_list,
server,
test_client,
diff --git a/web/regression/test_config.json.in b/web/regression/test_config.json.in
index ebc1466..15b133a 100644
--- a/web/regression/test_config.json.in
+++ b/web/regression/test_config.json.in
@@ -23,7 +23,12 @@
"maintenance_db": "postgres",
"sslmode": "prefer",
"tablespace_path": "",
- "enabled": true
+ "enabled": true,
+ "default_binary_paths": {
+ "pg": "/opt/PostgreSQL/9.4/bin/",
+ "ppas": "/opt/edb/as10/bin/",
+ "gpdb": ""
+ }
}
],
"server_update_data": [
view thread (29+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: [pgadmin4][Patch]: Test cases for the backup module
In-Reply-To: <CAFOhELc6zT5pLHJTWd68q=jn3rhBjROUAKi5DneURdd7=B0VqA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox