pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Add decoding of sequences to test_decoding
3+ messages / 3 participants
[nested] [flat]

Commits from this thread:
  2022-02-12 b779d7d8fd  Fix skip-empty-xacts with sequences in test_decoding
* pgsql: Add decoding of sequences to test_decoding
@ 2022-02-11 23:52 Tomas Vondra <tomas.vondra@postgresql.org>
  2022-02-12 22:04 ` Re: pgsql: Add decoding of sequences to test_decoding Andres Freund <andres@anarazel.de>
  0 siblings, 1 reply; 3+ messages in thread

From: Tomas Vondra @ 2022-02-11 23:52 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Add decoding of sequences to test_decoding

Commit 0da92dc530 improved the logical decoding infrastructure to handle
sequences, and did various changes to related parts (WAL logging etc.).
But it did not include any implementation of the new callbacks added to
OutputPluginCallbacks.

This extends test_decoding with two callbacks to decode sequences. The
decoding of sequences may be disabled using 'include-sequences', a new
option of the output plugin.

Author: Tomas Vondra, Cary Huang
Reviewed-by: Peter Eisentraut, Hannu Krosing, Andres Freund
Discussion: https://postgr.es/m/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com
Discussion: https://postgr.es/m/1710ed7e13b.cd7177461430746.3372264562543607781@highgo.ca

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/80901b32913ffa59bf157a4d88284b2b3a7511d9

Modified Files
--------------
contrib/test_decoding/Makefile                     |   3 +-
contrib/test_decoding/expected/ddl.out             |  12 +-
.../test_decoding/expected/decoding_in_xact.out    |   2 +-
.../test_decoding/expected/decoding_into_rel.out   |  10 +-
contrib/test_decoding/expected/mxact.out           |   8 +-
contrib/test_decoding/expected/ondisk_startup.out  |   4 +-
contrib/test_decoding/expected/replorigin.out      |   4 +-
contrib/test_decoding/expected/rewrite.out         |   4 +-
contrib/test_decoding/expected/sequence.out        | 327 +++++++++++++++++++++
contrib/test_decoding/expected/slot.out            |   2 +-
contrib/test_decoding/expected/toast.out           |  10 +-
contrib/test_decoding/expected/truncate.out        |   2 +-
contrib/test_decoding/specs/mxact.spec             |   2 +-
contrib/test_decoding/specs/ondisk_startup.spec    |   2 +-
contrib/test_decoding/sql/ddl.sql                  |  12 +-
contrib/test_decoding/sql/decoding_in_xact.sql     |   2 +-
contrib/test_decoding/sql/decoding_into_rel.sql    |  10 +-
contrib/test_decoding/sql/replorigin.sql           |   4 +-
contrib/test_decoding/sql/rewrite.sql              |   4 +-
contrib/test_decoding/sql/sequence.sql             | 119 ++++++++
contrib/test_decoding/sql/slot.sql                 |   2 +-
contrib/test_decoding/sql/toast.sql                |  10 +-
contrib/test_decoding/sql/truncate.sql             |   2 +-
contrib/test_decoding/test_decoding.c              |  65 ++++
24 files changed, 567 insertions(+), 55 deletions(-)



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: pgsql: Add decoding of sequences to test_decoding
  2022-02-11 23:52 pgsql: Add decoding of sequences to test_decoding Tomas Vondra <tomas.vondra@postgresql.org>
@ 2022-02-12 22:04 ` Andres Freund <andres@anarazel.de>
  2022-02-12 23:02   ` Re: pgsql: Add decoding of sequences to test_decoding Tomas Vondra <tomas.vondra@enterprisedb.com>
  0 siblings, 1 reply; 3+ messages in thread

From: Andres Freund @ 2022-02-12 22:04 UTC (permalink / raw)
  To: Tomas Vondra <tomas.vondra@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org

Hi,

On 2022-02-11 23:52:20 +0000, Tomas Vondra wrote:
> Add decoding of sequences to test_decoding
>
> Commit 0da92dc530 improved the logical decoding infrastructure to handle
> sequences, and did various changes to related parts (WAL logging etc.).
> But it did not include any implementation of the new callbacks added to
> OutputPluginCallbacks.
>
> This extends test_decoding with two callbacks to decode sequences. The
> decoding of sequences may be disabled using 'include-sequences', a new
> option of the output plugin.
>
> Author: Tomas Vondra, Cary Huang
> Reviewed-by: Peter Eisentraut, Hannu Krosing, Andres Freund
> Discussion: https://postgr.es/m/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com
> Discussion: https://postgr.es/m/1710ed7e13b.cd7177461430746.3372264562543607781@highgo.ca

The tests fail on slow machines (valgrind in this case), with some spurious
output differences:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2022-02-12%2010%3A24%3A22

inline patch 1: /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out (+3/-1 lines) download inline-1.patchdiff -U3 /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out --- /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out 2022-02-12 00:40:25.431913374 +0000 +++ /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out 2022-02-12 11:07:19.177487256 +0000 @@ -263,12 +263,14 @@ data ----------------------------------------------------------------------------------------- BEGIN + COMMIT + BEGIN sequence public.test_table_a_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0 sequence public.test_table_a_seq: transactional:1 last_value: 33 log_cnt: 0 is_called:1 table public.test_table: INSERT: a[integer]:1 b[integer]:100 table public.test_table: INSERT: a[integer]:2 b[integer]:200 COMMIT -(6 rows) +(8 rows)
I don't think it's a good idea to use skip-empty-xacts = 0 in tests, there always can be an autovacuum or such. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 3+ messages in thread

* Re: pgsql: Add decoding of sequences to test_decoding
  2022-02-11 23:52 pgsql: Add decoding of sequences to test_decoding Tomas Vondra <tomas.vondra@postgresql.org>
  2022-02-12 22:04 ` Re: pgsql: Add decoding of sequences to test_decoding Andres Freund <andres@anarazel.de>
@ 2022-02-12 23:02   ` Tomas Vondra <tomas.vondra@enterprisedb.com>
  0 siblings, 0 replies; 3+ messages in thread

From: Tomas Vondra @ 2022-02-12 23:02 UTC (permalink / raw)
  To: Andres Freund <andres@anarazel.de>; Tomas Vondra <tomas.vondra@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org

On 2/12/22 23:04, Andres Freund wrote:
> Hi,
> 
> On 2022-02-11 23:52:20 +0000, Tomas Vondra wrote:
>> Add decoding of sequences to test_decoding
>>
>> Commit 0da92dc530 improved the logical decoding infrastructure to handle
>> sequences, and did various changes to related parts (WAL logging etc.).
>> But it did not include any implementation of the new callbacks added to
>> OutputPluginCallbacks.
>>
>> This extends test_decoding with two callbacks to decode sequences. The
>> decoding of sequences may be disabled using 'include-sequences', a new
>> option of the output plugin.
>>
>> Author: Tomas Vondra, Cary Huang
>> Reviewed-by: Peter Eisentraut, Hannu Krosing, Andres Freund
>> Discussion: https://postgr.es/m/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com
>> Discussion: https://postgr.es/m/1710ed7e13b.cd7177461430746.3372264562543607781@highgo.ca
> 
> The tests fail on slow machines (valgrind in this case), with some spurious
> output differences:
> 
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2022-02-12%2010%3A24%3A22
> 
> diff -U3 /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out
> --- /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out	2022-02-12 00:40:25.431913374 +0000
> +++ /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out	2022-02-12 11:07:19.177487256 +0000
> @@ -263,12 +263,14 @@
>                                            data
>  -----------------------------------------------------------------------------------------
>   BEGIN
> + COMMIT
> + BEGIN
>   sequence public.test_table_a_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
>   sequence public.test_table_a_seq: transactional:1 last_value: 33 log_cnt: 0 is_called:1
>   table public.test_table: INSERT: a[integer]:1 b[integer]:100
>   table public.test_table: INSERT: a[integer]:2 b[integer]:200
>   COMMIT
> -(6 rows)
> +(8 rows)
> 
> I don't think it's a good idea to use skip-empty-xacts = 0 in tests, there
> always can be an autovacuum or such.
> 

Thanks, I pushed a fix. Turns out the bug was a bit worse, because the
sequence callbacks did not actually deal with empty xacts correctly, so
I fixed that too.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





^ permalink  raw  reply  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2022-02-12 23:02 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-02-11 23:52 pgsql: Add decoding of sequences to test_decoding Tomas Vondra <tomas.vondra@postgresql.org>
2022-02-12 22:04 ` Andres Freund <andres@anarazel.de>
2022-02-12 23:02   ` Tomas Vondra <tomas.vondra@enterprisedb.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox