pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Adrian Klaver <adrian.klaver@aklaver.com>
To: David Ventimiglia <davidaventimiglia@hasura.io>
To: pgsql-general@lists.postgresql.org
Subject: Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?
Date: Fri, 12 Jan 2024 11:48:42 -0800
Message-ID: <daf7934f-18f5-48fe-a7e1-924c8697dc6c@aklaver.com> (raw)
In-Reply-To: <CADE7j6jz3qEinUfeALyBVdoyMWxE3okxUjKH0Dw8A+JjEzZJ_Q@mail.gmail.com>
References: <CADE7j6jz3qEinUfeALyBVdoyMWxE3okxUjKH0Dw8A+JjEzZJ_Q@mail.gmail.com>

On 1/12/24 11:34, David Ventimiglia wrote:
> Hello! How do I redirect logical decoding output from the PostgreSQL CLI 
> tool |pg_recvlogical| either to a file or to another command via a pipe? 
> I ask because when I try the obvious, no output is recorded or sent:
> 
> |pg_recvlogical -d postgres --slot test --start -f - >> sample.jsonl |
> 
> Lest there be any confusion, I already created the slot in an earlier 
> step. Moreover, I can verify that if I omit the output redirection |>> 
> sample| then it does work, insofar as it emits the expected change 
> events when I perform DML in another terminal window. When I include the 
> redirection (or alternatively, set up a pipeline), then nothing happens.

You left out the important part of the SO question:

pg_recvlogical -d postgres --slot test --create-slot -P wal2json

I can get:

pg_recvlogical -d postgres --slot test --start -f - >> sample.jsonl

to work when I do:

pg_recvlogical -d postgres --slot test --create-slot

wal2json seems to be the issue.

> 
> Note that I am aware of the option to pass a filename to the -f switch 
> to write to a file.  That works, but it's not what I'm after because it 
> doesn't help update my mental model of how this is supposed to work.  
> Based on my current (flawed) mental model built up from command line 
> experience with other tools, this /should/ work.  I should be able to 
> send the output to stdout and then redirect it to a file.  It surprises 
> me that I cannot.
> 
> Anyway, thanks!
> 
> Best,
> 
> David
> 

-- 
Adrian Klaver
adrian.klaver@aklaver.com






view thread (17+ messages)  latest in thread

Message-ID: <daf7934f-18f5-48fe-a7e1-924c8697dc6c@aklaver.com>
Permalink:  ../daf7934f-18f5-48fe-a7e1-924c8697dc6c@aklaver.com/
Also on:    postgresql.org/message-id/daf7934f-18f5-48fe-a7e1-924c8697dc6c@aklaver.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: adrian.klaver@aklaver.com, davidaventimiglia@hasura.io
  Subject: Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?
  In-Reply-To: <daf7934f-18f5-48fe-a7e1-924c8697dc6c@aklaver.com>
  References: <CADE7j6jz3qEinUfeALyBVdoyMWxE3okxUjKH0Dw8A+JjEzZJ_Q@mail.gmail.com> <daf7934f-18f5-48fe-a7e1-924c8697dc6c@aklaver.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox