Date: Thu, 12 Sep 2024 21:30:44 +0200
From: Christoph Moench-Tegeder <cmt@burggraben.net>
To: Ron Johnson <ronljohnsonjr@gmail.com>
Cc: pgsql-general <pgsql-general@postgresql.org>
Subject: Re: Better way to process records in bash?
Message-ID: <ZuNBZM2BB62qvO29@elch.exwg.net>
References: <CANzqJaCaLwc6rOK7O8PkCgVDi_8fg6duzdUrMy0rtn_tQL144g@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <CANzqJaCaLwc6rOK7O8PkCgVDi_8fg6duzdUrMy0rtn_tQL144g@mail.gmail.com>
User-Agent: Mutt/2.2.13 (2024-03-09)
Archived-At: <https://www.postgresql.org/message-id/ZuNBZM2BB62qvO29%40elch.exwg.net>
Precedence: bulk

## Ron Johnson (ronljohnsonjr@gmail.com):

> I need to process table records in a bash script.  Currently, I read them
> using a while loop and redirection.  The table isn't that big (30ish
> thousand rows), and performance is adequate, but am always looking for
> "better".

Use python, or any other sane language, and don't do bash in python.
Anything non-trivial is unreasonably hard to get really right when
using shell.

Take this example (even when allowing some fuzz with the variables):
> Here's the current code:
> declare f1 f3 f8
> while IFS='|' read f1 f3 f8; do
>     something f8 f3 f1
> done < <(psql -XAt -c "select f1, f3, f8 from some.table_name;")

What happens when f3
 a) is NULL?
 b) contains a '|' character?
 c) contains spaces? (this one depends on how exactly you're handling
    the shell variables).
This might be fine if your data is only NOT NULL numbers, but that's
already quite a limitation on your data.

Regards,
Christoph

-- 
Spare Space