MIME-Version: 1.0
References: <CANzqJaCaLwc6rOK7O8PkCgVDi_8fg6duzdUrMy0rtn_tQL144g@mail.gmail.com>
 <ZuNBZM2BB62qvO29@elch.exwg.net>
In-Reply-To: <ZuNBZM2BB62qvO29@elch.exwg.net>
From: Ron Johnson <ronljohnsonjr@gmail.com>
Date: Thu, 12 Sep 2024 15:43:59 -0400
Message-ID: <CANzqJaAJe-HhKGtfvJXtL1q=U6FsJ+-hcRh=Wvzy=MKyAvTzeQ@mail.gmail.com>
Subject: Re: Better way to process records in bash?
To: pgsql-general <pgsql-general@postgresql.org>
Content-Type: multipart/alternative; boundary="000000000000b5f6970621f154b5"
Archived-At: <https://www.postgresql.org/message-id/CANzqJaAJe-HhKGtfvJXtL1q%3DU6FsJ%2B-hcRh%3DWvzy%3DMKyAvTzeQ%40mail.gmail.com>
Precedence: bulk

--000000000000b5f6970621f154b5
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Sep 12, 2024 at 3:30=E2=80=AFPM Christoph Moench-Tegeder <cmt@burgg=
raben.net>
wrote:

> ## Ron Johnson (ronljohnsonjr@gmail.com):
>
> > I need to process table records in a bash script.  Currently, I read th=
em
> > using a while loop and redirection.  The table isn't that big (30ish
> > thousand rows), and performance is adequate, but am always looking for
> > "better".
>
> Use python, or any other sane language, and don't do bash in python.
> Anything non-trivial is unreasonably hard to get really right when
> using shell.
>
> Take this example (even when allowing some fuzz with the variables):
> > Here's the current code:
> > declare f1 f3 f8
> > while IFS=3D'|' read f1 f3 f8; do
> >     something f8 f3 f1
> > done < <(psql -XAt -c "select f1, f3, f8 from some.table_name;")
>
> What happens when f3
>  a) is NULL?
>  b) contains a '|' character?
>  c) contains spaces? (this one depends on how exactly you're handling
>     the shell variables).
> This might be fine if your data is only NOT NULL numbers, but that's
> already quite a limitation on your data.
>

Good points.  Shouldn't be relevant in my situation, but
defensive programming never hurts.

--=20
Death to America, and butter sauce.
Iraq lobster!

--000000000000b5f6970621f154b5
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Thu, Sep 12, 2024 at 3:30=E2=80=AFPM C=
hristoph Moench-Tegeder &lt;<a href=3D"mailto:cmt@burggraben.net">cmt@burgg=
raben.net</a>&gt; wrote:<br></div><div class=3D"gmail_quote"><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid=
 rgb(204,204,204);padding-left:1ex">## Ron Johnson (<a href=3D"mailto:ronlj=
ohnsonjr@gmail.com" target=3D"_blank">ronljohnsonjr@gmail.com</a>):<br>
<br>
&gt; I need to process table records in a bash script.=C2=A0 Currently, I r=
ead them<br>
&gt; using a while loop and redirection.=C2=A0 The table isn&#39;t that big=
 (30ish<br>
&gt; thousand rows), and performance is adequate, but am always looking for=
<br>
&gt; &quot;better&quot;.<br>
<br>
Use python, or any other sane language, and don&#39;t do bash in python.<br=
>
Anything non-trivial is unreasonably hard to get really right when<br>
using shell.<br>
<br>
Take this example (even when allowing some fuzz with the variables):<br>
&gt; Here&#39;s the current code:<br>
&gt; declare f1 f3 f8<br>
&gt; while IFS=3D&#39;|&#39; read f1 f3 f8; do<br>
&gt;=C2=A0 =C2=A0 =C2=A0something f8 f3 f1<br>
&gt; done &lt; &lt;(psql -XAt -c &quot;select f1, f3, f8 from some.table_na=
me;&quot;)<br>
<br>
What happens when f3<br>
=C2=A0a) is NULL?<br>
=C2=A0b) contains a &#39;|&#39; character?<br>
=C2=A0c) contains spaces? (this one depends on how exactly you&#39;re handl=
ing<br>
=C2=A0 =C2=A0 the shell variables).<br>
This might be fine if your data is only NOT NULL numbers, but that&#39;s<br=
>
already quite a limitation on your data.<br></blockquote><div><br></div><di=
v>Good points.=C2=A0 Shouldn&#39;t be relevant in my=C2=A0situation, but de=
fensive=C2=A0programming never hurts.</div><div><br></div></div><span class=
=3D"gmail_signature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_s=
ignature"><div dir=3D"ltr">Death to America, and butter sauce.<div>Iraq lob=
ster!</div></div></div></div>

--000000000000b5f6970621f154b5--