MIME-Version: 1.0
References: 
 <CA+i5JwaC9Z-rjWiyo7LpncdK=CuQcKzuVNMnAhPW1oi=8c9Sug@mail.gmail.com>
 <CA+i5JwZ1_gu0g-X+5P8hVhyc1YS7F4nhrcEQdZKADOp+7o+1SQ@mail.gmail.com>
In-Reply-To: 
 <CA+i5JwZ1_gu0g-X+5P8hVhyc1YS7F4nhrcEQdZKADOp+7o+1SQ@mail.gmail.com>
From: Steve Midgley <science@misuse.org>
Date: Thu, 28 Apr 2022 10:14:47 -0700
Message-ID: 
 <CAJexoSJy1yXa9vQuoN0U8ykf0w29yWJWw9aODak6sHsx+wzjNw@mail.gmail.com>
Subject: Re: parallel-processing multiple similar query tasks - any example?
To: Shaozhong SHI <shishaozhong@gmail.com>
Cc: pgsql-sql <pgsql-sql@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="000000000000c778f505ddba10b0"
Archived-At: 
 <https://www.postgresql.org/message-id/CAJexoSJy1yXa9vQuoN0U8ykf0w29yWJWw9aODak6sHsx%2BwzjNw%40mail.gmail.com>
Precedence: bulk

--000000000000c778f505ddba10b0
Content-Type: text/plain; charset="UTF-8"

On Wed, Apr 27, 2022 at 4:34 PM Shaozhong SHI <shishaozhong@gmail.com>
wrote:

>
>
>
> multiple similar query tasks are as follows:
>
> select * from a_table where country ='UK'
> select * from a_table where country='France'
> and so on
>
> How best to parallel-processing such types of multiple similar query tasks?
>
>
> This depends on how you are engaging with the queries when they return
results. Let's assume you are running them from a programming environment
with an ORM layer. In that case you can run each query in a separate thread
and connection pipe, and the queries will run async just fine. If you are
running at the command line using psql, you can just open multiple shells
and run each query from a different shell terminal.

Postgres is very good at async queries, so your challenge is really
figuring out how you will use the results of each query and setting up the
environment sending the queries to perform asynchronously.

Steve

--000000000000c778f505ddba10b0
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Wed, Apr 27, 2022 at 4:34 PM Shaoz=
hong SHI &lt;<a href=3D"mailto:shishaozhong@gmail.com">shishaozhong@gmail.c=
om</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margi=
n:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex=
"><div dir=3D"ltr"><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gma=
il_attr"><br></div><br><br><div dir=3D"ltr">multiple similar query tasks ar=
e as follows:<div><br></div><div>select * from a_table where country =3D=
9;UK&#39;</div><div>select * from a_table where country=3D&#39;France&#39;<=
/div><div>and so on</div><div><br></div><div>How best to parallel-processin=
g such types of multiple similar query tasks?</div><div><br></div><div><br>=
</div></div></div></div></blockquote><div>This depends on how you are engag=
ing with the queries when they return results. Let&#39;s assume you are run=
ning them from a programming environment with an ORM layer. In that case yo=
u can run each query in a separate thread and connection pipe, and the quer=
ies will run async just fine. If you are running at the command line using =
psql, you can just open multiple shells and run each query from a different=
 shell terminal.</div><div><br></div><div>Postgres is very good at async qu=
eries, so your challenge is really figuring out how you will use the result=
s of each query and setting up the environment sending the queries to perfo=
rm asynchronously.</div><div><br></div><div>Steve</div></div></div>

--000000000000c778f505ddba10b0--