public inbox for [email protected]help / color / mirror / Atom feed
How best to do parallel query given tens of thousands of iteration of a loop of recursive queries? 3+ messages / 3 participants [nested] [flat]
* How best to do parallel query given tens of thousands of iteration of a loop of recursive queries? @ 2022-04-10 13:50 Shaozhong SHI <[email protected]> 2022-04-10 16:03 ` Re: How best to do parallel query given tens of thousands of iteration of a loop of recursive queries? Steve Midgley <[email protected]> 2022-04-10 16:13 ` Re: How best to do parallel query given tens of thousands of iteration of a loop of recursive queries? David G. Johnston <[email protected]> 0 siblings, 2 replies; 3+ messages in thread From: Shaozhong SHI @ 2022-04-10 13:50 UTC (permalink / raw) To: pgsql-sql <[email protected]> There is a plpgsql script that have a loop to carry out the same recursive queries. The estimation of iteration is in the order of tens of thousands. What is the best way of making using parallel query strategy. Any examples? Regards, David ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: How best to do parallel query given tens of thousands of iteration of a loop of recursive queries? 2022-04-10 13:50 How best to do parallel query given tens of thousands of iteration of a loop of recursive queries? Shaozhong SHI <[email protected]> @ 2022-04-10 16:03 ` Steve Midgley <[email protected]> 1 sibling, 0 replies; 3+ messages in thread From: Steve Midgley @ 2022-04-10 16:03 UTC (permalink / raw) To: Shaozhong SHI <[email protected]>; +Cc: pgsql-sql <[email protected]> On Sun, Apr 10, 2022, 6:50 AM Shaozhong SHI <[email protected]> wrote: > There is a plpgsql script that have a loop to carry out the same recursive > queries. > > The estimation of iteration is in the order of tens of thousands. > > What is the best way of making using parallel query strategy. > > Any examples? > Can you provide some sample ddl, data and sql to illustrate your question? You mention in one place "iteration" and "parallel" but in another you mention recursion.. Recursion (by definition?) involves pushing state data forward into each successive iteration.. Therefore you can't run recursion in parallel. If you can provide examples maybe we can see which if parallelism is a possibility.. Steve ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: How best to do parallel query given tens of thousands of iteration of a loop of recursive queries? 2022-04-10 13:50 How best to do parallel query given tens of thousands of iteration of a loop of recursive queries? Shaozhong SHI <[email protected]> @ 2022-04-10 16:13 ` David G. Johnston <[email protected]> 1 sibling, 0 replies; 3+ messages in thread From: David G. Johnston @ 2022-04-10 16:13 UTC (permalink / raw) To: Shaozhong SHI <[email protected]>; +Cc: pgsql-sql <[email protected]> On Sun, Apr 10, 2022 at 6:50 AM Shaozhong SHI <[email protected]> wrote: > There is a plpgsql script that have a loop to carry out the same recursive > queries. > > The estimation of iteration is in the order of tens of thousands. > > What is the best way of making using parallel query strategy. > An example would be helpful. However, as a general guideline, since parallelism is done at the per-row scope, removing looping logic from the script and turning the main script logic into one or more functions that operate on a single row, while obeying the rules such functions need to abide by in order to be marked parallel safe, will open up the possibility for the server to process different rows using different workers and then appending their results together for the next node to consume. David J. ^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2022-04-10 16:13 UTC | newest] Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-04-10 13:50 How best to do parallel query given tens of thousands of iteration of a loop of recursive queries? Shaozhong SHI <[email protected]> 2022-04-10 16:03 ` Steve Midgley <[email protected]> 2022-04-10 16:13 ` David G. Johnston <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox