From: Ron <ronljohnsonjr@gmail.com>
To: pgsql-general@lists.postgresql.org
Subject: Re: Multiple inserts with two levels of foreign keys
Date: Wed, 4 Oct 2023 21:31:15 -0500
Message-ID: <01d36161-6e20-6bef-41c0-9bdb11e50218@gmail.com> (raw)
In-Reply-To: <CACM7_5bOexfdrCsOD64CE+uBTkuZ0iR9Y_e=xq63gq+Ms5Crjw@mail.gmail.com>
References: <CACM7_5Zjm=hdPv9F+74bwocgABTLi_sFDeVgG2H0HFPYD3LR4w@mail.gmail.com>
<a2ba4c8c-746a-43f9-bc53-8e32cc49114b@gmail.com>
<CACM7_5aprQ5SkyQ-vgbqA6DKztNRZivpVO7k8RWYaNUrFu=PgA@mail.gmail.com>
<bc7acb2a-77db-7d0a-fe3b-ae878839db6f@gmail.com>
<CACM7_5bOexfdrCsOD64CE+uBTkuZ0iR9Y_e=xq63gq+Ms5Crjw@mail.gmail.com>
Have you considered writing a stored procedure to process records that have
been written to temporary tables?
0. Create temporary tables tmp_farms, tmp_crops and tmp_deliveries, which
don't have id columns.
1. Truncate the three temporary tables
2. Insert into the temp tables a "set" of prod data.
3. Call a stored procedure in the dev database that does INSERT INTO ...,
using RETURNING to get the relevant id values for the subsequent tables.
4. goto 1.
On 10/4/23 21:15, Dow Drake wrote:
> I see. That would definitely work, but part of this for me is to get a
> better understanding of PostgreSQL's capabilities. I'm going to keep
> working on a minimal solution that deletes no records from the dev
> database, and only inserts the required records.
>
> On Wed, Oct 4, 2023 at 6:58 PM Ron <ronljohnsonjr@gmail.com> wrote:
>
> Ah. We'd truncate all of the dev tables, then load a "slice" (for
> example, accounts 10000 to 19999, and all associated records from
> downstream tables; lots and lots of views!!) from the prod database.
>
> On 10/4/23 20:50, Dow Drake wrote:
>> Thanks for the reply, Ron!
>> I'm not sure I see how to make your suggestion work, though. Suppose
>> I dump the three tables to CSV as you suggest (and write a script to
>> extract the relevant records from those CSV dumps in the correct
>> order). It might be that in the dev database, the next generated key
>> values are 199 for farm's id, 2145 for crop's id and 10242 for
>> deliveries' id. The databases are independent.
>>
>> Just inserting the records in the same order doesn't take care of
>> setting the foreign key values correctly -- does it? I think I'm
>> really looking for a solution more along the lines of the link in my
>> original post.
>>
>> Best,
>> Dow
>>
>> On Wed, Oct 4, 2023 at 6:26 PM Ron <ronljohnsonjr@gmail.com> wrote:
>>
>> Frame challenge: why can't you just "\copy to" the dev database
>> tables in the correct order, to satisfy foreign key requirements?
>>
>> On 10/4/23 18:59, Dow Drake wrote:
>>> Hi,
>>>
>>> I'm trying to write a postgresql script to replicate a
>>> hierarchical structure in a live database into my development
>>> database, where I can debug and test more easily. I can extract
>>> the data from the live database that needs to be inserted, but
>>> I'm having trouble writing the insertion script
>>>
>>> Here's a simplified version of the problem I'm trying to solve:
>>> There are three tables: farms, crops and deliveries where a farm
>>> has many crops and a crop has many deliveries.
>>>
>>> create table farms (
>>> id bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY,
>>> name character varying(30)
>>> );
>>> create table crops (
>>> id bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY,
>>> farm_id bigint not null
>>> name character varying(30)
>>> );
>>> create table deliveries (
>>> id bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY,
>>> crop_id bigint not null
>>> ticket character varying(30)
>>> );
>>> I want to insert a farm record, then insert two crops associated
>>> with that farm, then insert two deliveries for each of the the
>>> two crops so that in the end, my tables look like this:
>>> farms
>>> id name
>>> 1 'Happy Valley Farm'
>>>
>>> crops
>>> id farm_id name
>>> 1 1 'corn'
>>> 2 1 'wheat'
>>>
>>> delvieries
>>> id crop_id ticket
>>> 1 1 '3124'
>>> 2 2 '3127'
>>> 3 1 '3133'
>>> 4 2 '3140'
>>>
>>> It's important that the deliveries get assigned to the right
>>> crops. I think this post:
>>> https://dba.stackexchange.com/questions/199916
>>> gets close to what I need, but I haven't been able to figure out
>>> how to adapt it to multiple records.
>>>
>>> Thanks for any help on this!
>>>
>>>
>>
>> --
>> Born in Arizona, moved to Babylonia.
>>
>
> --
> Born in Arizona, moved to Babylonia.
>
--
Born in Arizona, moved to Babylonia.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-general@postgresql.org
Cc: ronljohnsonjr@gmail.com
Subject: Re: Multiple inserts with two levels of foreign keys
In-Reply-To: <01d36161-6e20-6bef-41c0-9bdb11e50218@gmail.com>
References: <CACM7_5Zjm=hdPv9F+74bwocgABTLi_sFDeVgG2H0HFPYD3LR4w@mail.gmail.com> <a2ba4c8c-746a-43f9-bc53-8e32cc49114b@gmail.com> <CACM7_5aprQ5SkyQ-vgbqA6DKztNRZivpVO7k8RWYaNUrFu=PgA@mail.gmail.com> <bc7acb2a-77db-7d0a-fe3b-ae878839db6f@gmail.com> <CACM7_5bOexfdrCsOD64CE+uBTkuZ0iR9Y_e=xq63gq+Ms5Crjw@mail.gmail.com> <01d36161-6e20-6bef-41c0-9bdb11e50218@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox