pg.ddx.io  pgsql-general@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Adrian Klaver <adrian.klaver@aklaver.com>
To: Will Hartung <willhartung@gmail.com>
Cc: Ron <ronljohnsonjr@gmail.com>
Cc: pgsql-general@lists.postgresql.org
Subject: Re: Loading table with indexed jsonb field is stalling
Date: Mon, 20 May 2019 17:33:52 -0700
Message-ID: <9aa4804b-a5fe-9036-bb7d-07732c9fb99b@aklaver.com> (raw)
In-Reply-To: <52ACFC7C-47B8-4BF1-BDF1-8810F02BD297@gmail.com>
References: <CAKMEDdxE95SC76wadMzrH454HpvCV4tOdEmuZRO=pwArEw6YVg@mail.gmail.com>
	<20505.1558202555@sss.pgh.pa.us>
	<90835D2B-1FBA-4F26-8157-7C831C24B266@gmail.com>
	<cdc049f0-1980-f5a8-726a-629345b41ef0@aklaver.com>
	<3A35A039-D8F5-4B40-9B41-42E70BEF2C2E@gmail.com>
	<806d4915-f05d-8e75-0a42-e11b48596ab5@gmail.com>
	<12E9FEDD-37E9-4D51-9BA7-94FF5039BB1B@gmail.com>
	<563e995f-d7b1-7e2f-c1f1-b00db7a18589@gmail.com>
	<3EF519CA-17FE-4B68-8B7A-78200E0A408F@gmail.com>
	<7c0a001e-d3d5-c521-1a27-77d05f547fef@aklaver.com>
	<375E86FF-5B4D-4896-B4ED-FCA8C1D3A60B@gmail.com>
	<703a6c76-854d-3d1c-8be6-30864c873d05@aklaver.com>
	<52ACFC7C-47B8-4BF1-BDF1-8810F02BD297@gmail.com>

On 5/20/19 5:22 PM, Will Hartung wrote:
> 
>> On May 20, 2019, at 5:14 PM, Adrian Klaver <adrian.klaver@aklaver.com> wrote:
>>
>> Well looks like you are down to Tom's suggestion of creating a test case. Given that it seems confined to the jsonb field and corresponding index, I would think that is all that is needed for the test case. Start with some smaller subset, say 10,000 rows and work up till you start seeing an issue.
> 
> This will take quite some work, and I wouldn’t attempt it with less than 5M rows to load.
> 

 From this post:
https://www.postgresql.org/message-id/CAKMEDdxE95SC76wadMzrH454HpvCV4tOdEmuZRO%3DpwArEw6YVg%40mail.g...

You started seeing longer times by the time you got to 500,000 rows and 
then it fell off by 1.1 million.

Creating the data:

CREATE TABLE AS gin_test SELECT traits FROM eis_entity LIMIT 1000000;

Then:

pg_dump -t gin_test

> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com





view thread (30+ messages)  latest in thread

Message-ID: <9aa4804b-a5fe-9036-bb7d-07732c9fb99b@aklaver.com>
Permalink:  ../9aa4804b-a5fe-9036-bb7d-07732c9fb99b@aklaver.com/
Also on:    postgresql.org/message-id/9aa4804b-a5fe-9036-bb7d-07732c9fb99b@aklaver.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-general@postgresql.org
  Cc: adrian.klaver@aklaver.com, willhartung@gmail.com, ronljohnsonjr@gmail.com, pgsql-general@lists.postgresql.org
  Subject: Re: Loading table with indexed jsonb field is stalling
  In-Reply-To: <9aa4804b-a5fe-9036-bb7d-07732c9fb99b@aklaver.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox