Re: Loading table with indexed jsonb field is stalling

pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Will Hartung <willhartung@gmail.com>
Cc: Adrian Klaver <adrian.klaver@aklaver.com>
Cc: Ron <ronljohnsonjr@gmail.com>
Cc: pgsql-general@lists.postgresql.org
Subject: Re: Loading table with indexed jsonb field is stalling
Date: Mon, 20 May 2019 20:31:47 -0400
Message-ID: <22234.1558398707@sss.pgh.pa.us> (raw)
In-Reply-To: <52ACFC7C-47B8-4BF1-BDF1-8810F02BD297@gmail.com>
References: <CAKMEDdxE95SC76wadMzrH454HpvCV4tOdEmuZRO=pwArEw6YVg@mail.gmail.com>
	<20505.1558202555@sss.pgh.pa.us>
	<90835D2B-1FBA-4F26-8157-7C831C24B266@gmail.com>
	<cdc049f0-1980-f5a8-726a-629345b41ef0@aklaver.com>
	<3A35A039-D8F5-4B40-9B41-42E70BEF2C2E@gmail.com>
	<806d4915-f05d-8e75-0a42-e11b48596ab5@gmail.com>
	<12E9FEDD-37E9-4D51-9BA7-94FF5039BB1B@gmail.com>
	<563e995f-d7b1-7e2f-c1f1-b00db7a18589@gmail.com>
	<3EF519CA-17FE-4B68-8B7A-78200E0A408F@gmail.com>
	<7c0a001e-d3d5-c521-1a27-77d05f547fef@aklaver.com>
	<375E86FF-5B4D-4896-B4ED-FCA8C1D3A60B@gmail.com>
	<703a6c76-854d-3d1c-8be6-30864c873d05@aklaver.com>
	<52ACFC7C-47B8-4BF1-BDF1-8810F02BD297@gmail.com>

Will Hartung <willhartung@gmail.com> writes:
>> On May 20, 2019, at 5:14 PM, Adrian Klaver <adrian.klaver@aklaver.com> wrote:
>> Well looks like you are down to Tom's suggestion of creating a test case. Given that it seems confined to the jsonb field and corresponding index, I would think that is all that is needed for the test case. Start with some smaller subset, say 10,000 rows and work up till you start seeing an issue.

> This will take quite some work, and I wouldnt attempt it with less than 5M rows to load.

Well, you're the only one who's seen this problem, and none of the
rest of us have any idea how to reproduce it.  So if you want something
to get done in a timely fashion, it's up to you to show us a test case.

My guess is that it wouldn't be that hard to anonymize your data to
the point where it'd be OK to show to someone else.  It's unlikely
that the problem depends on the *exact* data you've got --- though it
might depend on string lengths and the number/locations of duplicates.
But you should be able to substitute random strings for the original
values while preserving that.

			regards, tom lane

view thread (30+ messages)  latest in thread

Message-ID: <22234.1558398707@sss.pgh.pa.us>
Permalink:  ../22234.1558398707@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/22234.1558398707@sss.pgh.pa.us

 ·

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-general@postgresql.org
  Cc: tgl@sss.pgh.pa.us, willhartung@gmail.com, adrian.klaver@aklaver.com, ronljohnsonjr@gmail.com, pgsql-general@lists.postgresql.org
  Subject: Re: Loading table with indexed jsonb field is stalling
  In-Reply-To: <22234.1558398707@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox