Subject: Re: Loading table with indexed jsonb field is stalling
To: pgsql-general@lists.postgresql.org
References: 
 <CAKMEDdxE95SC76wadMzrH454HpvCV4tOdEmuZRO=pwArEw6YVg@mail.gmail.com>
 <20505.1558202555@sss.pgh.pa.us>
 <90835D2B-1FBA-4F26-8157-7C831C24B266@gmail.com>
 <cdc049f0-1980-f5a8-726a-629345b41ef0@aklaver.com>
 <3A35A039-D8F5-4B40-9B41-42E70BEF2C2E@gmail.com>
 <806d4915-f05d-8e75-0a42-e11b48596ab5@gmail.com>
 <12E9FEDD-37E9-4D51-9BA7-94FF5039BB1B@gmail.com>
 <563e995f-d7b1-7e2f-c1f1-b00db7a18589@gmail.com>
 <3EF519CA-17FE-4B68-8B7A-78200E0A408F@gmail.com>
 <71e50367-2375-86a8-f795-97c894728998@gmail.com>
 <3CC10064-7BF0-4600-BC67-8B6F412C9CDC@gmail.com>
From: Ron <ronljohnsonjr@gmail.com>
Message-ID: <0a8876b8-f4de-4bc2-72c8-afe5dfe6fde2@gmail.com>
Date: Mon, 20 May 2019 19:15:16 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <3CC10064-7BF0-4600-BC67-8B6F412C9CDC@gmail.com>
Content-Type: multipart/alternative;
 boundary="------------A18D5D4EA4EE3EE8BD240DB2"
Content-Language: en-US
Precedence: bulk

This is a multi-part message in MIME format.
--------------A18D5D4EA4EE3EE8BD240DB2
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

On 5/20/19 6:51 PM, Will Hartung wrote:
>> On May 20, 2019, at 4:27 PM, Ron <ronljohnsonjr@gmail.com 
>> <mailto:ronljohnsonjr@gmail.com>> wrote:
>>
>> I'm confused.  You wrote above that loading without indexes and with just 
>> the PK works just fine; if you *really* need it loaded in Aurora or 
>> production, just drop the indexes beforehand?
>
> Because the jsonb_path_ops index does not rebuild, and it’s the heart of 
> that table.
>
> The table is essentially worthless without that index to us.

Clarified.

Are there a sufficiently small number of elements in each traits object that 
you can do something like this, on the UNINDEXED table?
SELECT traits->element1, traits->element2, count(*)
from eis_entry
group by traits->element1, traits->element2;

Sure it would run for a long time, but might show you where the problem lies.


-- 
Angular momentum makes the world go 'round.

--------------A18D5D4EA4EE3EE8BD240DB2
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    On 5/20/19 6:51 PM, Will Hartung wrote:<br class="">
    <blockquote type="cite"
      cite="mid:3CC10064-7BF0-4600-BC67-8B6F412C9CDC@gmail.com">
      <div>
        <blockquote type="cite" class="">
          <div class="">On May 20, 2019, at 4:27 PM, Ron &lt;<a
              href="mailto:ronljohnsonjr@gmail.com" class=""
              moz-do-not-send="true">ronljohnsonjr@gmail.com</a>&gt;
            wrote:</div>
          <div class="">
            <div bgcolor="#FFFFFF" text="#000000" class=""><br class="">
              I'm confused.  You wrote above that loading without
              indexes and with just the PK works just fine; if you <b
                class="">really</b> need it loaded in Aurora or
              production, just drop the indexes beforehand?<br class="">
            </div>
          </div>
        </blockquote>
        <br class="">
      </div>
      <div>Because the jsonb_path_ops index does not rebuild, and it’s
        the heart of that table.</div>
      <div><br class="">
      </div>
      <div>The table is essentially worthless without that index to us.</div>
    </blockquote>
    <br>
    Clarified.<br>
    <br>
    Are there a sufficiently small number of elements in each traits
    object that you can do something like this, on the UNINDEXED table?<br>
    SELECT traits-&gt;element1, traits-&gt;element2, count(*)<br>
    from eis_entry<br>
    group by traits-&gt;element1, traits-&gt;element2;<br>
    <br>
    Sure it would run for a long time, but might show you where the
    problem lies.<br>
    <br>
    <br>
    <div class="moz-signature">-- <br>
      Angular momentum makes the world go 'round.</div>
  </body>
</html>

--------------A18D5D4EA4EE3EE8BD240DB2--