public inbox for [email protected]  

help / color / mirror / Atom feed
From: Nicolas Paris <[email protected]>
To: Josh Berkus <[email protected]>
Cc: David G. Johnston <[email protected]>
Cc: pgsql-performance <[email protected]>
Subject: Re: array size exceeds the maximum allowed (1073741823) when building a json
Date: Tue, 7 Jun 2016 21:23:45 +0200
Message-ID: <CA+ssMOSji1OQg3Hby01T8mUkSMcvCOSJtfFC3jjV36MGpq83pg@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <CA+ssMORBYVFxedSXVVxL5VPeo=9AbYRQ-zbpuKJm6vEjmJ1CFA@mail.gmail.com>
	<CAKFQuwbjwyaaFUOEzxoYCBDm25ujOayCniwiWimPML_A_VaGYA@mail.gmail.com>
	<CA+ssMOT04Wg+OGfSc5AUk8nVQOxkVS6=CD-h7bH-pAH-ix50cw@mail.gmail.com>
	<CAKFQuwZEkX6o2mv2Ekqp8DpAs9F0bMsBnvQxzJGUr+SS6t2yTg@mail.gmail.com>
	<CA+ssMOSdHUTMWqswZ4njAgtf4vpAkuZ8duH7znB0G5pEswa38w@mail.gmail.com>
	<WM!d318efe352c6e124baed8053dba915699725885de80c4fdd70cff903faba6afc6e6233ba053fb428df2d56162a988029!@mailstronghold-1.zmailcloud.com>
	<[email protected]>
List-Unsubscribe:  <mailto:[email protected]?body=unsub%20pgsql-performance>

2016-06-07 15:03 GMT+02:00 Josh Berkus <[email protected]>:

> On 06/07/2016 08:42 AM, Nicolas Paris wrote:
> >     ​​You have to do something different.  Using multiple columns and/or
> >     multiple rows might we workable.
>

​Getting a unique document from multiple rows coming from postgresql is not
that easy... The external tools considers each postgresql JSON fields as
strings or have to parse it again. Parsing them would add an overhead on
the external tool, and I d'say this would be better to build the entire
JSON in the external tool. This leads not to use postgresql JSON builder at
all, and delegate this job to a tool that is able to deal with > 1GO
documents.



> >
> >
> > ​Certainly. Kind of disappointing, because I won't find any json builder
> > as performant as postgresql.​
>
> That's nice to hear.
>
> > Will this 1GO restriction is supposed to increase in a near future ?​
>
> Not planned, no.  Thing is, that's the limit for a field in general, not
> just JSON; changing it would be a fairly large patch.  It's desireable,
> but AFAIK nobody is working on it.
>

Comparing to mongoDB 16MO document limitation 1GO is great (
http://tech.tulentsev.com/2014/02/limitations-of-mongodb/)​. But for my use
case this is not sufficient.



> --
> --
> Josh Berkus
> Red Hat OSAS
> (any opinions are my own)
>


reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: array size exceeds the maximum allowed (1073741823) when building a json
  In-Reply-To: <CA+ssMOSji1OQg3Hby01T8mUkSMcvCOSJtfFC3jjV36MGpq83pg@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox