pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Rob Sargent <robjsargent@gmail.com>
To: pgsql-general@lists.postgresql.org
Subject: Re: Fatal Error : Invalid Memory alloc request size 1236252631
Date: Thu, 17 Aug 2023 10:34:20 -0600
Message-ID: <7adb97ac-5433-054d-32a0-a1046f7212da@gmail.com> (raw)
In-Reply-To: <CADBXDMWi1DKu2Zf52kkSgtUGO63LG+1ZpD3eXpuSYL2b53z47A@mail.gmail.com>
References: <CADBXDMXVtbyHbqMD73P2tX54u55XewvYLt+a+VEsyzrtQuo6jA@mail.gmail.com>
	<6BDA46BE-B926-4BF7-AEBF-C9153D59A51D@a-kretschmer.de>
	<CADBXDMWBR1jWXioyBOX60UTTOecyPEJ6Mb0-yR5C5r8ZrwF50A@mail.gmail.com>
	<CADBXDMU3J-ZcuCoRVaQGk9YufE=1wzZ4X1VEHFTvMXheiX8YTw@mail.gmail.com>
	<a110e3f6-4cf1-1706-9604-c3f20afd1082@gmail.com>
	<CADBXDMUsV948B06XwLamQ5PFhDO-fZEz3Bi9fPNzaEOP4sLiDg@mail.gmail.com>
	<fc1a13d5-3c9a-8480-94f9-0f20446d1e43@gmail.com>
	<CADBXDMUBs1uvrgeFrG0GcywY8m6_BOmuQRYoEFdFyrsPTK0Jtg@mail.gmail.com>
	<CADBXDMWaBR4GBg1ff=8NuJijtR+Qs6nGGn5BuTjOq-k_34v1Fw@mail.gmail.com>
	<1022628.1692069795@sss.pgh.pa.us>
	<CADBXDMWi1DKu2Zf52kkSgtUGO63LG+1ZpD3eXpuSYL2b53z47A@mail.gmail.com>

On 8/17/23 07:35, Sai Teja wrote:
> Hi Team,
>
> Even I used postgreSQL Large Objects by referring this link to store 
> and retrieve large files (As bytea not working)
> https://www.postgresql.org/docs/current/largeobjects.html
>
> But even now I am unable to fetch the data at once from large objects
>
> select lo_get(oid);
>
> Here I'm getting the same error message.
>
> But if I use select data from pg_large_object where loid = 49374
> Then I can fetch the data but in page wise (data splitting into rows 
> of each size 2KB)
>
> So, here how can I fetch the data at single step rather than page by 
> page without any error.
>
> And I'm just wondering how do many applications storing huge amount of 
> data in GBs? I know that there is 1GB limit for each field set by 
> postgreSQL. If so, how to deal with these kind of situations? Would 
> like to know about this to deal with real time scenarios.
>
> We need to store large content (huge volume of data) and retrieve it. 
> Currently It is not happening due to limit of field size set by 
> postgreSQL.
>
> Would request to share your insights and suggestions on this to help 
> me for resolving this issue.
>
>
>
My first attempt at handling large payload was to use Java Selector 
directly in my app.  This worked but manually chunking the data was 
tricky.  I switched to using Tomcat and it handles large http(s) 
payloads seamlessly.







view thread (12+ messages)

Message-ID: <7adb97ac-5433-054d-32a0-a1046f7212da@gmail.com>
Permalink:  ../7adb97ac-5433-054d-32a0-a1046f7212da@gmail.com/
Also on:    postgresql.org/message-id/7adb97ac-5433-054d-32a0-a1046f7212da@gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: robjsargent@gmail.com
  Subject: Re: Fatal Error : Invalid Memory alloc request size 1236252631
  In-Reply-To: <7adb97ac-5433-054d-32a0-a1046f7212da@gmail.com>
  References: <CADBXDMXVtbyHbqMD73P2tX54u55XewvYLt+a+VEsyzrtQuo6jA@mail.gmail.com> <6BDA46BE-B926-4BF7-AEBF-C9153D59A51D@a-kretschmer.de> <CADBXDMWBR1jWXioyBOX60UTTOecyPEJ6Mb0-yR5C5r8ZrwF50A@mail.gmail.com> <CADBXDMU3J-ZcuCoRVaQGk9YufE=1wzZ4X1VEHFTvMXheiX8YTw@mail.gmail.com> <a110e3f6-4cf1-1706-9604-c3f20afd1082@gmail.com> <CADBXDMUsV948B06XwLamQ5PFhDO-fZEz3Bi9fPNzaEOP4sLiDg@mail.gmail.com> <fc1a13d5-3c9a-8480-94f9-0f20446d1e43@gmail.com> <CADBXDMUBs1uvrgeFrG0GcywY8m6_BOmuQRYoEFdFyrsPTK0Jtg@mail.gmail.com> <CADBXDMWaBR4GBg1ff=8NuJijtR+Qs6nGGn5BuTjOq-k_34v1Fw@mail.gmail.com> <1022628.1692069795@sss.pgh.pa.us> <CADBXDMWi1DKu2Zf52kkSgtUGO63LG+1ZpD3eXpuSYL2b53z47A@mail.gmail.com> <7adb97ac-5433-054d-32a0-a1046f7212da@gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox