public inbox for [email protected]  

help / color / mirror / Atom feed

From: Kostas Chasialis <[email protected]>
To: [email protected]
Subject: [ERROR] Copy from CSV fails due to memory error.
Date: Wed, 19 Jan 2022 15:01:25 +0200
Message-ID: <[email protected]> (raw)

Hey.

I am facing an issue when I try to run the following command

COPY <table_name> FROM <file> WITH DELIMITER E',’;

This file, is rather large, it's around 178GBs. 

When I try to run this COPY command I get the following error:

ERROR:  out of memory
DETAIL:  Failed on request of size 2048 in memory context "AfterTriggerEvents".
CONTEXT:  COPY ssbm300_lineorder, line 50796791

Clearly a memory allocation function is failing but I have no clue how to fix it.

I have tried experimenting with shared_buffers value in postgresql.conf file but after searching a bit I quickly realized that I do not know what I am doing there so I left it with default value. Same with work_mem value.

Did you face this issue before? Can you help me resolve it?

Thanks in advance!

view thread (2+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected]
  Subject: Re: [ERROR] Copy from CSV fails due to memory error.
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox