public inbox for [email protected]  

help / color / mirror / Atom feed

From: Adrian Klaver <[email protected]>
To: sud <[email protected]>
To: pgsql-general <[email protected]>
Subject: Re: Load a csv or a avro?
Date: Fri, 5 Jul 2024 07:54:49 -0700
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAD=mzVUo9UpTw7F_8HKDK19ZmO6tE6Cfa4T-7i1J_QGfi6NpOw@mail.gmail.com>
References: <CAD=mzVUo9UpTw7F_8HKDK19ZmO6tE6Cfa4T-7i1J_QGfi6NpOw@mail.gmail.com>

On 7/5/24 02:08, sud wrote:
> Hello all,
> 
> Its postgres database. We have option of getting files in csv and/or in 
> avro format messages from another system to load it into our postgres 
> database. The volume will be 300million messages per day across many 
> files in batches.

Are dumping the entire contents of each file or are you pulling a 
portion of the data out?

> 
> My question was, which format should we chose in regards to faster data 
> loading performance ? and if any other aspects to it also should be 
> considered apart from just loading performance?
> 

-- 
Adrian Klaver
[email protected]

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Load a csv or a avro?
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox