Received: from malur.postgresql.org ([217.196.149.56])
	by arkaria.postgresql.org with esmtps
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(Exim 4.92)
	(envelope-from <psycopg-owner+archive@lists.postgresql.org>)
	id 1oxvvS-00020l-2q
	for psycopg@arkaria.postgresql.org; Wed, 23 Nov 2022 20:00:54 +0000
Received: from localhost ([127.0.0.1] helo=malur.postgresql.org)
	by malur.postgresql.org with esmtp (Exim 4.92)
	(envelope-from <psycopg-owner+archive@lists.postgresql.org>)
	id 1oxvvQ-0000rj-Jf
	for psycopg@arkaria.postgresql.org; Wed, 23 Nov 2022 20:00:52 +0000
Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29])
	by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(Exim 4.92)
	(envelope-from <daniele.varrazzo@gmail.com>)
	id 1oxvvQ-0000ra-CN
	for psycopg@lists.postgresql.org; Wed, 23 Nov 2022 20:00:52 +0000
Received: from mail-oi1-x229.google.com ([2607:f8b0:4864:20::229])
	by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128)
	(Exim 4.92)
	(envelope-from <daniele.varrazzo@gmail.com>)
	id 1oxvvO-00023M-9K
	for psycopg@lists.postgresql.org; Wed, 23 Nov 2022 20:00:51 +0000
Received: by mail-oi1-x229.google.com with SMTP id m204so20182271oib.6
        for <psycopg@lists.postgresql.org>;
 Wed, 23 Nov 2022 12:00:49 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20210112;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:from:to:cc:subject:date:message-id:reply-to;
        bh=84c/8/U1c0BZR8c1xAI+d+aW12+yungIlbDMy++bj6k=;
        b=FZ/ucuqqWDHEkQ4SQx8bBO/ohY/wwh4aQ1Q4I1B/hQb0W4FS/Xx55rLe24K791pC7q
         5CMtZYk57EGHr9cgWr57XxmjnnfdUhZ9+uSlALueMSbHc3Dc8Kckh9QZqzAmDSg5HgHc
         qcyM1Cw/PpTpyYEH0wXB4UvuBrGlHNEDgCrR2KCFg3Qo/BXME8EDPr9+FmxcwwBS4iMs
         E98yabTzzRMg8glQUzp9Wyf/86XXJWfqH9crBhiuUTJ+dfi7nug+7GkaAYdid2O+ckUP
         8DIGHupG47JJI8Bzl0UMm50xsEs+YUyRBjga/ty8hPW8Agfq3dIWnXXG2eqPh5ik9fmM
         h/tg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20210112;
        h=cc:to:subject:message-id:date:from:in-reply-to:references
         :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
         :reply-to;
        bh=84c/8/U1c0BZR8c1xAI+d+aW12+yungIlbDMy++bj6k=;
        b=MKJkNhFOyl9jr87bzFpfaGTE8T885PrymE7q8eKbPuD5BKq1vE9UuzR8SnobbdGQeq
         kffUt1KOvYPzy1xg1XfWydZiGK0DjlbINoBQRLaJ5gsQxb4Cyk2JAAdVX1hn1xCoiPyh
         VItiqGO1wGLWc8jWA2yKDtZh6xaiHWYXcbZ17YGXtf5vewcf1KU9nEk03EIeeHXiE3AW
         MVLZB7i9ETAKByEHphjwoOOxc/7NeHbLPv4Q+zb02NtWsfs4UxnG5v9a0urey+5SvgKq
         U786kOQ38egnB078W9aXTBlo4ZZJ1pGxyu9dlTQoBiQOyx3BpcbxYapR7gSk6XiswgCP
         SzyA==
X-Gm-Message-State: ANoB5pkE96qWwMAAeNkwqJkjtaKvAZyQDtSo6qz0Memsr4txVtxDEZXh
	0JbdTdniKNs1vjd3D0/nBDSACVYAF3961SUgOunhUsfohcs=
X-Google-Smtp-Source: 
 AA0mqf4L7BoSQfQwrSy+puezKQWoMP8MBrF+OP1REyUJRKSjpnl68xk86Zc6pPUDPgNUnCQE5wkzBqVVFNUZloFSz0o=
X-Received: by 2002:a05:6808:5d4:b0:34f:d9fc:dddd with SMTP id
 d20-20020a05680805d400b0034fd9fcddddmr4918149oij.44.1669233648490; Wed, 23
 Nov 2022 12:00:48 -0800 (PST)
MIME-Version: 1.0
References: 
 <CAKvpR7MaATqwiDYG1YyixGnUrqyjjq1z2aO3T-a6Rh82vTvcWg@mail.gmail.com>
 <CA+mi_8buxHRjeabOPWfPpg1RKvRpC2tm1Ass7uttgFvLRoRH7Q@mail.gmail.com>
 <CAMqTPqnvLxiUcfOXLwBP3NsKo1OfEE8E-YhSHFUJV0GX2M9eEw@mail.gmail.com>
In-Reply-To: 
 <CAMqTPqnvLxiUcfOXLwBP3NsKo1OfEE8E-YhSHFUJV0GX2M9eEw@mail.gmail.com>
From: Daniele Varrazzo <daniele.varrazzo@gmail.com>
Date: Wed, 23 Nov 2022 21:00:35 +0100
Message-ID: 
 <CA+mi_8b-c8Q4zpkd=X+mqRivscEoFmn_f1UUh-wr3UjEdD+sXg@mail.gmail.com>
Subject: Re: Parquet support
To: Vladimir Ryabtsev <greatvovan@gmail.com>
Cc: Christopher Bader <cbader@zscaler.com>, psycopg@lists.postgresql.org
Content-Type: text/plain; charset="UTF-8"
List-Id: <psycopg.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:psycopg@lists.postgresql.org>
List-Owner: <mailto:psycopg-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/psycopg>
Archived-At: 
 <https://www.postgresql.org/message-id/CA%2Bmi_8b-c8Q4zpkd%3DX%2BmqRivscEoFmn_f1UUh-wr3UjEdD%2BsXg%40mail.gmail.com>
Precedence: bulk

On Wed, 23 Nov 2022 at 20:56, Vladimir Ryabtsev <greatvovan@gmail.com> wrote:
>
> Just curious folks, what are your thoughts about the scope of that potential support? What is the use case? Is it loading data from Parquet to Postgres (and back)? Why is the combination with Python modules like pyarrow not enough?

I am not an expert, but I understand that Python-Postgres roundtrip
goes via generating and parsing CSV files, whereas there is some
performance gain to be had by creating native arrow data.

-- Daniele