Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tFE00-00BzxA-Ux for pgsql-hackers@arkaria.postgresql.org; Sun, 24 Nov 2024 14:54:09 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1tFDzy-00819k-54 for pgsql-hackers@arkaria.postgresql.org; Sun, 24 Nov 2024 14:54:06 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tFDzx-00819b-LS for pgsql-hackers@lists.postgresql.org; Sun, 24 Nov 2024 14:54:05 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tFDzp-003bux-C9 for pgsql-hackers@postgresql.org; Sun, 24 Nov 2024 14:54:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732460035; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=0HSIJxS63g9674Dk1XXCI3fQwxJaVMRLOeflmXHxP4A=; b=ZC6564vWBWOuFrXZ/g17Br/PC8Yd/SQgsZXvnI44Bkz4DB58pfbRa/G6NtrmhVmKuyWyku vEWA8zS/tdwfh6GnF7YHsokaimIBmy5kmEriNV7PXhawWc0SrkteixZeQkwnGDBBVdyk6c 0Hx1siPa5BHxpPaPZPcX0+4xkaomXXs= Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-451-RC0Y6Ii_PGGrf3V2gnxgKg-1; Sun, 24 Nov 2024 09:53:51 -0500 X-MC-Unique: RC0Y6Ii_PGGrf3V2gnxgKg-1 X-Mimecast-MFC-AGG-ID: RC0Y6Ii_PGGrf3V2gnxgKg Received: by mail-io1-f70.google.com with SMTP id ca18e2360f4ac-83aa904b231so399204839f.1 for ; Sun, 24 Nov 2024 06:53:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732460030; x=1733064830; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=volkb67C98lSXclp6aiiNzmwR7gZPYCsKktrRo1yTV4=; b=rnM0QvFs72p5wXrpfw3zgwdj+YgbZ9DH/svXEd/EW64k7FdHzm7zUoe1Q4v2AdGqz9 JEqQ/lpzs11zjnfdEhiJoHVX5I3q6mR/wWg+cXNqK7ebVneBnswVwgiO32N9WQa6MqUm gGRz6lne91PGnxHCjzmzsIJ2ldVPvek1qLzJHwuT0A/x4PVUAHO0ram4foHFnCQCsICC rAFw1eQYVt0FxgLflsmZEbs3DFcxSJ9vXn6aELmr5qCzLC+Li2GOwNbJmsFRXw5x9SxC RdgwMl2GwzutV09s2IJIdRzAKLeSKX46DCw0uiiZX+V5Gr0oUO2HzClrrfVH9fdu7lOG Ncog== X-Gm-Message-State: AOJu0YwpAaUp6i55Bhb9YT+1N8L7DzM1R/uZ2Avp8gGbEcKWfV4DjqbI PczVE7H3FNOHoTynBpDkKPSGl+2qX31cIoKJ0qF6QWX/Qj4rullFiF7AKSMNhZrE6gurLP438HV A8moEIfxtCa6BEHZ+JDb3OBWxEGXlORnEkOZmjoebAeA7UUqmBgF9bsRcOmYvbEc/Bc42vLdvuC boKT8I7ZZ/9idgOnPnuixPBxV6m/5vlvj5FZOpqQidbUsZSdU= X-Gm-Gg: ASbGncvtMhjvllAp+VSF5XaXXno1V24BlBa/OSIeg4HNzAC4MsUA9SOVzvaGC7lW230 uNlrKZo7E+HwQQjg7W9aeps3zXtYL2pc= X-Received: by 2002:a05:6602:26d2:b0:83a:b7c8:a3dc with SMTP id ca18e2360f4ac-83ecdc370e8mr865344139f.1.1732460030182; Sun, 24 Nov 2024 06:53:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IH5A7GxczRf9lun2mgN3YnnGecT/w6wEnLpltw9kmPBYlbW3qZ/SDPcsd0honR0F5O3CkbU/IGEy1hWag5L0sM= X-Received: by 2002:a05:6602:26d2:b0:83a:b7c8:a3dc with SMTP id ca18e2360f4ac-83ecdc370e8mr865342539f.1.1732460029719; Sun, 24 Nov 2024 06:53:49 -0800 (PST) MIME-Version: 1.0 References: <29c87c10-fdbc-4d0f-b0f4-15e14dd36bce@vondra.me> In-Reply-To: <29c87c10-fdbc-4d0f-b0f4-15e14dd36bce@vondra.me> From: Filip Janus Date: Sun, 24 Nov 2024 15:53:38 +0100 Message-ID: Subject: Re: Proposal: Adding compression of temporary files To: Tomas Vondra Cc: pgsql-hackers@postgresql.org X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: GdocGMX3p0uycoZtIK8IpxFyGSu8pMFTOw7VZisRq6g_1732460030 X-Mimecast-Originator: redhat.com Content-Type: multipart/mixed; boundary="000000000000a8fe760627a9c8b8" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000a8fe760627a9c8b8 Content-Type: multipart/alternative; boundary="000000000000a8fe710627a9c8b6" --000000000000a8fe710627a9c8b6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable -Filip- st 20. 11. 2024 v 1:35 odes=C3=ADlatel Tomas Vondra napsa= l: > Hi, > > On 11/18/24 22:58, Filip Janus wrote: > > ... > > Hi all, > > Postgresql supports data compression nowadays, but the compression = of > > temporary files has not been implemented yet. The huge queries can > > produce a significant amount of temporary data that needs to > > be stored on disk > > and cause many expensive I/O operations. > > I am attaching a proposal of the patch to enable temporary files > > compression for > > hashjoins for now. Initially, I've chosen the LZ4 compression > > algorithm. It would > > probably make better sense to start with pglz, but I realized it > late. > > > > Thanks for the idea & patch. I agree this might be quite useful for > workloads generating a lot of temporary files for stuff like sorts etc. > I think it will be interesting to think about the trade offs, i.e. how > to pick the compression level - at some point the compression ratio > stops improving while paying more and more CPU time. Not sure what the > right choice is, so using default seems fine. > > I agree it'd be better to start with pglz, and only then add lz4 etc. > Firstly, pglz is simply the built-in compression, supported everywhere. > And it's also simpler to implement, I think. > > > # Future possible improvements > > Reducing the number of memory allocations within the dumping and > > loading of > > the buffer. I have two ideas for solving this problem. I would > > either add a buffer into > > struct BufFile or provide the buffer as an argument from the caller= . > > For the sequential > > execution, I would prefer the second option. > > > > Yes, this would be good. Doing a palloc+pfree for each compression is > going to be expensive, especially because these buffers are going to be > large - likely larger than 8kB. Which means it's not cached in the > memory context, etc. > > Adding it to the BufFile is not going to fly, because that doubles the > amount of memory per file. And we already have major issues with hash > joins consuming massive amounts of memory. But at the same time the > buffer is only needed during compression, and there's only one at a > time. So I agree with passing a single buffer as an argument. > > > # Future plan/open questions > > In the future, I would like to add support for pglz and zstd. > > Further, I plan to > > extend the support of the temporary file compression also for > > sorting, gist index creation, etc. > > > > Experimenting with the stream mode of compression algorithms. The > > compression > > ratio of LZ4 in block mode seems to be satisfying, but the stream > > mode could > > produce a better ratio, but it would consume more memory due to the > > requirement to store > > context for LZ4 stream compression. > > > > One thing I realized is that this only enables temp file compression for > a single place - hash join spill files. AFAIK this is because compressed > files don't support random access, and the other places might need that. > > Is that correct? The patch does not explain this anywhere. If that's > correct, the patch probably should mention this in a comment for the > 'compress' argument added to BufFileCreateTemp(), so that it's clear > when it's legal to set compress=3Dtrue. > I will add the description there. > Which other places might compress temp files? Surely hash joins are not > the only place that could benefit from this, right? > Yes, you are definitely right. I have chosen the hash joins as a POC because there are no seeks besides seeks at the beginning of the buffer. I have focused on hashjoins, but there are definitely also other places where the compression could be used. I want to add support in other places in the feature. > Another thing is testing. If I run regression tests, it won't use > compression at all, because the GUC has "none" by default, right? But we > need some testing, so how would we do that? One option would be to add a > regression test that explicitly sets the GUC and does a hash join, but > that won't work with lz4 (because that may not be enabled). Right, it's "none" by default. My opinion is that we would like to test every supported compression method, so I will try to add environment variable as you recommended. > > Another option might be to add a PG_TEST_xxx environment variable that > determines compression to use. Something like PG_TEST_USE_UNIX_SOCKETS. > But perhaps there's a simpler way. > > > # Benchmark > > I prepared three different databases to check expectations. Each > > dataset is described below. My testing demonstrates that my patch > > improves the execution time of huge hash joins. > > Also, my implementation should not > > negatively affect performance within smaller queries. > > The usage of memory needed for temporary files was reduced in every > > execution without a significant impact on execution time. > > > > *## Dataset A:* > > Tables* > > * > > table_a(bigint id,text data_text,integer data_number) - 10000000 ro= ws > > table_b(bigint id, integer ref_id, numeric data_value, bytea > > data_blob) - 10000000 rows > > Query: SELECT * FROM table_a a JOIN table_b b ON a.id > a.id> =3D b.id ; > > > > The tables contain highly compressible data. > > The query demonstrated a reduction in the usage of the temporary > > files ~20GB -> 3GB, based on this reduction also caused the > execution > > time of the query to be reduced by about ~10s. > > > > > > *## Dataset B:* > > Tables:* > > * > > table_a(integer id, text data_blob) - 1110000 rows > > table_b(integer id, text data_blob) - 10000000 rows > > Query: SELECT * FROM table_a a JOIN table_b b ON a.id > a.id> =3D b.id ; > > > > The tables contain less compressible data. data_blob was generated > > by a pseudo-random generator. > > In this case, the data reduction was only ~50%. Also, the execution > > time was reduced > > only slightly with the enabled compression. > > > > The second scenario demonstrates no overhead in the case of enabled > > compression and extended work_mem to avoid temp file usage. > > > > *## Dataset C:* > > Tables > > customers (integer,text,text,text,text) > > order_items(integer,integer,integer,integer,numeric(10,2)) > > orders(integer,integer,timestamp,numeric(10,2)) > > products(integer,text,text,numeric(10,2),integer) > > > > Query: SELECT p.product_id, p.name , p.price, > > SUM(oi.quantity) AS total_quantity, AVG(oi.price) AS avg_item_price > > FROM eshop.products p JOIN eshop.order_items oi ON p.product_id =3D > > oi.product_id JOIN > > eshop.orders o ON oi.order_id =3D o.order_id WHERE o.order_date > > > '2020-01-01' AND p.price > 50 > > GROUP BY p.product_id, p.name , p.price HAVING > > SUM(oi.quantity) > 1000 > > ORDER BY total_quantity DESC LIMIT 100; > > > > This scenario should demonstrate a more realistic usage of the > database. > > Enabled compression slightly reduced the temporary memory usage, bu= t > > the execution > > time wasn't affected by compression. > > > > > > +------------+-------------------------+----------------------- > > +------------------------------+ > > | Dataset | Compression. | temp_bytes | Execution > > Time (ms) | > > +------------+-------------------------+----------------------- > > +----------------------------- + > > | A | Yes | 3.09 GiB > > | 22s586ms | work_mem =3D 4MB > > | | No | 21.89 GiB > > | 35s | work_mem =3D 4MB > > +------------+-------------------------+----------------------- > > +---------------------------------------- > > | B | Yes | 333 MB > > | 1815.545 ms | work_mem =3D 4MB > > | | No | 146 MB > > | 1500.460 ms | work_mem =3D 4MB > > | | Yes | 0 MB > > | 3262.305 ms | work_mem =3D 80MB > > | | No | 0 MB > > | 3174.725 ms | work_mem =3D 80MB > > +-------------+------------------------+------------------------ > > +------------------------------------- > > | C | Yes | 40 MB > > | 1011.020 ms | work_mem =3D 1MB > > | | No | 53 > > MB | 1034.142 ms | work_mem =3D 1MB > > +------------+------------------------+------------------------ > > +-------------------------------------- > > > > > > Thanks. I'll try to do some benchmarks on my own. > > Are these results fro ma single run, or an average of multiple runs? It is average from multiple runs. Do > you maybe have a script to reproduce this, including the data generation? I am attaching my SQL file for database preparation. I also did further testing with two other machines( see attachment huge_tables.rtf ). > > Also, can you share some information about the machine used for this? I > expect the impact to strongly depends on memory pressure - if the temp > file fits into page cache (and stays there), it may not benefit from the > compression, right? > If it fits into the page cache due to compression, I would consider it as a benefit from compression. I performed further testing on machines with different memory sizes. Both experiments showed that compression was beneficial for execution time. The execution time reduction was more significant in the case of the machine that had less memory available. Tests were performed on: MacBook PRO M3 36GB - MacOs Virtual machine ARM64 10GB/ 6CPU - Fedora 39 > > regards > > -- > Tomas Vondra > > --000000000000a8fe710627a9c8b6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

<= /div>
=

=C2=A0 =C2=A0 -Filip-


st 20. 11. 2024 v=C2=A01:35 odes=C3=ADlatel Tomas V= ondra <tomas@vondra= .me> napsal:
Hi,

On 11/18/24 22:58, Filip Janus wrote:
> ...
>=C2=A0 =C2=A0 =C2=A0Hi all,
>=C2=A0 =C2=A0 =C2=A0Postgresql supports data compression nowadays, but = the compression of
>=C2=A0 =C2=A0 =C2=A0temporary files has not been implemented yet. The h= uge queries can=C2=A0
>=C2=A0 =C2=A0 =C2=A0produce a significant amount of temporary data that= needs to
>=C2=A0 =C2=A0 =C2=A0be=C2=A0stored on disk=C2=A0
>=C2=A0 =C2=A0 =C2=A0and cause many expensive I/O operations.
>=C2=A0 =C2=A0 =C2=A0I am attaching a proposal of the patch to enable te= mporary files
>=C2=A0 =C2=A0 =C2=A0compression for
>=C2=A0 =C2=A0 =C2=A0hashjoins for now. Initially, I've chosen the L= Z4 compression
>=C2=A0 =C2=A0 =C2=A0algorithm. It would
>=C2=A0 =C2=A0 =C2=A0probably make better sense to start with pglz, but = I realized it late.
>

Thanks for the idea & patch. I agree this might be quite useful for
workloads generating a lot of temporary files for stuff like sorts etc.
I think it will be interesting to think about the trade offs, i.e. how
to pick the compression level - at some point the compression ratio
stops improving while paying more and more CPU time. Not sure what the
right choice is, so using default seems fine.

I agree it'd be better to start with pglz, and only then add lz4 etc. Firstly, pglz is simply the built-in compression, supported everywhere.
And it's also simpler to implement, I think.

>=C2=A0 =C2=A0 =C2=A0# Future possible improvements
>=C2=A0 =C2=A0 =C2=A0Reducing the number of memory allocations within th= e dumping and
>=C2=A0 =C2=A0 =C2=A0loading of
>=C2=A0 =C2=A0 =C2=A0the buffer. I have two ideas for solving this probl= em. I would
>=C2=A0 =C2=A0 =C2=A0either add a buffer into
>=C2=A0 =C2=A0 =C2=A0struct BufFile or provide the buffer as an argument= from the caller.
>=C2=A0 =C2=A0 =C2=A0For the sequential=C2=A0
>=C2=A0 =C2=A0 =C2=A0execution, I would prefer the second option.
>

Yes, this would be good. Doing a palloc+pfree for each compression is
going to be expensive, especially because these buffers are going to be
large - likely larger than 8kB. Which means it's not cached in the
memory context, etc.

Adding it to the BufFile is not going to fly, because that doubles the
amount of memory per file. And we already have major issues with hash
joins consuming massive amounts of memory. But at the same time the
buffer is only needed during compression, and there's only one at a
time. So I agree with passing a single buffer as an argument.

>=C2=A0 =C2=A0 =C2=A0# Future plan/open questions
>=C2=A0 =C2=A0 =C2=A0In the future, I would like to add support for pglz= and zstd.
>=C2=A0 =C2=A0 =C2=A0Further, I plan to
>=C2=A0 =C2=A0 =C2=A0extend the support of the temporary file compressio= n also for
>=C2=A0 =C2=A0 =C2=A0sorting, gist index creation, etc.
>
>=C2=A0 =C2=A0 =C2=A0Experimenting with the stream mode of compression a= lgorithms. The
>=C2=A0 =C2=A0 =C2=A0compression=C2=A0
>=C2=A0 =C2=A0 =C2=A0ratio of LZ4 in block mode seems to be satisfying, = but the stream
>=C2=A0 =C2=A0 =C2=A0mode could=C2=A0
>=C2=A0 =C2=A0 =C2=A0produce a better ratio, but it would consume more m= emory due to the
>=C2=A0 =C2=A0 =C2=A0requirement to store
>=C2=A0 =C2=A0 =C2=A0context for LZ4 stream compression.
>

One thing I realized is that this only enables temp file compression for a single place - hash join spill files. AFAIK this is because compressed files don't support random access, and the other places might need that= .

Is that correct? The patch does not explain this anywhere. If that's correct, the patch probably should mention this in a comment for the
'compress' argument added to BufFileCreateTemp(), so that it's = clear
when it's legal to set compress=3Dtrue.
=C2=A0
I will add the description there.
=C2=A0
Which other places might compress= temp files? Surely hash joins are not
the only place that could benefit from this, right?
Yes, you are definitely right. I have=C2=A0chosen the hash joi= ns as a POC because=C2=A0
there are no seeks besides seeks at the= beginning of the buffer.=C2=A0
I have focused on hashjoins, but = there are definitely also other places where
the compression coul= d be used. I want to add support in other places
in the feature.<= br>
=C2=A0
Another thing is testing. If I run regression tests, it won't use
compression at all, because the GUC has "none" by default, right?= But we
need some testing, so how would we do that? One option would be to add a regression test that explicitly sets the GUC and does a hash join, but
that won't work with lz4 (because that may not be enabled).
=C2=A0
Right, it's "none" by default. My opin= ion is that we would like to test
every supported compression met= hod, so I will try to add environment variable as
you recommended= .
=C2=A0

Another option might be to add a PG_TEST_xxx environment variable that
determines compression to use. Something like PG_TEST_USE_UNIX_SOCKETS.
But perhaps there's a simpler way.

>=C2=A0 =C2=A0 =C2=A0# Benchmark
>=C2=A0 =C2=A0 =C2=A0I prepared three different databases to check expec= tations. Each=C2=A0
>=C2=A0 =C2=A0 =C2=A0dataset is described below. My testing demonstrates= that my patch=C2=A0
>=C2=A0 =C2=A0 =C2=A0improves the execution time of huge hash joins.=C2= =A0
>=C2=A0 =C2=A0 =C2=A0Also, my implementation should not
>=C2=A0 =C2=A0 =C2=A0negatively affect performance within smaller querie= s.=C2=A0
>=C2=A0 =C2=A0 =C2=A0The usage of memory needed for temporary files was = reduced in every
>=C2=A0 =C2=A0 =C2=A0=C2=A0execution without a significant impact on exe= cution time.
>
>=C2=A0 =C2=A0 =C2=A0*## Dataset A:*
>=C2=A0 =C2=A0 =C2=A0Tables*
>=C2=A0 =C2=A0 =C2=A0*
>=C2=A0 =C2=A0 =C2=A0table_a(bigint id,text data_text,integer data_numbe= r) - 10000000 rows
>=C2=A0 =C2=A0 =C2=A0table_b(bigint id, integer ref_id, numeric data_val= ue, bytea
>=C2=A0 =C2=A0 =C2=A0data_blob) - 10000000 rows
>=C2=A0 =C2=A0 =C2=A0Query:=C2=A0 SELECT *=C2=A0 FROM table_a a JOIN tab= le_b b ON a.id= <http://
>=C2=A0 =C2=A0 =C2=A0a.id> =3D b.id <http://b.id>;
>
>=C2=A0 =C2=A0 =C2=A0The tables contain highly compressible data.
>=C2=A0 =C2=A0 =C2=A0The query demonstrated a reduction in the usage of = the temporary=C2=A0
>=C2=A0 =C2=A0 =C2=A0files ~20GB -> 3GB, based on this reduction also= caused the execution=C2=A0
>=C2=A0 =C2=A0 =C2=A0time of the query to be reduced by about ~10s.
>
>
>=C2=A0 =C2=A0 =C2=A0*## Dataset B:*
>=C2=A0 =C2=A0 =C2=A0Tables:*
>=C2=A0 =C2=A0 =C2=A0*
>=C2=A0 =C2=A0 =C2=A0table_a(integer id, text data_blob) - 1110000 rows<= br> >=C2=A0 =C2=A0 =C2=A0table_b(integer id, text data_blob) - 10000000 rows=
>=C2=A0 =C2=A0 =C2=A0Query:=C2=A0 SELECT *=C2=A0 FROM table_a a JOIN tab= le_b b ON a.id= <http://
>=C2=A0 =C2=A0 =C2=A0a.id> =3D b.id <http://b.id>;
>
>=C2=A0 =C2=A0 =C2=A0The tables contain less compressible data. data_blo= b was generated
>=C2=A0 =C2=A0 =C2=A0by a pseudo-random generator.
>=C2=A0 =C2=A0 =C2=A0In this case, the data reduction was only ~50%. Als= o, the execution
>=C2=A0 =C2=A0 =C2=A0time was reduced=C2=A0
>=C2=A0 =C2=A0 =C2=A0only slightly with the enabled compression.
>
>=C2=A0 =C2=A0 =C2=A0The second scenario demonstrates no overhead in the= case of enabled=C2=A0
>=C2=A0 =C2=A0 =C2=A0compression and extended work_mem to avoid temp fil= e usage.
>
>=C2=A0 =C2=A0 =C2=A0*## Dataset C:*
>=C2=A0 =C2=A0 =C2=A0Tables
>=C2=A0 =C2=A0 =C2=A0customers (integer,text,text,text,text)
>=C2=A0 =C2=A0 =C2=A0order_items(integer,integer,integer,integer,numeric= (10,2))
>=C2=A0 =C2=A0 =C2=A0orders(integer,integer,timestamp,numeric(10,2))
>=C2=A0 =C2=A0 =C2=A0products(integer,text,text,numeric(10,2),integer) >
>=C2=A0 =C2=A0 =C2=A0Query: SELECT p.product_id, p.name <http://p.name>, p.price,<= br> >=C2=A0 =C2=A0 =C2=A0SUM(oi.quantity) AS total_quantity, AVG(oi.price) A= S avg_item_price
>=C2=A0 =C2=A0 =C2=A0FROM eshop.products p JOIN eshop.order_items oi ON = p.product_id =3D
>=C2=A0 =C2=A0 =C2=A0oi.product_id JOIN=C2=A0
>=C2=A0 =C2=A0 =C2=A0eshop.orders o ON oi.order_id =3D o.order_id WHERE = o.order_date >
>=C2=A0 =C2=A0 =C2=A0'2020-01-01' AND p.price > 50
>=C2=A0 =C2=A0 =C2=A0GROUP BY p.product_id, p.name <http://p.name>, p.price HAVING=
>=C2=A0 =C2=A0 =C2=A0SUM(oi.quantity) > 1000
>=C2=A0 =C2=A0 =C2=A0ORDER BY total_quantity DESC LIMIT 100;
>
>=C2=A0 =C2=A0 =C2=A0This scenario should demonstrate a more realistic u= sage of the database.
>=C2=A0 =C2=A0 =C2=A0Enabled compression slightly reduced the temporary = memory usage, but
>=C2=A0 =C2=A0 =C2=A0the execution
>=C2=A0 =C2=A0 =C2=A0time wasn't affected by compression.
>
>
>=C2=A0 =C2=A0 =C2=A0+------------+-------------------------+-----------= ------------
>=C2=A0 =C2=A0 =C2=A0+------------------------------+
>=C2=A0 =C2=A0 =C2=A0| =C2=A0Dataset =C2=A0 | Compression. =C2=A0 =C2=A0= =C2=A0 | temp_bytes =C2=A0 =C2=A0 =C2=A0=C2=A0=C2=A0 | Execution
>=C2=A0 =C2=A0 =C2=A0Time (ms)=C2=A0=C2=A0 | =C2=A0 =C2=A0 =C2=A0
>=C2=A0 =C2=A0 =C2=A0+------------+-------------------------+-----------= ------------
>=C2=A0 =C2=A0 =C2=A0+----------------------------- +
>=C2=A0 =C2=A0 =C2=A0| A =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | Yes= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 |=C2=A0 3.09 GiB =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0 >=C2=A0 =C2=A0 =C2=A0| 22s586ms=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 | work_mem=C2=A0 =3D 4MB
>=C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 | No=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0=C2=A0 =C2=A0 |=C2=A0 21.89 GiB =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2= =A0
>=C2=A0 =C2=A0 =C2=A0| 35s=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 | work_mem=C2=A0 =3D 4MB
>=C2=A0 =C2=A0 =C2=A0+------------+-------------------------+-----------= ------------
>=C2=A0 =C2=A0 =C2=A0+----------------------------------------
>=C2=A0 =C2=A0 =C2=A0| B=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 | Yes=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0333 MB =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0
>=C2=A0 =C2=A0 =C2=A0| 1815.545 ms=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |= work_mem =3D 4MB
>=C2=A0 =C2=A0 =C2=A0|=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | No=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 146=C2=A0 MB =C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0
>=C2=A0 =C2=A0 =C2=A0=C2=A0 | 1500.460 ms=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 | work_mem =3D 4MB
>=C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0 = =C2=A0=C2=A0 | Yes=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= |=C2=A0 0 MB =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 =C2=A0
>=C2=A0 =C2=A0 =C2=A0=C2=A0 =C2=A0 |=C2=A03262.305 ms=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 | work_mem =3D 80MB
>=C2=A0 =C2=A0 =C2=A0|=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | No=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 0 MB =C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0=C2=A0
>=C2=A0 =C2=A0 =C2=A0=C2=A0=C2=A0 |=C2=A03174.725 ms=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | work_mem =3D 80MB
>=C2=A0 =C2=A0 =C2=A0+-------------+------------------------+-----------= -------------
>=C2=A0 =C2=A0 =C2=A0+-------------------------------------
>=C2=A0 =C2=A0 =C2=A0| C=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = | Yes=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 40 MB=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0
>=C2=A0 =C2=A0 =C2=A0|=C2=A01011.020 ms=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 | work_mem =3D 1MB
>=C2=A0 =C2=A0 =C2=A0|=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | No=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 53
>=C2=A0 =C2=A0 =C2=A0MB=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 1034.142 ms=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | work_mem =3D 1MB
>=C2=A0 =C2=A0 =C2=A0+------------+------------------------+------------= ------------
>=C2=A0 =C2=A0 =C2=A0+--------------------------------------
>
>

Thanks. I'll try to do some benchmarks on my own.

Are these results fro ma single run, or an average of multiple runs?
=C2=A0
It is average from multiple runs.

Do
you maybe have a script to reproduce this, including the data generation?
=C2=A0
I am attaching my SQL file for database pr= eparation. I also did further testing
with two other machines( se= e attachment huge_tables.rtf ).

Also, can you share some information about the machine used for this? I
expect the impact to strongly depends on memory pressure - if the temp
file fits into page cache (and stays there), it may not benefit from the compression, right?
=C2=A0
If it fits=C2=A0i= nto the page cache due to compression, I would consider it as a benefit fro= m compression.
I performed further testing on machines with diffe= rent memory sizes.
Both experiments showed that compression was b= eneficial for execution time.=C2=A0
The execution time reduction = was more significant in the case of the machine that had
less mem= ory available.

Tests were performed on:
= MacBook PRO M3 36GB - MacOs
Virtual machine ARM64 10GB/ 6CPU - Fe= dora 39



regards

--
Tomas Vondra

--000000000000a8fe710627a9c8b6-- --000000000000a8fe760627a9c8b8 Content-Type: text/rtf; charset="US-ASCII"; name="huge_table.rtf" Content-Disposition: attachment; filename="huge_table.rtf" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_m3vpcns60 e1xydGYxXGFuc2lcYW5zaWNwZzEyNTJcY29jb2FydGYyODIwClxjb2NvYXRleHRzY2FsaW5nMFxj b2NvYXBsYXRmb3JtMHtcZm9udHRibFxmMFxmbmlsXGZjaGFyc2V0MCBNZW5sby1SZWd1bGFyO30K e1xjb2xvcnRibDtccmVkMjU1XGdyZWVuMjU1XGJsdWUyNTU7XHJlZDBcZ3JlZW4wXGJsdWUwO30K e1wqXGV4cGFuZGVkY29sb3J0Ymw7O1xjc2dyYXlcYzA7fQpccGFwZXJ3MTE5MDBccGFwZXJoMTY4 NDBcbWFyZ2wxNDQwXG1hcmdyMTQ0MFx2aWV3dzM1ODAwXHZpZXdoMjAzMDBcdmlld2tpbmQwClxw YXJkXHR4NTYwXHR4MTEyMFx0eDE2ODBcdHgyMjQwXHR4MjgwMFx0eDMzNjBcdHgzOTIwXHR4NDQ4 MFx0eDUwNDBcdHg1NjAwXHR4NjE2MFx0eDY3MjBccGFyZGlybmF0dXJhbFxwYXJ0aWdodGVuZmFj dG9yMAoKXGYwXGZzMjIgXGNmMiBcQ29jb2FMaWdhdHVyZTAgTWFjQm9vayBQUk8gTTMgMzZHQiBc ClwnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3 XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XApcCmh1Z2Vf bmV3PSMgc2V0IHRlbXBfZmlsZV9jb21wcmVzc2lvbiA9ICJubyI7XApTRVRcClxwYXJkXHR4NTYw XHR4MTEyMFx0eDE2ODBcdHgyMjQwXHR4MjgwMFx0eDMzNjBcdHgzOTIwXHR4NDQ4MFx0eDUwNDBc dHg1NjAwXHR4NjE2MFx0eDY3MjBccGFyZGlybmF0dXJhbFxwYXJ0aWdodGVuZmFjdG9yMApodWdl X25ldz0jIFNFTEVDVCBkYXRuYW1lLCB0ZW1wX2J5dGVzICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICBGUk9NIHBnX3N0YXRfZGF0YWJhc2U7XAogICAgZGF0bmFtZSAg ICB8ICB0ZW1wX2J5dGVzICBcCi0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLVwKICAgICAg ICAgICAgICAgfCAgICAgICAgICAgIDBcCiBwb3N0Z3JlcyAgICAgIHwgICAgMTU0MDAwMDAwXAog ZmphbnVzICAgICAgICB8ICAgICAgICAgICAgMFwKIHRlbXBsYXRlMSAgICAgfCAgICAgICAgICAg IDBcCiB0ZW1wbGF0ZTAgICAgIHwgICAgICAgICAgICAwXAogaGFzaGpvaW5fdGVzdCB8ICAgICAg ICAgICAgMFwKIGh1Z2VfdGFibGVzICAgfCAxNTc1OTI1ODIxMTNcCiByYW5kb20gICAgICAgIHwg ICAgODkwMTY4NjU4XAogZXNob3AgICAgICAgICB8ICAgICA1NjMzNTIyOVwKIGh1Z2VfbmV3ICAg ICAgfCAgICAgICAgICAgIDBcCigxMCByb3dzKVwKXApodWdlX25ldz0jICBFWFBMQUlOIEFOQUxZ WkUgU0VMRUNUIGEuaWQsIGEuZGF0YSwgYi5kYXRhIEZST00gdGFibGVfYSBhIEpPSU4gdGFibGVf YiBiIE9OIGEuaWQgPSBiLmlkOyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICBRVUVSWSBQTEFOICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIFwKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS1cCiBHYXRoZXIgIChjb3N0PTEzMTg2NjYuMTguLjQzMDk4ODAuNDQgcm93cz0xMDAwMDAwOCB3 aWR0aD04MTYpIChhY3R1YWwgdGltZT0xMTM0ODkuMDUwLi4xNjg4NzEuNDY2IHJvd3M9MTAwMDAw MDAgbG9vcHM9MSlcCiAgIFdvcmtlcnMgUGxhbm5lZDogN1wKICAgV29ya2VycyBMYXVuY2hlZDog N1wKICAgLT4gIEhhc2ggSm9pbiAgKGNvc3Q9MTMxNzY2Ni4xOC4uMzMwODg3OS42NCByb3dzPTE0 Mjg1NzMgd2lkdGg9ODE2KSAoYWN0dWFsIHRpbWU9MTEzNTI4LjU3OS4uMTYxNDcwLjI1NiByb3dz PTEyNTAwMDAgbG9vcHM9OClcCiAgICAgICAgIEhhc2ggQ29uZDogKGIuaWQgPSBhLmlkKVwKICAg ICAgICAgLT4gIFBhcmFsbGVsIFNlcSBTY2FuIG9uIHRhYmxlX2IgYiAgKGNvc3Q9MC4wMC4uMTEz OTY4My40NSByb3dzPTI4NTcxNDUgd2lkdGg9NDEyKSAoYWN0dWFsIHRpbWU9MC42MzguLjk1NDcu NjUzIHJvd3M9MjUwMDAwMCBsb29wcz04KVwKICAgICAgICAgLT4gIEhhc2ggIChjb3N0PTY1NTU1 Ni4wOC4uNjU1NTU2LjA4IHJvd3M9MTAwMDAwMDggd2lkdGg9NDEyKSAoYWN0dWFsIHRpbWU9MTEz MjA2LjM2Ny4uMTEzMjA2LjM2NyByb3dzPTEwMDAwMDAwIGxvb3BzPTgpXAogICAgICAgICAgICAg ICBCdWNrZXRzOiAzMjc2OCAgQmF0Y2hlczogMTAyNCAgTWVtb3J5IFVzYWdlOiA0NDg2a0JcCiAg ICAgICAgICAgICAgIC0+ICBTZXEgU2NhbiBvbiB0YWJsZV9hIGEgIChjb3N0PTAuMDAuLjY1NTU1 Ni4wOCByb3dzPTEwMDAwMDA4IHdpZHRoPTQxMikgKGFjdHVhbCB0aW1lPTAuNTI0Li45MzI2LjI0 NCByb3dzPTEwMDAwMDAwIGxvb3BzPTgpXAogUGxhbm5pbmcgVGltZTogMS43MjAgbXNcCiBFeGVj dXRpb24gVGltZTogMTY5MTgwLjEwNyBtc1wKKDExIHJvd3MpXApcCmh1Z2VfbmV3PSMgU0VMRUNU IGRhdG5hbWUsIHRlbXBfYnl0ZXMgLyAxMDI0IC8gMTAyNCBBUyB0ZW1wX2J5dGVzX21iIEZST00g cGdfc3RhdF9kYXRhYmFzZTtcCiAgICBkYXRuYW1lICAgIHwgdGVtcF9ieXRlc19tYiBcCi0tLS0t LS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS1cCiAgICAgICAgICAgICAgIHwgICAgICAgICAgICAg MFwKIHBvc3RncmVzICAgICAgfCAgICAgICAgICAgMTQ2XAogZmphbnVzICAgICAgICB8ICAgICAg ICAgICAgIDBcCiB0ZW1wbGF0ZTEgICAgIHwgICAgICAgICAgICAgMFwKIHRlbXBsYXRlMCAgICAg fCAgICAgICAgICAgICAwXAogaGFzaGpvaW5fdGVzdCB8ICAgICAgICAgICAgIDBcCiBodWdlX3Rh YmxlcyAgIHwgICAgICAgIDE1MDI5MVwKIHJhbmRvbSAgICAgICAgfCAgICAgICAgICAgODQ4XAog ZXNob3AgICAgICAgICB8ICAgICAgICAgICAgNTNcCiBodWdlX25ldyAgICAgIHwgICAgICAgICA0 MTIzNFwKKDEwIHJvd3MpXApcCmh1Z2VfbmV3PSMgc2V0IHRlbXBfZmlsZV9jb21wcmVzc2lvbiA9 ICJsejQiO1wKU0VUXApodWdlX25ldz0jIFNFTEVDVCBwZ19zdGF0X3Jlc2V0KCk7XAogcGdfc3Rh dF9yZXNldCBcCi0tLS0tLS0tLS0tLS0tLVwKIFwKKDEgcm93KVwKXApodWdlX25ldz0jICBFWFBM QUlOIEFOQUxZWkUgU0VMRUNUIGEuaWQsIGEuZGF0YSwgYi5kYXRhIEZST00gdGFibGVfYSBhIEpP SU4gdGFibGVfYiBiIE9OIGEuaWQgPSBiLmlkOyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBRVUVSWSBQTEFOICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFwKLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS1cCiBHYXRoZXIgIChjb3N0PTEzMTg2NjYuMTguLjQzMDk4ODAuNDQgcm93cz0x MDAwMDAwOCB3aWR0aD04MTYpIChhY3R1YWwgdGltZT05MzYzMi4yNTQuLjEyODYzNS4wODYgcm93 cz0xMDAwMDAwMCBsb29wcz0xKVwKICAgV29ya2VycyBQbGFubmVkOiA3XAogICBXb3JrZXJzIExh dW5jaGVkOiA3XAogICAtPiAgSGFzaCBKb2luICAoY29zdD0xMzE3NjY2LjE4Li4zMzA4ODc5LjY0 IHJvd3M9MTQyODU3MyB3aWR0aD04MTYpIChhY3R1YWwgdGltZT05MzY1Ny43NTAuLjEyMzc1OS45 NDcgcm93cz0xMjUwMDAwIGxvb3BzPTgpXAogICAgICAgICBIYXNoIENvbmQ6IChiLmlkID0gYS5p ZClcCiAgICAgICAgIC0+ICBQYXJhbGxlbCBTZXEgU2NhbiBvbiB0YWJsZV9iIGIgIChjb3N0PTAu MDAuLjExMzk2ODMuNDUgcm93cz0yODU3MTQ1IHdpZHRoPTQxMikgKGFjdHVhbCB0aW1lPTAuNDc4 Li41Mjc4Ljc2NSByb3dzPTI1MDAwMDAgbG9vcHM9OClcCiAgICAgICAgIC0+ICBIYXNoICAoY29z dD02NTU1NTYuMDguLjY1NTU1Ni4wOCByb3dzPTEwMDAwMDA4IHdpZHRoPTQxMikgKGFjdHVhbCB0 aW1lPTkzMzI4Ljk3My4uOTMzMjguOTczIHJvd3M9MTAwMDAwMDAgbG9vcHM9OClcCiAgICAgICAg ICAgICAgIEJ1Y2tldHM6IDMyNzY4ICBCYXRjaGVzOiAxMDI0ICBNZW1vcnkgVXNhZ2U6IDQ0ODZr QlwKICAgICAgICAgICAgICAgLT4gIFNlcSBTY2FuIG9uIHRhYmxlX2EgYSAgKGNvc3Q9MC4wMC4u NjU1NTU2LjA4IHJvd3M9MTAwMDAwMDggd2lkdGg9NDEyKSAoYWN0dWFsIHRpbWU9MC45NTcuLjY2 MDUuMDQ2IHJvd3M9MTAwMDAwMDAgbG9vcHM9OClcCiBQbGFubmluZyBUaW1lOiAwLjgwNyBtc1wK IEV4ZWN1dGlvbiBUaW1lOiAxMjg5ODAuNjg4IG1zXAooMTEgcm93cylcClwKaHVnZV9uZXc9IyBT RUxFQ1QgZGF0bmFtZSwgdGVtcF9ieXRlcyAvIDEwMjQgLyAxMDI0IEFTIHRlbXBfYnl0ZXNfbWIg RlJPTSBwZ19zdGF0X2RhdGFiYXNlO1wKICAgIGRhdG5hbWUgICAgfCB0ZW1wX2J5dGVzX21iIFwK LS0tLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tLS0tLVwKICAgICAgICAgICAgICAgfCAgICAgICAg ICAgICAwXAogcG9zdGdyZXMgICAgICB8ICAgICAgICAgICAxNDZcCiBmamFudXMgICAgICAgIHwg ICAgICAgICAgICAgMFwKIHRlbXBsYXRlMSAgICAgfCAgICAgICAgICAgICAwXAogdGVtcGxhdGUw ICAgICB8ICAgICAgICAgICAgIDBcCiBoYXNoam9pbl90ZXN0IHwgICAgICAgICAgICAgMFwKIGh1 Z2VfdGFibGVzICAgfCAgICAgICAgMTUwMjkxXAogcmFuZG9tICAgICAgICB8ICAgICAgICAgICA4 NDhcCiBlc2hvcCAgICAgICAgIHwgICAgICAgICAgICA1M1wKIGh1Z2VfbmV3ICAgICAgfCAgICAg ICAgICAxOTM3XAooMTAgcm93cylcClwKaHVnZV9uZXc9IyBTRUxFQ1QgcGdfc2l6ZV9wcmV0dHko cGdfZGF0YWJhc2Vfc2l6ZSgnaHVnZV9uZXcnKSk7XAogcGdfc2l6ZV9wcmV0dHkgXAotLS0tLS0t LS0tLS0tLS0tXAogMTMgR0JcCigxIHJvdylcClwKXApcClwKVmlydHVhbCBtYWNoaW5lIEFSTTY0 IDEwR0IvIDZDUFVcClwnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3 XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdc Jzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XCc5N1wnOTdcJzk3XApcClxwYXJkXHR4NTYwXHR4 MTEyMFx0eDE2ODBcdHgyMjQwXHR4MjgwMFx0eDMzNjBcdHgzOTIwXHR4NDQ4MFx0eDUwNDBcdHg1 NjAwXHR4NjE2MFx0eDY3MjBccGFyZGlybmF0dXJhbFxwYXJ0aWdodGVuZmFjdG9yMApodWdlPSMg U0VMRUNUIGRhdG5hbWUsXAogICAgICAgdGVtcF9ieXRlcyAvIDEwMjQgLyAxMDI0IEFTIHRlbXBf Ynl0ZXNfbWJcCkZST00gcGdfc3RhdF9kYXRhYmFzZTtcCiAgICAgZGF0bmFtZSAgICAgIHwgdGVt cF9ieXRlc19tYiBcCi0tLS0tLS0tLS0tLS0tLS0tLSstLS0tLS0tLS0tLS0tLS1cCiAgICAgICAg ICAgICAgICAgIHwgICAgICAgICAgICAgMFwKIHBvc3RncmVzICAgICAgICAgfCAgICAgICAgICAg ICAwXAogdGVtcGxhdGUxICAgICAgICB8ICAgICAgICAgICAgIDBcCiB0ZW1wbGF0ZTAgICAgICAg IHwgICAgICAgICAgICAgMFwKIGh1Z2UgICAgICAgICAgICAgfCAgICAgICAgICAgICAwXAooMTEg cm93cylcClwKaHVnZT0jIHNldCB0ZW1wX2ZpbGVfY29tcHJlc3Npb24gPSAibHo0IjtcClNFVFwK aHVnZT0jIEVYUExBSU4gQU5BTFlaRSBTRUxFQ1QgYS5pZCwgYS5kYXRhLCBiLmRhdGEgRlJPTSB0 YWJsZV9hIGEgSk9JTiB0YWJsZV9iIGIgT04gYS5pZCA9IGIuaWQ7XAogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBRVUVSWSBQ TEFOICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIFwKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS1cCiBHYXRoZXIgIChj b3N0PTEzMTg2NjYuMTguLjQ5NjcyODEuNDAgcm93cz0xMDAwMDAwOCB3aWR0aD04MTYpIChhY3R1 YWwgdGltZT0xNjUxNS4zNjUuLjM1MTU1LjQ1MyByb3dzPTEwMDAwMDAwIGxvb3BzPTEpXAogICBX b3JrZXJzIFBsYW5uZWQ6IDJcCiAgIFdvcmtlcnMgTGF1bmNoZWQ6IDJcCiAgIC0+ICBIYXNoIEpv aW4gIChjb3N0PTEzMTc2NjYuMTguLjM5NjYyODAuNjAgcm93cz00MTY2NjcwIHdpZHRoPTgxNikg KGFjdHVhbCB0aW1lPTE2NDk3LjQ0Mi4uMzEyMTYuNTg2IHJvd3M9MzMzMzMzMyBsb29wcz0zKVwK ICAgICAgICAgSGFzaCBDb25kOiAoYi5pZCA9IGEuaWQpXAogICAgICAgICAtPiAgUGFyYWxsZWwg U2VxIFNjYW4gb24gdGFibGVfYiBiICAoY29zdD0wLjAwLi4xMTk0NDQ1LjQwIHJvd3M9ODMzMzM0 MCB3aWR0aD00MTIpIChhY3R1YWwgdGltZT0wLjg3My4uMzkxNi4zODQgcm93cz02NjY2NjY3IGxv b3BzPTMpXAogICAgICAgICAtPiAgSGFzaCAgKGNvc3Q9NjU1NTU2LjA4Li42NTU1NTYuMDggcm93 cz0xMDAwMDAwOCB3aWR0aD00MTIpIChhY3R1YWwgdGltZT0xNjQ3OS4wMDIuLjE2NDc5LjAwMyBy b3dzPTEwMDAwMDAwIGxvb3BzPTMpXAogICAgICAgICAgICAgICBCdWNrZXRzOiAzMjc2OCAgQmF0 Y2hlczogMTAyNCAgTWVtb3J5IFVzYWdlOiA0NDg2a0JcCiAgICAgICAgICAgICAgIC0+ICBTZXEg U2NhbiBvbiB0YWJsZV9hIGEgIChjb3N0PTAuMDAuLjY1NTU1Ni4wOCByb3dzPTEwMDAwMDA4IHdp ZHRoPTQxMikgKGFjdHVhbCB0aW1lPTAuMzU2Li45NDY2LjA0MyByb3dzPTEwMDAwMDAwIGxvb3Bz PTMpXAogUGxhbm5pbmcgVGltZTogMC44MTQgbXNcCiBFeGVjdXRpb24gVGltZTogMzU0MzEuNzkx IG1zXAooMTEgcm93cylcClwKaHVnZT0jIFNFTEVDVCBkYXRuYW1lLFwKICAgICAgIHRlbXBfYnl0 ZXMgLyAxMDI0IC8gMTAyNCBBUyB0ZW1wX2J5dGVzX21iXApGUk9NIHBnX3N0YXRfZGF0YWJhc2U7 XAogICAgIGRhdG5hbWUgICAgICB8IHRlbXBfYnl0ZXNfbWIgXAotLS0tLS0tLS0tLS0tLS0tLS0r LS0tLS0tLS0tLS0tLS0tXAogICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgIDBcCiBwb3N0 Z3JlcyAgICAgICAgIHwgICAgICAgICAgICAgMFwKIHRlbXBsYXRlMSAgICAgICAgfCAgICAgICAg ICAgICAwXAogdGVtcGxhdGUwICAgICAgICB8ICAgICAgICAgICAgIDBcCiBmamFudXMgICAgICAg ICAgIHwgICAgICAgICAgICAgMFwKIHRlc3QgICAgICAgICAgICAgfCAgICAgICAgICAgICAwXAog ZDIyICAgICAgICAgICAgICB8ICAgICAgICAgICAgIDBcCiBoYXNoam9pbl90ZXN0ICAgIHwgICAg ICAgICAgICAxNlwKIGNvbXByZXNzaW9uX3Rlc3QgfCAgICAgICAgICAgMTk2XAogY21wdGVzdCAg ICAgICAgICB8ICAgICAgICAgICAgIDFcCiBodWdlICAgICAgICAgICAgIHwgICAgICAgICAgIDky MVwKKDExIHJvd3MpXApcCmh1Z2U9IyBzZXQgdGVtcF9maWxlX2NvbXByZXNzaW9uID0gIm5vIjtc ClNFVFwKaHVnZT0jIFNFTEVDVCBwZ19zdGF0X3Jlc2V0KCk7XAogcGdfc3RhdF9yZXNldCBcCi0t LS0tLS0tLS0tLS0tLVwKIFwKKDEgcm93KVwKXApodWdlPSMgRVhQTEFJTiBBTkFMWVpFIFNFTEVD VCBhLmlkLCBhLmRhdGEsIGIuZGF0YSBGUk9NIHRhYmxlX2EgYSBKT0lOIHRhYmxlX2IgYiBPTiBh LmlkID0gYi5pZDtcCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBRVUVSWSBQTEFOICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFwKLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tXAogR2F0aGVyICAoY29zdD0xMzE4NjY2LjE4Li40OTY3MjgxLjQw IHJvd3M9MTAwMDAwMDggd2lkdGg9ODE2KSAoYWN0dWFsIHRpbWU9MjM1NTUuMDQ2Li42OTgzOC41 ODkgcm93cz0xMDAwMDAwMCBsb29wcz0xKVwKICAgV29ya2VycyBQbGFubmVkOiAyXAogICBXb3Jr ZXJzIExhdW5jaGVkOiAyXAogICAtPiAgSGFzaCBKb2luICAoY29zdD0xMzE3NjY2LjE4Li4zOTY2 MjgwLjYwIHJvd3M9NDE2NjY3MCB3aWR0aD04MTYpIChhY3R1YWwgdGltZT0yMzU0Mi4yNTMuLjU5 OTgxLjA0NCByb3dzPTMzMzMzMzMgbG9vcHM9MylcCiAgICAgICAgIEhhc2ggQ29uZDogKGIuaWQg PSBhLmlkKVwKICAgICAgICAgLT4gIFBhcmFsbGVsIFNlcSBTY2FuIG9uIHRhYmxlX2IgYiAgKGNv c3Q9MC4wMC4uMTE5NDQ0NS40MCByb3dzPTgzMzMzNDAgd2lkdGg9NDEyKSAoYWN0dWFsIHRpbWU9 MC4zOTcuLjEzMjgzLjAzOSByb3dzPTY2NjY2NjcgbG9vcHM9MylcCiAgICAgICAgIC0+ICBIYXNo ICAoY29zdD02NTU1NTYuMDguLjY1NTU1Ni4wOCByb3dzPTEwMDAwMDA4IHdpZHRoPTQxMikgKGFj dHVhbCB0aW1lPTIzNTIwLjQ5OS4uMjM1MjAuNTAwIHJvd3M9MTAwMDAwMDAgbG9vcHM9MylcCiAg ICAgICAgICAgICAgIEJ1Y2tldHM6IDMyNzY4ICBCYXRjaGVzOiAxMDI0ICBNZW1vcnkgVXNhZ2U6 IDQ0ODZrQlwKICAgICAgICAgICAgICAgLT4gIFNlcSBTY2FuIG9uIHRhYmxlX2EgYSAgKGNvc3Q9 MC4wMC4uNjU1NTU2LjA4IHJvd3M9MTAwMDAwMDggd2lkdGg9NDEyKSAoYWN0dWFsIHRpbWU9MC4z NzguLjE1Mzc1Ljg5MSByb3dzPTEwMDAwMDAwIGxvb3BzPTMpXAogUGxhbm5pbmcgVGltZTogMS4y MzEgbXNcCiBFeGVjdXRpb24gVGltZTogNzAxNDAuMDAzIG1zXAooMTEgcm93cylcClwKaHVnZT0j IFNFTEVDVCBkYXRuYW1lLFwKICAgICAgIHRlbXBfYnl0ZXMgLyAxMDI0IC8gMTAyNCBBUyB0ZW1w X2J5dGVzX21iXApGUk9NIHBnX3N0YXRfZGF0YWJhc2U7XAogICAgIGRhdG5hbWUgICAgICB8IHRl bXBfYnl0ZXNfbWIgXAotLS0tLS0tLS0tLS0tLS0tLS0rLS0tLS0tLS0tLS0tLS0tXAogICAgICAg ICAgICAgICAgICB8ICAgICAgICAgICAgIDBcCiBwb3N0Z3JlcyAgICAgICAgIHwgICAgICAgICAg ICAgMFwKIHRlbXBsYXRlMSAgICAgICAgfCAgICAgICAgICAgICAwXAogdGVtcGxhdGUwICAgICAg ICB8ICAgICAgICAgICAgIDBcCiBmamFudXMgICAgICAgICAgIHwgICAgICAgICAgICAgMFwKIHRl c3QgICAgICAgICAgICAgfCAgICAgICAgICAgICAwXAogZDIyICAgICAgICAgICAgICB8ICAgICAg ICAgICAgIDBcCiBoYXNoam9pbl90ZXN0ICAgIHwgICAgICAgICAgICAxNlwKIGNvbXByZXNzaW9u X3Rlc3QgfCAgICAgICAgICAgMTk2XAogY21wdGVzdCAgICAgICAgICB8ICAgICAgICAgICAgIDFc CiBodWdlICAgICAgICAgICAgIHwgICAgICAgICAyMDY1NVwKKDExIHJvd3MpXApcCmh1Z2U9IyBT RUxFQ1QgcGdfc2l6ZV9wcmV0dHkocGdfZGF0YWJhc2Vfc2l6ZSgnaHVnZScpKTtcCiBwZ19zaXpl X3ByZXR0eSBcCi0tLS0tLS0tLS0tLS0tLS1cCiAxMyBHQlwKKDEgcm93KVwKXHBhcmRcdHg1NjBc dHgxMTIwXHR4MTY4MFx0eDIyNDBcdHgyODAwXHR4MzM2MFx0eDM5MjBcdHg0NDgwXHR4NTA0MFx0 eDU2MDBcdHg2MTYwXHR4NjcyMFxwYXJkaXJuYXR1cmFsXHBhcnRpZ2h0ZW5mYWN0b3IwClwKXApc ClwKfQ== --000000000000a8fe760627a9c8b8 Content-Type: application/octet-stream; name="lz4.sql" Content-Disposition: attachment; filename="lz4.sql" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_m3vpcvvo1 U0VUIGVuYWJsZV9wYXJhbGxlbF9oYXNoID0gT0ZGOwpTRVQgZW5hYmxlX21lcmdlam9pbiA9IE9G RjsKU0VUIGVuYWJsZV9uZXN0bG9vcCBUTyBvZmY7CgpjcmVhdGUgZGF0YWJhc2UgaHVnZV90YWJs ZXM7ClxjIGh1Z2VfdGFibGVzIApDUkVBVEUgVEFCTEUgdGFibGVfYSAoaWQgQklHSU5UIFBSSU1B UlkgS0VZLGRhdGEgVEVYVCk7CkNSRUFURSBUQUJMRSB0YWJsZV9iICAoaWQgQklHSU5UIFBSSU1B UlkgS0VZLCBkYXRhIFRFWFQpOwpJTlNFUlQgSU5UTyB0YWJsZV9hIChpZCwgZGF0YSkgU0VMRUNU IGksIHJlcGVhdCgnQUJDRCcsIDEwMCkgIC0tICJBQkNEQUJDRC4uLiIgcmVwZWF0ZWQgMTAwIHRp bWVzCkZST00gZ2VuZXJhdGVfc2VyaWVzKDEsIDEwMDAwMDAwKSBBUyBpOwotLSBJbnNlcnQgYSBz aW1pbGFyIHJlcGVhdGVkIHBhdHRlcm4gaW50byB0YWJsZV9iCklOU0VSVCBJTlRPIHRhYmxlX2Ig KGlkLCBkYXRhKSBTRUxFQ1QgaSwgcmVwZWF0KCdFRkdIJywgMTAwKSAgLS0gIkVGR0hFRkdILi4u IiByZXBlYXRlZCAxMDAgdGltZXMKRlJPTSBnZW5lcmF0ZV9zZXJpZXMoMSwgMTAwMDAwMDApIEFT IGk7Ci0tIEluc2VydCBvdmVybGFwcGluZyBkYXRhIGluIHRhYmxlX2IgCklOU0VSVCBJTlRPIHRh YmxlX2IgKGlkLCBkYXRhKSBTRUxFQ1QgaSwgcmVwZWF0KCdBQkNEJywgMTAwKSAgLS0gIkFCQ0RB QkNELi4uIiByZXBlYXRlZCAxMDAgdGltZXMKRlJPTSBnZW5lcmF0ZV9zZXJpZXMoMTAwMDAwMDEs IDIwMDAwMDAwKSBBUyBpOwoKc2V0IHRlbXBfZmlsZV9jb21wcmVzc2lvbiA9ICJubyI7ClNFTEVD VCBkYXRuYW1lLCB0ZW1wX2J5dGVzIC8gMTAyNCAvIDEwMjQgQVMgdGVtcF9ieXRlc19tYiBGUk9N IHBnX3N0YXRfZGF0YWJhc2U7ClNFTEVDVCBwZ19zdGF0X3Jlc2V0KCk7CgpzZXQgdGVtcF9maWxl X2NvbXByZXNzaW9uID0gImx6NCI7CkVYUExBSU4gQU5BTFlaRSBTRUxFQ1QgYS5pZCwgYS5kYXRh LCBiLmRhdGEgRlJPTSB0YWJsZV9hIGEgSk9JTiB0YWJsZV9iIGIgT04gYS5pZCA9IGIuaWQ7CgpT RUxFQ1QgZGF0bmFtZSwgdGVtcF9ieXRlcyAvIDEwMjQgLyAxMDI0IEFTIHRlbXBfYnl0ZXNfbWIg RlJPTSBwZ19zdGF0X2RhdGFiYXNlOwpTRUxFQ1QgcGdfc3RhdF9yZXNldCgpOwoKY3JlYXRlIGRh dGFiYXNlIHJhbmRvbTsKXGMgcmFuZG9tOwpDUkVBVEUgVEFCTEUgdGFibGVfYSAoaWQgU0VSSUFM IFBSSU1BUlkgS0VZLCBkYXRhX2Jsb2IgVEVYVCk7CkNSRUFURSBUQUJMRSB0YWJsZV9iIChpZCBT RVJJQUwgUFJJTUFSWSBLRVksZGF0YV9ibG9iIFRFWFQpOwpJTlNFUlQgSU5UTyB0YWJsZV9hIChk YXRhX2Jsb2IpIFNFTEVDVCBtZDUocmFuZG9tKCk6OnRleHQgfHwgY2xvY2tfdGltZXN0YW1wKCk6 OnRleHQpIEZST00gZ2VuZXJhdGVfc2VyaWVzKDEsIDEwMDAwMCk7CklOU0VSVCBJTlRPIHRhYmxl X2EgKGRhdGFfYmxvYikgU0VMRUNUIG1kNShyYW5kb20oKTo6dGV4dCB8fCBjbG9ja190aW1lc3Rh bXAoKTo6dGV4dCkgRlJPTSBnZW5lcmF0ZV9zZXJpZXMoMSwgMTAwMDApOwoKCi0tIEVYUExBSU4g QU5BTFlaRSBTRUxFQ1QgYS5pZCwgYi5pZCBGUk9NIHRhYmxlX2EgYSBKT0lOIHRhYmxlX2IgYiBP TiBhLmlkID0gYi5pZDsKCg== --000000000000a8fe760627a9c8b8--