Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rAApN-005ASc-AD for pgsql-jdbc-commits@arkaria.postgresql.org; Mon, 04 Dec 2023 15:25:45 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rAApM-00HXj9-2J for pgsql-jdbc-commits@arkaria.postgresql.org; Mon, 04 Dec 2023 15:25:44 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rAApL-00HXj0-Pc for pgsql-jdbc-commits@lists.postgresql.org; Mon, 04 Dec 2023 15:25:43 +0000 Received: from out-21.smtp.github.com ([192.30.252.204]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rAApI-00A4RI-Kj for pgsql-jdbc-commits@lists.postgresql.org; Mon, 04 Dec 2023 15:25:43 +0000 Received: from github.com (hubbernetes-node-398b48c.ac4-iad.github.net [10.52.133.33]) by smtp.github.com (Postfix) with ESMTPA id A4CB67005ED for ; Mon, 4 Dec 2023 07:25:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=github.com; s=pf2023; t=1701703538; bh=yJaGhHoxDIJ7NWAHJta4QAyMam5SkI/y0QDR3dItiu8=; h=Date:From:To:Subject:From; b=glqrZhr5s56RoticggPPKnd36+SZDp9LXwUJEf6AKmu8X/yDV0Hdjvq3fasQhEKTh Zbe+IeJtz6ILBRdklQotQ/bXEFl7div2W/lXts/UdY4BrNolJiZBoItnFScowpDOT3 CqHOtozQXbkgQIYiZf3LJ9qSXbBekZRfDUH3Tl0g= Date: Mon, 04 Dec 2023 07:25:38 -0800 From: Vladimir Sitnikov To: pgsql-jdbc-commits@lists.postgresql.org Message-ID: Subject: [pgjdbc/pgjdbc] 7ee6a3: chore: fix test -> generateKar dependency Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Approved: 58GruLoMopuc X-GitHub-Recipient-Address: pgsql-jdbc-commits@lists.postgresql.org X-Auto-Response-Suppress: All List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Branch: refs/heads/master Home: https://github.com/pgjdbc/pgjdbc Commit: 7ee6a373de3ec324a3e8f691f77be9400f571f57 https://github.com/pgjdbc/pgjdbc/commit/7ee6a373de3ec324a3e8f691f77be9400f571f57 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M pgjdbc/build.gradle.kts Log Message: ----------- chore: fix test -> generateKar dependency Commit: b4b28bf7ea29ded5bdd68d2beed4b67f9450f84c https://github.com/pgjdbc/pgjdbc/commit/b4b28bf7ea29ded5bdd68d2beed4b67f9450f84c Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M benchmarks/build.gradle.kts M benchmarks/src/jmh/java/org/postgresql/util/ConnectionUtil.java M pgjdbc/build.gradle.kts R pgjdbc/src/test/java/org/postgresql/test/TestUtil.java A pgjdbc/src/testFixtures/java/org/postgresql/test/TestUtil.java Log Message: ----------- chore: add src/testFixtures to share test-related code across subprojects Previously TestUtil.java was declared in src/test/..., and there was no way to reuse it in the benchmarks. Commit: 3dea8576e0005f29f69f9379b86f6098de74cb75 https://github.com/pgjdbc/pgjdbc/commit/3dea8576e0005f29f69f9379b86f6098de74cb75 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: R pgjdbc/src/test/java/org/postgresql/test/util/StrangeInputStream.java A pgjdbc/src/testFixtures/java/org/postgresql/test/util/StrangeInputStream.java A pgjdbc/src/testFixtures/java/org/postgresql/test/util/StrangeOutputStream.java Log Message: ----------- test: improve StrangeInputStream, add StrangeOutputStream Commit: 306e03ec13f82c18f38a1f12941614bbafddb448 https://github.com/pgjdbc/pgjdbc/commit/306e03ec13f82c18f38a1f12941614bbafddb448 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: A benchmarks/src/jmh/java/org/postgresql/benchmark/largeobject/LargeObjectRead.java A benchmarks/src/jmh/java/org/postgresql/benchmark/largeobject/LargeObjectWrite.java A benchmarks/src/jmh/java/org/postgresql/benchmark/statement/BlobWrite.java A pgjdbc/src/testFixtures/java/org/postgresql/test/util/LargeObjectVacuum.java Log Message: ----------- perf: add large object read/write benchmarks Commit: d2484424c416ea105b0ab5e1f97d0da7aaade346 https://github.com/pgjdbc/pgjdbc/commit/d2484424c416ea105b0ab5e1f97d0da7aaade346 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M pgjdbc/src/main/java/org/postgresql/jdbc/PgPreparedStatement.java Log Message: ----------- perf: allocate min(blob.length, 8192) buffer in PgPreparedStatement.createBlob Commit: ac57b5202e07d0abc3e03ece711c1cada62f0ed3 https://github.com/pgjdbc/pgjdbc/commit/ac57b5202e07d0abc3e03ece711c1cada62f0ed3 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M pgjdbc/src/main/java/org/postgresql/largeobject/BlobInputStream.java M pgjdbc/src/main/java/org/postgresql/largeobject/LargeObject.java M pgjdbc/src/test/java/org/postgresql/test/jdbc2/BlobTest.java M pgjdbc/src/test/java/org/postgresql/test/jdbc4/BlobTest.java Log Message: ----------- perf: improve performance of BlobInputStream.read(b, off, len) with buffering reads This reverts commit 3af3b32cc5b77db3e7af1cbc217d6288fd0cf9b9. Co-authored-by: Dave Cramer Co-authored-by: Brett Okken There are several fixes on top of 3af3b32cc5b77db3e7af1cbc217d6288fd0cf9b9: 1) The buffer is created lazily, and it is created only in case the number of requested bytes is less than the buffer size 2) "small" read(byte[], int, int) can be buffered as well (e.g. if the buffer size exceeds the requested len) 3) Added locks to read calls for consistency with the locking in mark/reset 4) read(byte[], int, int) should return 0 when len=0 5) refilling the bufer should read "bufferSize" amount rather than "the remaining length" Commit: 46aee2c873dfd41031ce709b827f1cbabaadef57 https://github.com/pgjdbc/pgjdbc/commit/46aee2c873dfd41031ce709b827f1cbabaadef57 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M pgjdbc/src/main/java/org/postgresql/largeobject/BlobInputStream.java M pgjdbc/src/main/java/org/postgresql/largeobject/LargeObject.java Log Message: ----------- perf: add buffer scaling to BlobInputStream We start with 65K buffer, and increase it as the user requests more reads. It reaches reasonable performance for "full stream readouts", and it avoids loading large amount of data when the user needs first bytes only. Commit: b419750929ae5e4b6f8b4db2b03acace49d152ce https://github.com/pgjdbc/pgjdbc/commit/b419750929ae5e4b6f8b4db2b03acace49d152ce Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M pgjdbc/src/main/java/org/postgresql/fastpath/FastpathArg.java M pgjdbc/src/main/java/org/postgresql/largeobject/LargeObject.java A pgjdbc/src/main/java/org/postgresql/util/ByteBuffersByteStreamWriter.java M pgjdbc/src/main/java/org/postgresql/util/ByteStreamWriter.java Log Message: ----------- feat: add LargeObject.write(ByteStreamWriter) so multiple buffers can be written at once without copying Commit: b97031ecf29ce4e80aadcf80fe3e262aa4422423 https://github.com/pgjdbc/pgjdbc/commit/b97031ecf29ce4e80aadcf80fe3e262aa4422423 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M pgjdbc/src/main/java/org/postgresql/util/ByteBufferByteStreamWriter.java M pgjdbc/src/main/java/org/postgresql/util/ByteBuffersByteStreamWriter.java M pgjdbc/src/test/java/org/postgresql/test/util/ByteStreamWriterTest.java Log Message: ----------- perf: reduce array allocation in ByteStreamWriter.writeTo(ByteStreamTarget) Previously, the implementation always created a Channel with Channels.newChannel(outputStream), however, OpenJDK (e.g. 22) still allocates arrays. The improvement is to access the arrays in case ByteBuffer allows so. DirectByteBuffer and read-only ByteBuffers forbid array access. Commit: bbcbc553398eba6fe5d08f7b871aa594622a5e70 https://github.com/pgjdbc/pgjdbc/commit/bbcbc553398eba6fe5d08f7b871aa594622a5e70 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M pgjdbc/src/main/java/org/postgresql/largeobject/BlobOutputStream.java Log Message: ----------- perf: optimize BlobOutputStream.write by buffering small consequent writes, and allocating the buffer lazily Previously, each each blobOutput.write(byte[], ..) resulted in database call even in the case the write size is small. It could impact cases when users write in small chunks. At the same time, if they already write with big chunks, we should not create a buffer in BlobOutputStream, so now we create the buffer lazily. Commit: e10616805976c1c8050ca783bd2282827dfc1815 https://github.com/pgjdbc/pgjdbc/commit/e10616805976c1c8050ca783bd2282827dfc1815 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M pgjdbc/src/main/java/org/postgresql/largeobject/BlobOutputStream.java M pgjdbc/src/test/java/org/postgresql/jdbc/LargeObjectManagerTest.java Log Message: ----------- perf: rework BlobOutputStream.write so it uses gathering write and aligns writes Commit: f7d82b8ee07d3435588c76e257a9479878a6d5a6 https://github.com/pgjdbc/pgjdbc/commit/f7d82b8ee07d3435588c76e257a9479878a6d5a6 Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M pgjdbc/src/main/java/org/postgresql/largeobject/BlobOutputStream.java M pgjdbc/src/main/java/org/postgresql/largeobject/LargeObject.java Log Message: ----------- perf: allow BlobOutputStream to resize its buffer up to 512KiB It enables to cover variety of incoming write requests, and it does not incur much overhead as the buffer grows lazily. Commit: e0277f1867a6067cb2c3f37dbfc1b36484c44bac https://github.com/pgjdbc/pgjdbc/commit/e0277f1867a6067cb2c3f37dbfc1b36484c44bac Author: Vladimir Sitnikov Date: 2023-12-04 (Mon, 04 Dec 2023) Changed paths: M docs/content/documentation/binary-data.md Log Message: ----------- doc: update documentation samples for accessing binary data We suggest using Blob APIs as they are standard. Compare: https://github.com/pgjdbc/pgjdbc/compare/22e68522a2c6...e0277f1867a6