Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1aL9XR-0008NV-K6 for pgsql-sql@arkaria.postgresql.org; Mon, 18 Jan 2016 13:08:05 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84) (envelope-from ) id 1aL9XR-00058U-2L for pgsql-sql@arkaria.postgresql.org; Mon, 18 Jan 2016 13:08:05 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84) (envelope-from ) id 1aL9WR-00043B-AY for pgsql-sql@postgresql.org; Mon, 18 Jan 2016 13:07:03 +0000 Received: from post.visena.com ([46.226.10.50]) by magus.postgresql.org with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84) (envelope-from ) id 1aL9WN-00089k-SS for pgsql-sql@postgresql.org; Mon, 18 Jan 2016 13:07:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=visena.com; s=20141101.wh; h=Content-Type:MIME-Version:Subject:In-Reply-To:Message-ID:To:From:Date; bh=BfgMYFYPMf//pvqUchtbFBjo161pmeXen8pe/hLL+gk=; b=ZFTsmWvdKhcNpDyo7+t+2d87OUDStS750qT6nr2NFGeKLv+f+cZlWF3MH+VSMXpGpoYS5n7RBFSbRAtAC8zlOMsw1+EW3bt5LKtpXMM/Aj/2griXS7AAbwnrYULOK3FVlz9sJ/w3/IZ/mwtMWBtY3601OSDnSGGKgEyq1SZOL+M=; Received: from [10.0.1.10] (helo=tc7-visena.wh.internal.visena.com) by post.visena.com with esmtp (Exim 4.82) (envelope-from ) id 1aL9WJ-0004rf-Kb for pgsql-sql@postgresql.org; Mon, 18 Jan 2016 14:06:58 +0100 Received: from localhost ([127.0.0.1] helo=tc7-visena.wh.internal.visena.com) by tc7-visena.wh.internal.visena.com with esmtp (Exim 4.82) (envelope-from ) id 1aL9WJ-0001fG-Jl for pgsql-sql@postgresql.org; Mon, 18 Jan 2016 14:06:55 +0100 Date: Mon, 18 Jan 2016 14:06:55 +0100 (CET) From: Andreas Joseph Krogh To: pgsql-sql@postgresql.org Message-ID: In-Reply-To: Subject: Re: BYTEA MIME-Version: 1.0 X-Mailer: Visena Mail 2.0.0-SNAPSHOT X-Spam-Score: 0.6 X-Spam-Report: SpamAssasin (score=0.6, required 5.0 ALL_TRUSTED=-1, HTML_MESSAGE=0.001, SUBJ_ALL_CAPS=1.625) X-Pg-Spam-Score: -0.5 (/) Content-Type: multipart/related; boundary="----=_Part_846_578407284.1453122415557" List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-sql Precedence: bulk Sender: pgsql-sql-owner@postgresql.org ------=_Part_845_227234319.1453122415557 Content-Type: multipart/related; boundary="----=_Part_846_578407284.1453122415557" ------=_Part_846_578407284.1453122415557 Content-Type: multipart/alternative; boundary="----=_Part_847_1735596724.1453122415568" ------=_Part_847_1735596724.1453122415568 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable P=C3=A5 s=C3=B8ndag 17. januar 2016 kl. 23:13:09, skrev Thomas Kellerer < spam_eater@gmx.net >: Andreas Joseph Krogh schrieb am 17.01.2016 um 23:09: >=C2=A0 =C2=A0 =C2=A0 > Do I=C2=A0 really *Need to escape/encode binary da= ta before sending to DB >=C2=A0 =C2=A0 =C2=A0 > then do the reverse after retrieving the data?* >=C2=A0 =C2=A0 =C2=A0 > * >=C2=A0 =C2=A0 =C2=A0 > * >=C2=A0 =C2=A0 =C2=A0 > If so, what (*Java*) codes should I use to achieve= this goal (I am=20 using >=C2=A0 =C2=A0 =C2=A0 > the Java to interface with the DB)? > >=C2=A0 =C2=A0 =C2=A0https://jdbc.postgresql.org/documentation/94/binary-d= ata.html > > Save yourself the trouble and don't go this route. Use=20 https://github.com/impossibl/pgjdbc-ng instead. Can you elaborate? Using the "official" JDBC driver with bytea column works just fine for me. =C2=A0 Depends on what "works" is. Using BLOBs (that is SQL-BLOB, not *ps.setBinaryStream etc.) with=20 ps.setBlob/rs.getBlob and Connection.createBlob certainly doesn't work usin= g=20 the official driver.=C2=A0 https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postg= resql/jdbc/PgConnection.java#L1284-L1287 =C2=A0 public Blob createBlob() throws SQLException { checkClosed(); throw=20 org.postgresql.Driver.notImplemented(this.getClass(), "createBlob()"); }=20 =C2=A0 AFAIU this thread is about working with LARGE OBJECTS, not only binary data= . Also, using BYTEA with LARGE objects (not just binary data) quickly leads = to=20 OutOfMemoryError. Which is why I recommend using pgjdbc-ng and real BLOBs= =20 (using OID) instead. It is true that get/setBinaryStream "works", in essenc= e=20 that it appears to do the jobb. The problem is that despite using=20 get/setBinaryStream with BYTEAappears to use streams, it doesn't, and the w= hole=20 byte-array is kept in memory, both in the JAVA-appand in PG. The only way t= o=20 work with real streams all the way is using OID, not BYTEA. =C2=A0 But, of course, if you only work with small-ish binary data, yes - BYTEA do= es=20 the job. =C2=A0 -- Andreas Joseph Krogh CTO / Partner - Visena AS Mobile: +47 909 56 963 andreas@visena.com www.visena.com =C2=A0 ------=_Part_847_1735596724.1453122415568 Content-Type: text/html;charset=UTF-8 Content-Transfer-Encoding: quoted-printable
P=C3=A5 s=C3=B8ndag 17. januar 2016 kl. 23:13:09, skrev Thomas Kellere= r <spam_eater@gmx.net>:
And= reas Joseph Krogh schrieb am 17.01.2016 um 23:09:
>=C2=A0 =C2=A0 =C2=A0 > Do I=C2=A0 really *Need to escape/encode bina= ry data before sending to DB
>=C2=A0 =C2=A0 =C2=A0 > then do the reverse after retrieving the data= ?*
>=C2=A0 =C2=A0 =C2=A0 > *
>=C2=A0 =C2=A0 =C2=A0 > *
>=C2=A0 =C2=A0 =C2=A0 > If so, what (*Java*) codes should I use to ac= hieve this goal (I am using
>=C2=A0 =C2=A0 =C2=A0 > the Java to interface with the DB)?
>
>=C2=A0 =C2=A0 =C2=A0https://jdbc.postgresql.org/documentation/94/binary= -data.html
>
> Save yourself the trouble and don't go this route. Use https://github.= com/impossibl/pgjdbc-ng instead.

Can you elaborate?

Using the "official" JDBC driver with bytea column works just fin= e for me.
=C2=A0
Depends on what "works" is.
Using BLOBs (that is SQL-BLOB, not *ps.setBinaryStream etc.) with ps.s= etBlob/rs.getBlob and Connection.createBlob certainly doesn't work using th= e official driver.
=C2=A0
https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/= postgresql/jdbc/PgConnection.java#L1284-L1287
=C2=A0
  public Blob createBlob() throws SQLException {
    checkClosed();
    throw org.postgresql.Driver.notImplemented(this.getClass(), "creat=
eBlob()");
  }
=C2=A0
AFAIU this thread is about working with LARGE OBJECTS, not only binary= data.
Also, using BYTEA with LARGE objects (not just binary data) quickly leads t= o OutOfMemoryError. Which is why I recommend using pgjdbc-ng and real BLOBs= (using OID) instead. It is true that get/setBinaryStream "works"= , in essence that it appears to do the jobb. The problem is that despite us= ing get/setBinaryStream with BYTEA appears to use streams, it does= n't, and the whole byte-array is kept in memory, both in the JAVA-app a= nd in PG. The only way to work with real streams all the way is using = OID, not BYTEA.
=C2=A0
But, of course, if you only work with small-ish binary data, yes - BYT= EA does the job.
=C2=A0
--
Andrea= s Joseph Krogh
CTO / Partner<= /span> - Visena AS
Mobile: +47 90= 9 56 963
=3D""
=C2=A0
------=_Part_847_1735596724.1453122415568-- ------=_Part_846_578407284.1453122415557 Content-Type: image/png Content-Transfer-Encoding: base64 Content-Disposition: inline Content-ID: iVBORw0KGgoAAAANSUhEUgAAAIUAAAAYCAYAAADUIj6hAAAABHNCSVQICAgIfAhkiAAABzBJREFU aEPtmNFxHDcMhmVP3i1VECpvnjzkVIHWFfhcgVcVRKrAUgWRK/C6Al8H3lTgy0PGbzFdQc4VJP/H ADs43q6kROeJNbOYgQACIAgCWJKng4MZ5gxUGXh0U0b+SIuF9G+EWXj2Q15vbrE/NPtk9uub7Gfd t5mBx1NhqSEo8HshjbEUtlO2yIM9tsx5dZP9rPt2MzDZFAqZE4LGcFjdsg3saQaHt7fY70X949On aS+OZidDBkavD33157L4xaw2os+Mp/DAC10l2XhOCeStj0W5ajrJL8WfCi80Xgf9vVlrhg9yRONe /f7xI2vNsIcM7JwU9o6oGyJrLT8JOA1aX9sKP4wl94bAniukEbo/n7YPmuTETzIab4Y9ZWCrKexd 8M58lxPCvvD6aihfvexbEQoPYO8NQRO0JnddGO6FJYaVsBde7cXj7KRk4LsqDxQzCYeGsJNgaXZe +JXkyGgWINq3Gp+bHNILz+wEYs5qH1eJrouNrpDXtk4O6x1IvtD40GWyJYYdkF0jIbYZlN16xygI gl9smbMDsmFdfAKb23xGB8H/mv3tOJfArs1kukk79DEPdQ5u0g1vCvvqKXIW8mZYW+F3Tg4r8HvZ kQCCLydK8EFMQCe5N8RgL9mRG0SqQFuNvdE6beSs0qPDBuCdg0+gvClso8SbTO4ki7mQzQqBrcMH QPwReg2wW8umEe/+L8T/LEzBGF9nXjzZ46s+ITHvhegW8LL39xm6App7KYL/GE+vcYnFbJiP/0aY hdiCnRC7jaj7eiK2ETInC5PRF6LYkSN0+HabF75WuT6syCyI0YkVGGMvUC33AmfZTDUEj8u6IVhu EhRUJ2U2g6UlugyNb03Hl9obHwlxJbcRzcYjO4QPjVfGFTQav4vrmp7cpMp2qbHnBxWJbisbho2Q XI6C1sIHDUFhH4Hij4UbIev66eANeiwbkA+LBsO36zAHzoXU7AhbqLAXEiOYTXcSdO8VS9L4wN8U BIYhBd6oSUgYMijOkecRuTdQa/YiBXhbXFcnCnI2SrfeBG9NydrLYMhGHa5qB9pQIxlzAE6Okjzx JO5afGe6kmhBicWKQNKuTZ5EW+MjQY8v/9rQ0bhJSJyNGUe/JH1t8h1iMbdSPAvxHYin6YmN9YBX QmTYZZNh14vHhhhal5vtcIrJjmvszPRJdExH3MWHvykoBEc9CoCGWJisOLOGoCORs1FvIMbYA8z3 kwM59oemy6LlWrLxFLmWgiQAfEGd8S+NssbK+EhyGLxUkhhiy717waBqHOJYSEacwBejkFNhjJNj v/gANOdQxPfciP/JVBCK2cOIcg3RRJ+CPrLPNVhhN6F38VKMF3XLlIJrjU5C8gMFxvKDvOcPc4rV NjCn7KM0BV+161V8viSCuJZ8SITGJGEh7IUUlxOFMYUH1kJOCN4Wh+I5pqCuK01k40kSNtnKiKIl UfxAgW5sU5JlS05rtt5YFJF1SWoqHv6BRgQcA4/bdb9WRjmMk3jyUMAbIoyJq9e4cVmgzKt9j5gN J/aYDhk+hhjExwaPcz5PObA5Zd9bvz5UzCTZuZDidu5AchpiKSwPR+TV1bCWqBQ9nCjJ5neivC82 Nr4LeS2j1gw5LUqwBuhGQQU5UwHeSslXkwIynz2U2A2yKDgG7OffwLA3TpGRpk0Tzpj3ZEJXi/GR a6GN0e0NtppChePdcAz1FTS+FN8Kpxoiykk+J8fC5tenjbu9kSqpHLu9jBrhUuhNwVGbpyZrzrl0 HPVD8SXj5EOOD/eDC64VjvYBZLuUbIVAfBN1t/C/SU+cAGtdGo+fVnzycUX5wmn6eCKPmWYJG2E/ ppTsuRCbvUD9fwquksG5GqLVKhzDQ3GrEyLKSXhsiK3T5j9EyxffCFOYi2wUlPyFFDQAhehESDjQ GoX0ho0oj8RPou6TxHJdcT0NTcWkO0AnG7+uXsnHqcas/72wvWF+mSf7N/Watp9kTXoluzeS0fB9 9CcZ/hvhSZTfh99pispZOXL9KrGrARkNUMu9ITbScZWs7xOYNt9pwxSZtYDsX/GE3xTkrXgwAsXm fud08FiTeC+m2/o7ppo+PTS/NBK5ARpDG5YHr++DpmVfreYdiX8mnp+DzPEG9WbqJON0JBenZofs sxBA1gj5NXGvfJu/Qh7HwQh/VDUEyUxCit5hH94QfKkEdu+GwK8BX0hvCF+D67xhjmXQCXMwJCZ+ olK0A1EKRCE4snsh42w8Mv/Zhxw9iD7Cjo7CyQC/KyF6oBfShL4PYgG+CHsYKyZxvxZSZBAgjhIz YDz+AbfD37GtbaoSKzgGt+lKfI/GZtay6vG4VXTpPsh+IcQhOk9I7WYeP5AM3HZ9+DaSGIp9HIuu huC4pCGGx+YD2fcc5tfIAA0h/Et4/jX8zz4fWAasIf4UbR9Y6HO4d8jAnd4U0Y8ageuCB+c+H5R3 CHU2mTMwZ+B/y8DfSMBLLOYXVuEAAAAASUVORK5CYII= ------=_Part_846_578407284.1453122415557-- ------=_Part_845_227234319.1453122415557--