Date: Mon, 18 Jan 2016 18:26:32 +0100 (CET)
From: Andreas Joseph Krogh <andreas@visena.com>
To: pgsql-sql@postgresql.org
Message-ID: <VisenaEmail.ce.469a2cd6e8503cc8.15255bae04c@tc7-visena>
In-Reply-To: <614277427.7358116.1453134608680.JavaMail.yahoo@mail.yahoo.com>
Subject: Re: BYTEA
MIME-Version: 1.0
Content-Type: multipart/related;
 boundary="----=_Part_906_1038477243.1453137992331"
Precedence: bulk
Sender: pgsql-sql-owner@postgresql.org

------=_Part_905_101929767.1453137992331
Content-Type: multipart/related; 
	boundary="----=_Part_906_1038477243.1453137992331"

------=_Part_906_1038477243.1453137992331
Content-Type: multipart/alternative;
 boundary="----=_Part_907_571859276.1453137992348"

------=_Part_907_571859276.1453137992348
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

P=C3=A5 mandag 18. januar 2016 kl. 17:30:08, skrev Eugene Yin <eugeneymail@=
ymail.com=20
<mailto:eugeneymail@ymail.com>>:
To understand it further, let me give an example that I have a group of=20
pictures (assume they a set of house pictures, such as living room, bedroom=
,=20
kitchen,patio, yard, etc).=C2=A0 I first save them (larger size) to the DB.=
=C2=A0 Then=20
when the user retrieve them, the user first see a line-up ofsmall pics, the=
n=20
they pick and click one, the selected one will popup a larger picture.=C2=
=A0
=C2=A0
I have two ways to do this:
=C2=A0
1) BYTEA
=C2=A0
2) OID
=C2=A0
For BYTEA, the whole set of pictures (larger size) are retrieved (dumped) i=
nto=20
the memories on both the Java and Pg sides, occupying the same size as the=
=20
original picture is.=C2=A0 Hence taking up larger memories.
=C2=A0
For OID, it takes smaller memories on both Java and Pg sides (store the byt=
e=20
array in a buffer?).=C2=A0 Whenever the user clicks the smaller pic, it the=
n get the=20
byte array from the buffer and display the larger (original) sized one, hen=
ce=20
taking up less memories (avoiding the memory leaking)?

=C2=A0
One is always free to design an app in such a way that no matter how you st=
ore=20
the images it may still use all available memory. My point is that using BL=
OB=20
with OID (together with the pgjdbc-ng driver) lets you get away with using =
as=20
little memory as you choose. It's the same as working with a large file usi=
ng=20
FileInputStream. Consuming such streams is done by reading chunks of it int=
o a=20
pre-allocated byte-array, which is the common way in most=20
programming-languages. This byte-array is the only extra penalty in you app=
.=20
How you feed the contents of the stream back to the browser is up to you, a=
nd=20
there are lots of ways to do that inefficiently, even if the underlying=20
producer of data is a stream of bytes. If you are in a JEE servlet-environm=
ent=20
then writing to ServletOutputStream while reading form the BLOB's inputstre=
am=20
is the most memory-efficient way to stream data from PG to the browser.
=C2=A0
Note that reading BLOBs in PG requires a transaction.
=C2=A0
-- Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
andreas@visena.com <mailto:andreas@visena.com>
www.visena.com <https://www.visena.com>
 <https://www.visena.com>


=C2=A0

------=_Part_907_571859276.1453137992348
Content-Type: text/html;charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div>P=C3=A5 mandag 18. januar 2016 kl. 17:30:08, skrev Eugene Yin &lt;<a h=
ref=3D"mailto:eugeneymail@ymail.com">eugeneymail@ymail.com</a>&gt;:</div>

<blockquote style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt=
 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style=3D"color:#000; background-color:#fff; font-family:HelveticaNeue,=
 Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px=
">
<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_14654"><span id=3D"yui_3_=
16_0_1_1453087055678_14795">To understand it further, let me give an exampl=
e that I have a group of pictures (assume they a set of house pictures, suc=
h as living room, bedroom, kitchen, </span><span id=3D"yui_3_16_0_1_1453087=
055678_14891"><span id=3D"yui_3_16_0_1_1453087055678_14797">patio, </span>y=
ard, etc).=C2=A0 I first save them (</span><span id=3D"yui_3_16_0_1_1453087=
055678_14891"><span id=3D"yui_3_16_0_1_1453087055678_14891"><b id=3D"yui_3_=
16_0_1_1453087055678_15718">larger size) </b></span>to the DB.=C2=A0 Then w=
hen the user retrieve them, the user first see a line-up of <b>small pics</=
b>, then they pick and click one, the selected one will popup a<b id=3D"yui=
_3_16_0_1_1453087055678_15718"> larger picture</b>.=C2=A0</span></div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15245">=C2=A0</div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15244"><span id=3D"yui_3_=
16_0_1_1453087055678_14891">I have two ways to do this:</span></div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15243">=C2=A0</div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15024"><span id=3D"yui_3_=
16_0_1_1453087055678_14891">1) BYTEA</span></div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15242">=C2=A0</div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15169"><span id=3D"yui_3_=
16_0_1_1453087055678_14891">2) OID</span></div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15204">=C2=A0</div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15205">For BYTEA, the who=
le set of pictures (larger size) are retrieved (dumped) into the memories o=
n both the Java and Pg sides, occupying the same size as the original pictu=
re is.=C2=A0 Hence taking up larger memories.</div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15241">=C2=A0</div>

<div dir=3D"ltr" id=3D"yui_3_16_0_1_1453087055678_15440">For OID, it takes =
smaller memories on both Java and Pg sides (store the byte array in a buffe=
r?).=C2=A0 Whenever the user clicks the smaller pic, it then get the byte a=
rray from the buffer and display the larger (original) sized one, hence tak=
ing up less memories (avoiding the memory leaking)?</div>
</div>
</blockquote>

<div>=C2=A0</div>

<div>One is always free to design an app in such a way that no matter how y=
ou store the images it may still use all available memory. My point is that=
 using BLOB with OID (together with the pgjdbc-ng driver) lets you get away=
 with using as little memory as you choose. It's the same as working with a=
 large file using FileInputStream. Consuming such streams is done by readin=
g chunks of it into a pre-allocated byte-array, which is the common way in =
most programming-languages. This byte-array is the only extra penalty in yo=
u app. How you feed the contents of the stream back to the browser is up to=
 you, and there are lots of ways to do that inefficiently, even if the unde=
rlying producer of data is a stream of bytes. If you are in a JEE servlet-e=
nvironment then writing to ServletOutputStream while reading form the BLOB'=
s inputstream is the most memory-efficient way to stream data from PG to th=
e browser.</div>

<div>=C2=A0</div>

<div>Note that reading BLOBs in PG requires a transaction.</div>

<div>=C2=A0</div>

<div class=3D"origo-email-signature">
<div>--
<div><span style=3D"font-family:arial,helvetica,sans-serif;"><strong>Andrea=
s Joseph Krogh</strong></span></div>

<div><span style=3D"font-family:arial,helvetica,sans-serif;">CTO / Partner<=
/span> - Visena AS</div>

<div><span style=3D"font-family:arial,helvetica,sans-serif;">Mobile: +47 90=
9 56 963</span></div>

<div><a href=3D"mailto:andreas@visena.com">andreas@visena.com</a></div>

<div><a href=3D"https://www.visena.com">www.visena.com</a></div>

<div style=3D"padding-top: 5px"><a href=3D"https://www.visena.com"><img alt=
=3D"" src=3D"cid:part_F15216439038054ILXO"></a></div>
</div>
</div>

<div>=C2=A0</div>
------=_Part_907_571859276.1453137992348--

------=_Part_906_1038477243.1453137992331
Content-Type: image/png
Content-Transfer-Encoding: base64
Content-Disposition: inline
Content-ID: <part_F15216439038054ILXO>

iVBORw0KGgoAAAANSUhEUgAAAIUAAAAYCAYAAADUIj6hAAAABHNCSVQICAgIfAhkiAAABzBJREFU
aEPtmNFxHDcMhmVP3i1VECpvnjzkVIHWFfhcgVcVRKrAUgWRK/C6Al8H3lTgy0PGbzFdQc4VJP/H
ADs43q6kROeJNbOYgQACIAgCWJKng4MZ5gxUGXh0U0b+SIuF9G+EWXj2Q15vbrE/NPtk9uub7Gfd
t5mBx1NhqSEo8HshjbEUtlO2yIM9tsx5dZP9rPt2MzDZFAqZE4LGcFjdsg3saQaHt7fY70X949On
aS+OZidDBkavD33157L4xaw2os+Mp/DAC10l2XhOCeStj0W5ajrJL8WfCi80Xgf9vVlrhg9yRONe
/f7xI2vNsIcM7JwU9o6oGyJrLT8JOA1aX9sKP4wl94bAniukEbo/n7YPmuTETzIab4Y9ZWCrKexd
8M58lxPCvvD6aihfvexbEQoPYO8NQRO0JnddGO6FJYaVsBde7cXj7KRk4LsqDxQzCYeGsJNgaXZe
+JXkyGgWINq3Gp+bHNILz+wEYs5qH1eJrouNrpDXtk4O6x1IvtD40GWyJYYdkF0jIbYZlN16xygI
gl9smbMDsmFdfAKb23xGB8H/mv3tOJfArs1kukk79DEPdQ5u0g1vCvvqKXIW8mZYW+F3Tg4r8HvZ
kQCCLydK8EFMQCe5N8RgL9mRG0SqQFuNvdE6beSs0qPDBuCdg0+gvClso8SbTO4ki7mQzQqBrcMH
QPwReg2wW8umEe/+L8T/LEzBGF9nXjzZ46s+ITHvhegW8LL39xm6App7KYL/GE+vcYnFbJiP/0aY
hdiCnRC7jaj7eiK2ETInC5PRF6LYkSN0+HabF75WuT6syCyI0YkVGGMvUC33AmfZTDUEj8u6IVhu
EhRUJ2U2g6UlugyNb03Hl9obHwlxJbcRzcYjO4QPjVfGFTQav4vrmp7cpMp2qbHnBxWJbisbho2Q
XI6C1sIHDUFhH4Hij4UbIev66eANeiwbkA+LBsO36zAHzoXU7AhbqLAXEiOYTXcSdO8VS9L4wN8U
BIYhBd6oSUgYMijOkecRuTdQa/YiBXhbXFcnCnI2SrfeBG9NydrLYMhGHa5qB9pQIxlzAE6Okjzx
JO5afGe6kmhBicWKQNKuTZ5EW+MjQY8v/9rQ0bhJSJyNGUe/JH1t8h1iMbdSPAvxHYin6YmN9YBX
QmTYZZNh14vHhhhal5vtcIrJjmvszPRJdExH3MWHvykoBEc9CoCGWJisOLOGoCORs1FvIMbYA8z3
kwM59oemy6LlWrLxFLmWgiQAfEGd8S+NssbK+EhyGLxUkhhiy717waBqHOJYSEacwBejkFNhjJNj
v/gANOdQxPfciP/JVBCK2cOIcg3RRJ+CPrLPNVhhN6F38VKMF3XLlIJrjU5C8gMFxvKDvOcPc4rV
NjCn7KM0BV+161V8viSCuJZ8SITGJGEh7IUUlxOFMYUH1kJOCN4Wh+I5pqCuK01k40kSNtnKiKIl
UfxAgW5sU5JlS05rtt5YFJF1SWoqHv6BRgQcA4/bdb9WRjmMk3jyUMAbIoyJq9e4cVmgzKt9j5gN
J/aYDhk+hhjExwaPcz5PObA5Zd9bvz5UzCTZuZDidu5AchpiKSwPR+TV1bCWqBQ9nCjJ5neivC82
Nr4LeS2j1gw5LUqwBuhGQQU5UwHeSslXkwIynz2U2A2yKDgG7OffwLA3TpGRpk0Tzpj3ZEJXi/GR
a6GN0e0NtppChePdcAz1FTS+FN8Kpxoiykk+J8fC5tenjbu9kSqpHLu9jBrhUuhNwVGbpyZrzrl0
HPVD8SXj5EOOD/eDC64VjvYBZLuUbIVAfBN1t/C/SU+cAGtdGo+fVnzycUX5wmn6eCKPmWYJG2E/
ppTsuRCbvUD9fwquksG5GqLVKhzDQ3GrEyLKSXhsiK3T5j9EyxffCFOYi2wUlPyFFDQAhehESDjQ
GoX0ho0oj8RPou6TxHJdcT0NTcWkO0AnG7+uXsnHqcas/72wvWF+mSf7N/Watp9kTXoluzeS0fB9
9CcZ/hvhSZTfh99pispZOXL9KrGrARkNUMu9ITbScZWs7xOYNt9pwxSZtYDsX/GE3xTkrXgwAsXm
fud08FiTeC+m2/o7ppo+PTS/NBK5ARpDG5YHr++DpmVfreYdiX8mnp+DzPEG9WbqJON0JBenZofs
sxBA1gj5NXGvfJu/Qh7HwQh/VDUEyUxCit5hH94QfKkEdu+GwK8BX0hvCF+D67xhjmXQCXMwJCZ+
olK0A1EKRCE4snsh42w8Mv/Zhxw9iD7Cjo7CyQC/KyF6oBfShL4PYgG+CHsYKyZxvxZSZBAgjhIz
YDz+AbfD37GtbaoSKzgGt+lKfI/GZtay6vG4VXTpPsh+IcQhOk9I7WYeP5AM3HZ9+DaSGIp9HIuu
huC4pCGGx+YD2fcc5tfIAA0h/Et4/jX8zz4fWAasIf4UbR9Y6HO4d8jAnd4U0Y8ageuCB+c+H5R3
CHU2mTMwZ+B/y8DfSMBLLOYXVuEAAAAASUVORK5CYII=
------=_Part_906_1038477243.1453137992331--

------=_Part_905_101929767.1453137992331--