Date: Fri, 12 Oct 2018 16:27:45 +0000 (UTC)
From: "Sam R." <samruohola@yahoo.com>
Reply-To: "samruohola@yahoo.com" <samruohola@yahoo.com>
To: 
	"pgsql-performance@lists.postgresql.org"
 <pgsql-performance@lists.postgresql.org>
Message-ID: <1288355243.8943320.1539361665724@mail.yahoo.com>
In-Reply-To: <4dbb8bea-bea3-74d8-a537-36bb9864cb9f@portavita.eu>
References: 
 <CAMqTPq=aFwfiqv=P9EDs+UxeOgfe03kyQ0uvtUHvSxvfSvYR+w@mail.gmail.com>
 <4dbb8bea-bea3-74d8-a537-36bb9864cb9f@portavita.eu>
Subject: One big table or split data? Writing data. From disk point of view.
 With a good storage (GBs/s, writing speed)
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_8943319_527060043.1539361665723"
Content-Length: 4522
Precedence: bulk

------=_Part_8943319_527060043.1539361665723
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi!

Could someone discuss about following? It would be great to hear comments!
There is a good storage. According to "fio", write speed could be e.g. 3 GB=
/s.=C2=A0(It is First time using the command for me, so I am not certain of=
 the real speed with "fio". E.g. with --bs=3D100m, direct=3D1, in fio. The =
measurement result may be faulty also. So, checking still.)
Currently there is one table without partitioning. The table contains json =
data. In PostgreSQL, in Linux.
Write speed can be e.g. 300 - 600 MB/s, through PostgreSQL. Measured with d=
stat while inserting. Shared buffers is large is PostgreSQL.
With a storage/disk which "scales", is there some way to write faster to th=
e disk in the system through PostgreSQL?
Inside same server.
Does splitting data help? Partitioned table / splitting to smaller tables? =
Should I test it?
Change settings somewhere? Block sizes? 8 KB / 16 KB, ... "Dangerous" to ch=
ange?
2nd question, sharding:
If the storage / "disk" scales, could better *disk writing speed* be achiev=
ed (in total) with sharding kind of splitting of data? (Same NAS storage, w=
hich scales, in use in all shards.)Sharding or use only one server? From pu=
re disk writing speed point of view.

BR Sam

------=_Part_8943319_527060043.1539361665723
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div id=3D"yMail_cursorElementTracker_1539360822968"><div id=3D"yMail_curso=
rElementTracker_1539360832049">Hi!<br></div><div id=3D"yMail_cursorElementT=
racker_1539361146432"><br></div><div id=3D"yMail_cursorElementTracker_15393=
61147353">Could someone discuss about following? It would be great to hear =
comments!</div><div id=3D"yMail_cursorElementTracker_1539360871320"><br></d=
iv><div id=3D"yMail_cursorElementTracker_1539360874197">There is a good sto=
rage. According to "fio", write speed could be e.g. 3 GB/s.</div><div id=3D=
"yMail_cursorElementTracker_1539360901998">&nbsp;(It is First time using th=
e command for me, so I am not certain of the real speed with "fio". E.g. wi=
th --bs=3D100m, direct=3D1, in fio. The measurement result may be faulty al=
so. So, checking still.)</div><div id=3D"yMail_cursorElementTracker_1539360=
940090"><br></div><div id=3D"yMail_cursorElementTracker_1539360943515">Curr=
ently there is one table without partitioning. The table contains json data=
. In PostgreSQL, in Linux.</div><div id=3D"yMail_cursorElementTracker_15393=
60944512"><br></div><div id=3D"yMail_cursorElementTracker_1539360950666">Wr=
ite speed can be e.g. 300 - 600 MB/s, through PostgreSQL. Measured with dst=
at while inserting. Shared buffers is large is PostgreSQL.</div><div id=3D"=
yMail_cursorElementTracker_1539360951853"><br></div><div id=3D"yMail_cursor=
ElementTracker_1539360957430">With a storage/disk which "scales", is there =
some way to write faster to the disk in the system through PostgreSQL?</div=
><div id=3D"yMail_cursorElementTracker_1539360974566"><br></div><div id=3D"=
yMail_cursorElementTracker_1539360971344">Inside same server.</div><div id=
=3D"yMail_cursorElementTracker_1539360972624"><br></div><div id=3D"yMail_cu=
rsorElementTracker_1539361412800">Does splitting data help? Partitioned tab=
le / splitting to smaller tables? Should I test it?</div><div id=3D"yMail_c=
ursorElementTracker_1539361413888"><br></div><div id=3D"yMail_cursorElement=
Tracker_1539360978960">Change settings somewhere? Block sizes? 8 KB / 16 KB=
, ... "Dangerous" to change?</div><div id=3D"yMail_cursorElementTracker_153=
9360980134"><br></div><div id=3D"yMail_cursorElementTracker_1539360983752">=
2nd question, sharding:</div><div id=3D"yMail_cursorElementTracker_15393609=
84620"><br></div><div id=3D"yMail_cursorElementTracker_1539360826906">If th=
e storage / "disk" scales, could better *disk writing speed* be achieved (i=
n total) with sharding kind of splitting of data? (Same NAS storage, which =
scales, in use in all shards.)</div><div id=3D"yMail_cursorElementTracker_1=
539361044144">Sharding or use only one server? From pure disk writing speed=
 point of view.</div><div id=3D"yMail_cursorElementTracker_1539361110758"><=
br></div><div id=3D"yMail_cursorElementTracker_1539360827961"><br></div><di=
v id=3D"yMail_cursorElementTracker_1539360822968">BR Sam</div><div id=3D"yM=
ail_cursorElementTracker_1539360825205"><br></div></div>
------=_Part_8943319_527060043.1539361665723--