Message-Id: <201804282007.w3SK7g9R001586@rs111.luxsci.com>
From: "g@luxsci.net" <g@luxsci.net>
Date: Sat, 28 Apr 2018 20:07:42 +0000
To: pgsql-general@postgresql.org
Reply-To: g@luxsci.net
Errors-To: g@luxsci.net
MIME-Version: 1.0
Subject: Re: Rationale for aversion to the central database?
Content-Type: multipart/mixed;
              boundary="------------------_Boundary_47119361.3116131";
              charset="utf-8"
Precedence: bulk

This is a multi-part message in MIME format.

If you cannot view the attachment(s), then your email client does not
support MIME.


--------------------_Boundary_47119361.3116131
Content-Type: multipart/alternative; boundary="-------------1541229.81100533"

---------------1541229.81100533
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable

=A0
=A0
On April 28, 2018 11:18:02 am PDT, "Peter J. Holzer" <hjp-pgsql@hjp.at> wro=
te:

On 2018-04-28 09:54:27 -0500, Steven Lembark wrote:
> On Sat, 28 Apr 2018 08:02:21 +0200
> "Peter J. Holzer" <hjp-pgsql@hjp.at> wrote:
>
> > On 2018-04-27 22:52:39 +0000, g@luxsci.net wrote:
> > > Perhaps I'm extreme. In my ideal world, developers might not even
> > > know table names! I'm kidding ,sorta...=A0
> >
> > If they don't know the table names, how can they write those stored
> > procedures?
>
> One of the main reasons DBA's need to be pigs. Much spaghetti can be
> avoided in ORM frameworks with well-designed views.

Apparently my remark was too short (and perhaps with too little context)
to be clear.

As I understood g@luxsci.net, they were arguing for writing lots of
stored procedures so that developers wouldn't even have to know the
table names (they would just call the procedures) [Although I now see
that it was somebody else who proposed that as the only interface].

But those stored procedures don't write themselves. Somebody has to
write them and a person who writes code is called a developer. So there
needs to be at least one developer who knows the table names - the
developer of the stored procedures. And that developer should better be
good - API design is hard, and if you take away SQL from your fellow
developers you should give them something better (for the task at hand),
not something worse.

=3D=3D=3D

Yes, sorry, I meant application developers. Well, actually I meant=20
developers who only look at the db through an ORM and who like it that=20
way. At my current position, the ones called "developers" all use=20
Django, and they practically never look at ( let alone interact with)=20
the db using psql. Forget about writing and using pgsql functions.=20
There are no DBAs or database developers. I am trying to change this=20
culture a bit but I may give up soon. Tragic, really, given the mostly=20
terrible performance that they get with naive Django code. Some of=20
these developers have even gone so far as to proclaim that PG is too=20
slow ( they didn't even configure it !) for their "big" data, which=20
really aren't big, and move on to something like ES, which they really=20
don't need. For example, they have code that could benefit from simple=20
FTS operations, and instead of taking seconds or minutes, could be done=20
in milliseconds. Or perhaps they could not execute certain queries for=20
user interfaces quickly
enough with their Django code so they think that they need ES for those=20
queries. So now they have to manage and design their ES=20
implementations, activities which frankly should _not_ be taken=20
lighlty, and manage their data in the PG db which they eventually come=20
back to for at least some sane data integrity. Now they have to perform=20
very long and convoluted processes to keep both in sync. Ouch. Know=20
your tools!

But there is no all or nothing, as others have pointed out. Some things=20
are better done and more easily managed outside of the DB, at least for=20
me.
For example, I tried doing lots of bigint math in pgsql and I moved to=20
external C code for that.
That C code still called tried and true pgsql convenience functions for=20
getting at the correct data however, and these were the same functions=20
that my Java code used for other activities.

Gerry


---------------1541229.81100533
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML>
<HTML>
<HEAD><meta http-equiv=3D"content-type" content=3D"text/html; charset=3DUTF=
-8"></HEAD>
<BODY style=3D"padding:5px; line-height: 1.3em; font-size: 13px; color: #33=
3; background-color: #FFF; font-family: Arial, Helvetica, sans-serif">
&nbsp;<br />
&nbsp;
<div style=3D"padding:3px 0 3px 0;border-top:1px solid #DDD; border-bottom:=
 1px solid #DDD;margin:3px 0 3px 0">On April 28, 2018 11:18:02 am PDT, &quo=
t;Peter J. Holzer&quot; &lt;hjp-pgsql@hjp.at&gt; wrote:</div>
<br />
<br />
On 2018-04-28 09:54:27 -0500, Steven Lembark wrote:<br />
&gt; On Sat, 28 Apr 2018 08:02:21 +0200<br />
&gt; &quot;Peter J. Holzer&quot; &lt;hjp-pgsql@hjp.at&gt; wrote:<br />
&gt;<br />
&gt; &gt; On 2018-04-27 22:52:39 +0000, g@luxsci.net wrote:<br />
&gt; &gt; &gt; Perhaps I&#39;m extreme. In my ideal world, developers might=
 not even<br />
&gt; &gt; &gt; know table names! I&#39;m kidding ,sorta...&nbsp;<br />
&gt; &gt;<br />
&gt; &gt; If they don&#39;t know the table names, how can they write those =
stored<br />
&gt; &gt; procedures?<br />
&gt;<br />
&gt; One of the main reasons DBA&#39;s need to be pigs. Much spaghetti can =
be<br />
&gt; avoided in ORM frameworks with well-designed views.<br />
<br />
Apparently my remark was too short (and perhaps with too little context)<br=
 />
to be clear.<br />
<br />
As I understood g@luxsci.net, they were arguing for writing lots of<br />
stored procedures so that developers wouldn&#39;t even have to know the<br =
/>
table names (they would just call the procedures) [Although I now see<br />
that it was somebody else who proposed that as the only interface].<br />
<br />
But those stored procedures don&#39;t write themselves. Somebody has to<br =
/>
write them and a person who writes code is called a developer. So there<br =
/>
needs to be at least one developer who knows the table names - the<br />
developer of the stored procedures. And that developer should better be<br =
/>
good - API design is hard, and if you take away SQL from your fellow<br />
developers you should give them something better (for the task at hand),<br=
 />
not something worse.<br />
<br />
=3D=3D=3D<br />
<br />
Yes, sorry, I meant application developers. Well, actually I meant develope=
rs who only look at the db through an ORM and who like it that way. At my c=
urrent position, the ones called &quot;developers&quot; all use Django, and=
 they practically never look at ( let alone interact with) the db using psq=
l. Forget about writing and using pgsql functions. There are no DBAs or dat=
abase developers. I am trying to change this culture a bit but I may give u=
p soon. Tragic, really, given the mostly terrible performance that they get=
 with naive Django code. Some of these developers have even gone so far as =
to proclaim that PG is too slow ( they didn&#39;t even configure it !) for =
their &quot;big&quot; data, which really aren&#39;t big, and move on to som=
ething like ES, which they really don&#39;t need. For example, they have co=
de that could benefit from simple FTS operations, and instead of taking sec=
onds or minutes, could be done in milliseconds. Or perhaps they could not e=
xecute certain queries for user interfaces quickly enough with their Django=
 code so they think that they need ES for those queries. So now they have t=
o manage and design their ES implementations, activities which frankly shou=
ld _not_ be taken lighlty, and manage their data in the PG db which they ev=
entually come back to for at least some sane data integrity. Now they have =
to perform very long and convoluted processes to keep both in sync. Ouch. K=
now your tools!<br />
<br />
But there is no all or nothing, as others have pointed out. Some things are=
 better done and more easily managed outside of the DB, at least for me.<br=
 />
For example, I tried doing lots of bigint math in pgsql and I moved to exte=
rnal C code for that.<br />
That C code still called tried and true pgsql convenience functions for get=
ting at the correct data however, and these were the same functions that my=
 Java code used for other activities.<br />
<br />
Gerry
</BODY>
</HTML>


---------------1541229.81100533--


--------------------_Boundary_47119361.3116131--