MIME-Version: 1.0
In-Reply-To: 
 <CAM5-9D-EayFdbqNQN-E5znYRf8PSk-DNxswB-ctb8Nmi6xHnkg@mail.gmail.com>
References: 
 <CAM5-9D92zW=qJqKT_HFjtMQvOAXuz5a5DJuABnz9fWWLKeZZrw@mail.gmail.com>
 <CA+OCxoz3Y7aR0JJ0mq-YnnZnztML-Hkv3Trt3hf74GvW0-0Urw@mail.gmail.com>
 <CAM5-9D-EayFdbqNQN-E5znYRf8PSk-DNxswB-ctb8Nmi6xHnkg@mail.gmail.com>
From: Dave Page <dpage@pgadmin.org>
Date: Mon, 22 May 2017 22:33:13 +0100
Message-ID: 
 <CA+OCxozaTaaj_mEQRK=N14S=Q8FuepeY2SqPyW5cz8hw6f+pVg@mail.gmail.com>
Subject: Re: [pgAdmin4][Patch]: Load module's JS files only
 when required
To: Surinder Kumar <surinder.kumar@enterprisedb.com>
Cc: pgadmin-hackers <pgadmin-hackers@postgresql.org>
Content-Type: multipart/alternative; boundary="001a1145ecc65efc7f0550239e04"
Precedence: bulk
Sender: pgadmin-hackers-owner@postgresql.org

--001a1145ecc65efc7f0550239e04
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi

On Monday, May 22, 2017, Surinder Kumar <surinder.kumar@enterprisedb.com>
wrote:

> Hi
>
> On Mon, May 22, 2017 at 4:22 PM, Dave Page <dpage@pgadmin.org
> <javascript:_e(%7B%7D,'cvml','dpage@pgadmin.org');>> wrote:
>
>> Hi
>>
>> On Monday, May 22, 2017, Surinder Kumar <surinder.kumar@enterprisedb.com=
>
>> wrote:
>>
>>> Hi
>>>
>>> As per pgAdmin4 design, template JS files can either be preloaded or
>>> load when a specific node expands (by adding for e.g.: when: 'server').
>>>
>>> The JS files of several modules found to be loaded when pgAdmin4 loads
>>> which results in increasing:
>>>
>>>    - the number of http requests
>>>    - latency(greater request time)
>>>    - pgAdmin4 load time
>>>
>>> *Tested on Firefox:*
>>>
>>> Before applying patch
>>>
>>>    - http requests - 143
>>>    - Content size - 3.4 MB
>>>    - Load time: 4.1s (onload: 524ms)
>>>
>>> After applying patch
>>>
>>>    - http requests: 68
>>>    - Content size: 2.1 MB
>>>    - Load time: 2.84s (onload: 481ms)
>>>
>>> This is great work!
>>
>> However (sorry!) - I'm planning on working on an alternative change on m=
y
>> flight in a couple of hours. Joao has broken the Grunt code out of the
>> History tab patch for me to work with - the idea is something like:
>>
>> - We continue to migrate all the JS out of templates and into static
>> files whereever possible, using the client-side translation.
>>
> =E2=80=8BYes. It will be then easier to cover most of the JS code for min=
ify if
> client side =E2=80=8Btranslation is used.
>

Unfortunately, I looked at this on my flight and it looks like it'll be a
mammoth task. Many/most JS files also currently use the url_for template
function - so we need to figure out how to replace that with JS code first.


>
>> - We then have a set of Grunt tasks that will collect all the static JS,
>> minimise it, and pack it into a single file to load at startup.
>>
> =E2=80=8BI wrote few grunt tasks using r.js optimizer this weekend which:
>
> 1. Finds all static JS file recursively into the modules using regex
> pattern and then creates the tasks to minify each JS file and store its
> 'min.js' in same directory and loads min.js when current_app.debug is Fal=
se
> else returns JS file.
>

> 2. A task to merge given list of CSS files into a single file and then
> minify using 'grunt-contrib-cssmin' task and then insert that file with
> <script> tags in base.html file.
>

There's a script in the tools directory to do something similar already.

I am already starting to wonder if Grunt is the best way to do this though.
It might be easier (though not quite as efficient) to have the Python
module all return their static/template JS code at initialisation,
effectively dynamically building a single packed file containing nearly
everything.


>
>> I'm currently thinking that non-debug/package builds will pre-pack
>> everything for installation. The aim is to have JS HTTP requests be down=
 in
>> single figures to maximise loading time.
>>
> =E2=80=8BThat will make pgAdmin4 much faster.
>
>>
>> The are various issues to work through however; at least, how do we
>> handle debug builds, and how do we re-pack everything if a user installs=
 a
>> new module (or updates something).
>>
> =E2=80=8BI don't know much about builds but the debug/package builds can =
be
> switched using 'current_app.debug' config parameter.
> Using Regex Pattern will help to search JS files throughout the App
> directory to perform minify task.
>

Yeah, though I think there are more considerations we haven't yet thought
of.


>
>> We'd also need to get rid of requirejs.
>>
> =E2=80=8BAny alternative to requirejs ?=E2=80=8B
>

If everything is packed together, then we don't need anything like it.

I think this needs more discussion.


>
>>
>>
>>>
>>> Also, I found the http request for gravtar takes much time(depending on
>>> internet speed) to load which increases load time.
>>> https://secure.gravatar.com/avatar/d30aea269994f01256a99a8e7
>>> 154a328?s=3D100&d=3Dretro&r=3Dg
>>>
>>> Shouldn't the gravatar image be replaced with custom image ? I don't se=
e
>>> its any use except displaying for user.
>>>
>>
>> Maybe. Doesn't it load asynchronously though?
>>
>
>>
>>>
>>> *Patch changes:*
>>>
>>> 1) Load following modules JS when 'servers' node expands:
>>>
>>>    -  Roles
>>>    -  FileManager
>>>    -  Backup
>>>    -  Datagrid
>>>    -  Grant wizard
>>>    -  Import/Export, Maintenance & restore
>>>    -  Sqleditor
>>>
>>> 2) Load Debugger JS when database node expands
>>>
>>> 3) Load Flotr2 when render function is called in dashboard.js
>>>
>>> 4) Load BigNumber library when sorting is performed on Statistics tab i=
n
>>> backgrid.pgadmin.js
>>>
>>> 5) Remove 'backgrid.select.all' from dependency list(user_management.js=
)
>>> as it had no use.
>>>
>>> After applying this patch, Menu items such "Query tool", "Grant wizard"
>>> don't appear in Tools menu.
>>> I have logged an RM2424 <https://redmine.postgresql.org/issues/2424> to
>>> track this.
>>>
>>
>> OK. Lets hold fire on this for now though, until I can PoC the work abov=
e.
>>
>> Of course, any thoughts or comments are welcome.
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>>
>

--=20
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--001a1145ecc65efc7f0550239e04
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi<br><br>On Monday, May 22, 2017, Surinder Kumar &lt;<a href=3D"mailto:sur=
inder.kumar@enterprisedb.com">surinder.kumar@enterprisedb.com</a>&gt; wrote=
:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-le=
ft:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_de=
fault" style=3D"font-size:small">Hi</div><div class=3D"gmail_extra"><br><di=
v class=3D"gmail_quote">On Mon, May 22, 2017 at 4:22 PM, Dave Page <span di=
r=3D"ltr">&lt;<a href=3D"javascript:_e(%7B%7D,&#39;cvml&#39;,&#39;dpage@pga=
dmin.org&#39;);" target=3D"_blank">dpage@pgadmin.org</a>&gt;</span> wrote:<=
br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord=
er-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:soli=
d;padding-left:1ex">Hi<span><br><br>On Monday, May 22, 2017, Surinder Kumar=
 &lt;<a>surinder.kumar@enterprisedb.c<wbr>om</a>&gt; wrote:<br><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1=
px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:=
1ex"><div dir=3D"ltr"><div style=3D"font-size:small">Hi</div><div style=3D"=
font-size:small"><br></div><div style=3D"font-size:small">As per pgAdmin4 d=
esign, template JS files can either be preloaded or load when a specific no=
de expands (by adding for e.g.: when: &#39;server&#39;).</div><div style=3D=
"font-size:small"><br></div><div style=3D"font-size:small">The JS files of =
several modules found to be loaded when pgAdmin4 loads which results in inc=
reasing:</div><div style=3D"font-size:small"><ul><li>the number of http req=
uests<br></li><li>latency(greater request time)<br></li><li>pgAdmin4 load t=
ime</li></ul></div><div style=3D"font-size:small"><div><div style=3D"font-s=
ize:12px;color:rgb(0,0,0);font-family:-apple-system-font;line-height:16px">=
<u><b>Tested on Firefox:</b></u></div><div style=3D"font-size:12px;color:rg=
b(0,0,0);font-family:-apple-system-font;line-height:16px"><u><b><br></b></u=
></div><div style=3D"font-size:12px;color:rgb(0,0,0);font-family:-apple-sys=
tem-font;line-height:16px">Before applying patch</div><ul><li>http requests=
 - 143<br></li><li>Content size - 3.4 MB<br></li><li>Load time:=C2=A04.1s (=
onload: 524ms)</li></ul></div><div>After applying patch<br><ul><li>http req=
uests: 68<br></li><li>Content size: 2.1 MB<br></li><li>Load time:=C2=A02.84=
s (onload: 481ms)</li></ul></div></div></div></blockquote></span><div>This =
is great work!</div><div><br></div><div>However (sorry!) - I&#39;m planning=
 on working on an alternative change on my flight in a couple of hours. Joa=
o has broken the Grunt code out of the History tab patch for me to work wit=
h - the idea is something like:</div><div><br></div><div>- We continue to m=
igrate all the JS out of templates and into static files whereever possible=
, using the client-side translation.</div></blockquote><div class=3D"gmail_=
default" style=3D"font-size:small">=E2=80=8BYes. It will be then easier to =
cover most of the JS code for minify if client side =E2=80=8Btranslation is=
 used.</div></div></div></div></blockquote><div><br></div><div>Unfortunatel=
y, I looked at this on my flight and it looks like it&#39;ll be a mammoth t=
ask. Many/most JS files also currently use the url_for template function - =
so we need to figure out how to replace that with JS code first.</div><div>=
=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bo=
rder-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div class=3D"g=
mail_extra"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb=
(204,204,204);border-left-style:solid;padding-left:1ex"><div><br></div><div=
>- We then have a set of Grunt tasks that will collect all the static JS, m=
inimise it, and pack it into a single file to load at startup.</div></block=
quote><div class=3D"gmail_default" style=3D"font-size:small">=E2=80=8BI wro=
te few grunt tasks using r.js optimizer this weekend which:</div><div class=
=3D"gmail_default" style=3D"font-size:small"><br></div><div class=3D"gmail_=
default" style=3D"font-size:small">1. Finds all static JS file recursively =
into the modules using regex pattern and then creates the tasks to minify e=
ach JS file and store its &#39;min.js&#39; in same directory and loads min.=
js when current_app.debug is False else returns JS file.<span style=3D"font=
-size:15px">=C2=A0</span></div></div></div></div></blockquote><blockquote c=
lass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;=
padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D=
"gmail_quote"><div class=3D"gmail_default" style=3D"font-size:small"><br></=
div><div class=3D"gmail_default" style=3D"font-size:small">2. A task to mer=
ge given list of CSS files into a single file and then minify using &#39;gr=
unt-contrib-cssmin&#39; task and then insert that file with &lt;script&gt; =
tags in base.html file.</div></div></div></div></blockquote><div><br></div>=
<div>There&#39;s a script in the tools directory to do something similar al=
ready.</div><div><br></div><div>I am already starting to wonder if Grunt is=
 the best way to do this though. It might be easier (though not quite as ef=
ficient) to have the Python module all return their static/template JS code=
 at initialisation, effectively dynamically building a single packed file c=
ontaining nearly everything.</div><div>=C2=A0</div><blockquote class=3D"gma=
il_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-lef=
t:1ex"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quot=
e"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord=
er-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:soli=
d;padding-left:1ex"><div><br></div><div>I&#39;m currently thinking that non=
-debug/package builds will pre-pack everything for installation. The aim is=
 to have JS HTTP requests be down in single figures to maximise loading tim=
e.</div></blockquote><div class=3D"gmail_default" style=3D"font-size:small"=
>=E2=80=8BThat will make pgAdmin4 much faster.</div><blockquote class=3D"gm=
ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-l=
eft-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><=
br></div><div>The are various issues to work through however; at least, how=
 do we handle debug builds, and how do we re-pack everything if a user inst=
alls a new module (or updates something).</div></blockquote><div class=3D"g=
mail_default" style=3D"font-size:small">=E2=80=8BI don&#39;t know much abou=
t builds but the debug/package builds can be switched using &#39;current_ap=
p.debug&#39; config parameter.</div><div class=3D"gmail_default" style=3D"f=
ont-size:small">Using Regex Pattern will help to search JS files throughout=
 the App directory to perform minify task.</div></div></div></div></blockqu=
ote><div><br></div><div>Yeah, though I think there are more considerations =
we haven&#39;t yet thought of.</div><div>=C2=A0</div><blockquote class=3D"g=
mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l=
eft:1ex"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_qu=
ote"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo=
rder-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:so=
lid;padding-left:1ex"><div><br></div><div>We&#39;d also need to get rid of =
requirejs.</div></blockquote><div class=3D"gmail_default" style=3D"font-siz=
e:small">=E2=80=8BAny alternative to requirejs ?=E2=80=8B</div></div></div>=
</div></blockquote><div><br></div><div>If everything is packed together, th=
en we don&#39;t need anything like it.</div><div><br></div><div>I think thi=
s needs more discussion.</div><div>=C2=A0</div><blockquote class=3D"gmail_q=
uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e=
x"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><=
blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l=
eft-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;pa=
dding-left:1ex"><span><div><br></div><div>=C2=A0</div><blockquote class=3D"=
gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border=
-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div=
 dir=3D"ltr"><div style=3D"font-size:small"><div><div><br></div><div>Also, =
I found the http request for gravtar takes much time(depending on internet =
speed) to load which increases load time.</div><div><a href=3D"https://secu=
re.gravatar.com/avatar/d30aea269994f01256a99a8e7154a328?s=3D100&amp;d=3Dret=
ro&amp;r=3Dg" target=3D"_blank">https://secure.gravatar.com/av<wbr>atar/d30=
aea269994f01256a99a8e7<wbr>154a328?s=3D100&amp;d=3Dretro&amp;r=3Dg</a><br><=
/div><div><br></div><div>Shouldn&#39;t the gravatar image be replaced with =
custom image ? I don&#39;t see its any use except displaying for user.</div=
></div></div></div></blockquote><div><br></div></span><div>Maybe. Doesn&#39=
;t it load asynchronously though?</div></blockquote><blockquote class=3D"gm=
ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-l=
eft-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span>=
<div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px =
0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-l=
eft-style:solid;padding-left:1ex"><div dir=3D"ltr"><div style=3D"font-size:=
small"><div><div><br></div></div></div><div style=3D"font-size:small"><u><b=
>Patch changes:</b></u></div><div style=3D"font-size:small"><br></div><div =
style=3D"font-size:small">1) Load following modules JS when &#39;servers=
9; node expands:</div><div style=3D"font-size:small"><ul><li>=C2=A0Roles<br=
></li><li>=C2=A0FileManager<br></li><li>=C2=A0Backup<br></li><li>=C2=A0Data=
grid<br></li><li>=C2=A0Grant wizard<br></li><li>=C2=A0Import/Export, Mainte=
nance &amp; restore</li><li>=C2=A0Sqleditor</li></ul></div><div style=3D"fo=
nt-size:small">2) Load Debugger JS when database node expands</div><div sty=
le=3D"font-size:small"><br></div><div style=3D"font-size:small">3) Load Flo=
tr2 when render function is called in dashboard.js<br></div><div style=3D"f=
ont-size:small"><br></div><div style=3D"font-size:small">4) Load BigNumber =
library when sorting is performed on Statistics tab in backgrid.pgadmin.js<=
/div><div style=3D"font-size:small"><br></div><div style=3D"font-size:small=
">5) Remove &#39;backgrid.select.all&#39; from dependency list(user_managem=
ent.js) as it had no use.</div><div style=3D"font-size:small"><br></div><di=
v style=3D"font-size:small">After applying this patch, Menu items such &quo=
t;Query tool&quot;, &quot;Grant wizard&quot; don&#39;t appear in Tools menu=
.</div><div style=3D"font-size:small">I have logged an <a href=3D"https://r=
edmine.postgresql.org/issues/2424" target=3D"_blank">RM2424</a> to track th=
is.</div></div></blockquote><div><br></div></span><div>OK. Lets hold fire o=
n this for now though, until I can PoC the work above.</div><div><br></div>=
<div>Of course, any thoughts or comments are welcome.=C2=A0</div><span><fon=
t color=3D"#888888">
<br><br>-- <br>Dave Page<br>Blog: <a href=3D"http://pgsnake.blogspot.com" t=
arget=3D"_blank">http://pgsnake.blogspot.com</a><br>Twitter: @pgsnake<br><b=
r>EnterpriseDB UK: <a href=3D"http://www.enterprisedb.com" target=3D"_blank=
">http://www.enterprisedb.com</a><br>The Enterprise PostgreSQL Company<br><=
br>
</font></span></blockquote></div><br></div></div>
</blockquote><br><br>-- <br>Dave Page<br>Blog: <a href=3D"http://pgsnake.bl=
ogspot.com" target=3D"_blank">http://pgsnake.blogspot.com</a><br>Twitter: @=
pgsnake<br><br>EnterpriseDB UK: <a href=3D"http://www.enterprisedb.com" tar=
get=3D"_blank">http://www.enterprisedb.com</a><br>The Enterprise PostgreSQL=
 Company<br><br>

--001a1145ecc65efc7f0550239e04--