public inbox for [email protected]
help / color / mirror / Atom feedFrom: Aditya Toshniwal <[email protected]>
To: Dave Page <[email protected]>
Cc: Khushboo Vashi <[email protected]>
Cc: Akshay Joshi <[email protected]>
Cc: pgadmin-hackers <[email protected]>
Subject: Re: [pgAdmin4][RM3941] Dashboard graphs needs optimizations
Date: Mon, 4 Feb 2019 15:27:06 +0530
Message-ID: <CAM9w-_kj8qVkOO7VNt3qumUHK21OvkAgE=T+YdwOEDJ07Ue-8A@mail.gmail.com> (raw)
In-Reply-To: <CA+OCxox+2nDXK9gjg9B4ZVujEuAKCUKi5dCLL=1NZiBZODAKOA@mail.gmail.com>
References: <CAM9w-_=uidEaAy=VH8VWq5YpvWSCjZFUt=WNaRt6dyhUYC6WXA@mail.gmail.com>
<CANxoLDfDJayFiSpsBf+nFW74c5VOsNgsDKfRTrLaCQe=MDOuwg@mail.gmail.com>
<CAFOhELdah8SNhc+eiqDUewNY21DmqrUr2FkPRyRcwPkXKNdoeg@mail.gmail.com>
<CA+OCxox+2nDXK9gjg9B4ZVujEuAKCUKi5dCLL=1NZiBZODAKOA@mail.gmail.com>
Hi Hackers,
Attached is the updated patch. The poller will wait 3 times the normal
interval before sending new request if the previous request is pending.
After that, it will send the ajax.
Kindly review.
On Mon, Feb 4, 2019 at 3:12 PM Dave Page <[email protected]> wrote:
> I can't get to this right now, but Akshay, can we get it in before the
> release?
>
> I don't think the problem Khushboo notes is really a problem. Waiting for
> a response before issuing another one seems sensible... as long as there's
> a timeout so things don't get stuck if a response gets lost somehow. Maybe
> the timeout should be something like 3x the normal interval?
>
> On Mon, Feb 4, 2019 at 9:02 AM Khushboo Vashi <
> [email protected]> wrote:
>
>> Hi Aditya,
>>
>> I have reviewed the patch and it looks good to me except the logic to
>> wait until we get the response from the AJAX request.
>> So, in this case at a time only one request is being sent to the server.
>>
>> @ Dave/Ashesh, if you think this is okay then we can commit the patch.
>>
>> Thanks,
>> Khushboo
>>
>> On Mon, Feb 4, 2019 at 10:45 AM Akshay Joshi <
>> [email protected]> wrote:
>>
>>> Hi Khushboo
>>>
>>> Can you please review it.
>>>
>>> On Fri, Feb 1, 2019 at 7:04 PM Aditya Toshniwal <
>>> [email protected]> wrote:
>>>
>>>> Hi Hackers,
>>>>
>>>> Attached is the patch to optimise the dashboard. Changes include:
>>>> 1) One ajax request for the all the charts instead of separate for
>>>> every chart. The next request will not hit until the previous is completed.
>>>> 2) The code will check for intervals which is greatest common divisor
>>>> of all the refresh rates. Chart names which will be eligible for refresh
>>>> will be sent with the ajax. If none are eligible, no ajax will be fired.
>>>> 3) In the backend, query is generated in such a way that data for all
>>>> the requested chart names will be fetched in one hit to the database.
>>>> 4) Test cases for charts rest api and javascript added.
>>>>
>>>> --
>>>> Thanks and Regards,
>>>> Aditya Toshniwal
>>>> Software Engineer | EnterpriseDB Software Solutions | Pune
>>>> "Don't Complain about Heat, Plant a tree"
>>>>
>>>
>>>
>>> --
>>> *Akshay Joshi*
>>>
>>> *Sr. Software Architect *
>>>
>>>
>>>
>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246*
>>>
>>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
--
Thanks and Regards,
Aditya Toshniwal
Software Engineer | EnterpriseDB Software Solutions | Pune
"Don't Complain about Heat, Plant a tree"
Attachments:
[application/octet-stream] RM3941_v2.patch (40.1K, 3-RM3941_v2.patch)
download | inline diff:
diff --git a/web/pgadmin/dashboard/__init__.py b/web/pgadmin/dashboard/__init__.py
index 9a0118c8..3688e165 100644
--- a/web/pgadmin/dashboard/__init__.py
+++ b/web/pgadmin/dashboard/__init__.py
@@ -9,7 +9,7 @@
"""A blueprint module implementing the dashboard frame."""
from functools import wraps
-from flask import render_template, url_for, Response, g
+from flask import render_template, url_for, Response, g, request
from flask_babelex import gettext
from flask_security import login_required
from pgadmin.utils import PgAdminModule
@@ -154,21 +154,9 @@ class DashboardModule(PgAdminModule):
return [
'dashboard.index', 'dashboard.get_by_sever_id',
'dashboard.get_by_database_id',
- 'dashboard.session_stats',
- 'dashboard.get_session_stats_by_sever_id',
- 'dashboard.get_session_stats_by_database_id',
- 'dashboard.tps_stats',
- 'dashboard.tps_stats_by_server_id',
- 'dashboard.tps_stats_by_database_id',
- 'dashboard.ti_stats',
- 'dashboard.ti_stats_by_server_id',
- 'dashboard.ti_stats_by_database_id',
- 'dashboard.to_stats',
- 'dashboard.to_stats_by_server_id',
- 'dashboard.to_stats_by_database_id',
- 'dashboard.bio_stats',
- 'dashboard.bio_stats_by_server_id',
- 'dashboard.bio_stats_by_database_id',
+ 'dashboard.dashboard_stats',
+ 'dashboard.dashboard_stats_sid',
+ 'dashboard.dashboard_stats_did',
'dashboard.activity',
'dashboard.get_activity_by_server_id',
'dashboard.get_activity_by_database_id',
@@ -356,87 +344,36 @@ def get_data(sid, did, template):
)
[email protected]('/session_stats/', endpoint='session_stats')
[email protected](
- '/session_stats/<int:sid>', endpoint='get_session_stats_by_sever_id'
-)
[email protected](
- '/session_stats/<int:sid>/<int:did>',
- endpoint='get_session_stats_by_database_id'
-)
[email protected]('/dashboard_stats',
+ endpoint='dashboard_stats')
[email protected]('/dashboard_stats/<int:sid>',
+ endpoint='dashboard_stats_sid')
[email protected]('/dashboard_stats/<int:sid>/<int:did>',
+ endpoint='dashboard_stats_did')
@login_required
@check_precondition
-def session_stats(sid=None, did=None):
- """
- This function returns server session statistics
- :param sid: server id
- :return:
- """
- return get_data(sid, did, 'session_stats.sql')
-
+def dashboard_stats(sid=None, did=None):
+ resp_data = {}
[email protected]('/tps_stats/', endpoint='tps_stats')
[email protected]('/tps_stats/<int:sid>', endpoint='tps_stats_by_server_id')
[email protected](
- '/tps_stats/<int:sid>/<int:did>', endpoint='tps_stats_by_database_id'
-)
-@login_required
-@check_precondition
-def tps_stats(sid=None, did=None):
- """
- This function returns server TPS throughput
- :param sid: server id
- :return:
- """
- return get_data(sid, did, 'tps_stats.sql')
+ if request.args['chart_names'] != '':
+ chart_names = request.args['chart_names'].split(',')
+ if not sid:
+ return internal_server_error(errormsg='Server ID not specified.')
[email protected]('/ti_stats/', endpoint='ti_stats')
[email protected]('/ti_stats/<int:sid>', endpoint='ti_stats_by_server_id')
[email protected](
- '/ti_stats/<int:sid>/<int:did>', endpoint='ti_stats_by_database_id'
-)
-@login_required
-@check_precondition
-def ti_stats(sid=None, did=None):
- """
- This function returns server tuple input statistics
- :param sid: server id
- :return:
- """
- return get_data(sid, did, 'ti_stats.sql')
-
-
[email protected]('/to_stats/', endpoint='to_stats')
[email protected]('/to_stats/<int:sid>', endpoint='to_stats_by_server_id')
[email protected](
- '/to_stats/<int:sid>/<int:did>', endpoint='to_stats_by_database_id'
-)
-@login_required
-@check_precondition
-def to_stats(sid=None, did=None):
- """
- This function returns server tuple output statistics
- :param sid: server id
- :return:
- """
- return get_data(sid, did, 'to_stats.sql')
+ sql = render_template(
+ "/".join([g.template_path, 'dashboard_stats.sql']), did=did,
+ chart_names=chart_names,
+ )
+ status, res = g.conn.execute_dict(sql)
+ for chart_row in res['rows']:
+ resp_data[chart_row['chart_name']] = chart_row['chart_data']
[email protected]('/bio_stats/', endpoint='bio_stats')
[email protected]('/bio_stats/<int:sid>', endpoint='bio_stats_by_server_id')
[email protected](
- '/bio_stats/<int:sid>/<int:did>', endpoint='bio_stats_by_database_id'
-)
-@login_required
-@check_precondition
-def bio_stats(sid=None, did=None):
- """
- This function returns server block IO statistics
- :param sid: server id
- :return:
- """
- return get_data(sid, did, 'bio_stats.sql')
+ return ajax_response(
+ response=resp_data,
+ status=200
+ )
@blueprint.route('/activity/', endpoint='activity')
diff --git a/web/pgadmin/dashboard/static/js/charting.js b/web/pgadmin/dashboard/static/js/charting.js
index 64d9027d..b60a02ad 100644
--- a/web/pgadmin/dashboard/static/js/charting.js
+++ b/web/pgadmin/dashboard/static/js/charting.js
@@ -68,7 +68,9 @@ export class Chart {
}
getOtherData(key) {
- return this._otherData[key];
+ if(this._otherData[key]) {
+ return this._otherData[key];
+ }
}
setOtherData(key, value) {
diff --git a/web/pgadmin/dashboard/static/js/dashboard.js b/web/pgadmin/dashboard/static/js/dashboard.js
index 3c9a43a3..67bc3335 100644
--- a/web/pgadmin/dashboard/static/js/dashboard.js
+++ b/web/pgadmin/dashboard/static/js/dashboard.js
@@ -10,11 +10,11 @@
define('pgadmin.dashboard', [
'sources/url_for', 'sources/gettext', 'require', 'jquery', 'underscore',
'sources/pgadmin', 'backbone', 'backgrid', './charting',
- 'pgadmin.alertifyjs', 'pgadmin.backform',
- 'sources/nodes/dashboard', 'pgadmin.browser', 'bootstrap', 'wcdocker',
+ 'pgadmin.alertifyjs', 'pgadmin.backform', 'sources/nodes/dashboard',
+ 'sources/utils', 'pgadmin.browser', 'bootstrap', 'wcdocker',
], function(
url_for, gettext, r, $, _, pgAdmin, Backbone, Backgrid, charting,
- Alertify, Backform, NodesDashboard
+ Alertify, Backform, NodesDashboard, commonUtils
) {
pgAdmin.Browser = pgAdmin.Browser || {};
@@ -214,8 +214,9 @@ define('pgadmin.dashboard', [
// Load the default welcome dashboard
var url = url_for('dashboard.index');
- /* Store the chart objects and there interval ids in this store */
- this.chartStore = {};
+ /* Store the chart objects, refresh freq and next refresh time */
+ this.chart_store = {};
+ this.charts_poller_int_id = -1;
var dashboardPanel = pgBrowser.panels['dashboard'].panel;
if (dashboardPanel) {
@@ -373,96 +374,178 @@ define('pgadmin.dashboard', [
}
},
- renderChartLoop: function(chartObj, sid, did, url, counter, refresh) {
- var data = [],
- dataset = [];
+ // Render the charts
+ renderCharts: function(charts_config) {
- var theIntervalFunc = function() {
- var path = url + sid;
- if (did != -1) {
- path += '/' + did;
+ let self = this,
+ tooltipFormatter = function(refresh, currVal) {
+ return(`Seconds ago: ${parseInt(currVal.x * refresh)}</br>
+ Value: ${currVal.y}`);
+ },
+ curr_epoch=commonUtils.getEpoch();
+
+ self.stopChartsPoller();
+
+ charts_config.map((chart_config) => {
+ if(self.chart_store[chart_config.chart_name]
+ && self.old_preferences[chart_config.refresh_pref_name] !=
+ self.preferences[chart_config.refresh_pref_name]) {
+ self.clearChartFromStore(chart_config.chart_name);
}
- $.ajax({
- url: path,
- type: 'GET',
- dataType: 'html',
- })
- .done(function(resp) {
- $(chartObj.getContainer()).removeClass('graph-error');
- data = JSON.parse(resp);
-
- var y = 0,
- x;
- if (dataset.length == 0) {
- if (counter == true) {
- // Have we stashed initial values?
- if (_.isUndefined(chartObj.getOtherData('counter_previous_vals'))) {
- chartObj.setOtherData('counter_previous_vals', data[0]);
- } else {
- // Create the initial data structure
- for (x in data[0]) {
- dataset.push({
- 'data': [
- [0, data[0][x] - chartObj.getOtherData('counter_previous_vals')[x]],
- ],
- 'label': x,
- });
- }
- }
- } else {
- // Create the initial data structure
- for (x in data[0]) {
- dataset.push({
- 'data': [
- [0, data[0][x]],
- ],
- 'label': x,
- });
- }
- }
- } else {
- for (x in data[0]) {
- // Push new values onto the existing data structure
- // If this is a counter stat, we need to subtract the previous value
- if (counter == false) {
- dataset[y]['data'].unshift([0, data[0][x]]);
- } else {
- // Store the current value, minus the previous one we stashed.
- // It's possible the tab has been reloaded, in which case out previous values are gone
- if (_.isUndefined(chartObj.getOtherData('counter_previous_vals')))
- return;
-
- dataset[y]['data'].unshift([0, data[0][x] - chartObj.getOtherData('counter_previous_vals')[x]]);
- }
- // Reset the time index to get a proper scrolling display
- for (var z = 0; z < dataset[y]['data'].length; z++) {
- dataset[y]['data'][z][0] = z;
- }
+ if(self.chart_store[chart_config.chart_name]) {
+ let chart_obj = self.chart_store[chart_config.chart_name].chart_obj;
+ chart_obj.setOptions(chart_config.options, false);
+ chart_obj.setTooltipFormatter(
+ tooltipFormatter.bind(null, self.preferences[chart_config.refresh_pref_name])
+ );
+ }
- y++;
- }
- chartObj.setOtherData('counter_previous_vals', data[0]);
+ if(!self.chart_store[chart_config.chart_name]) {
+ let chart_obj = new charting.Chart(chart_config.container, chart_config.options);
+
+ chart_obj.setTooltipFormatter(
+ tooltipFormatter.bind(null, self.preferences[chart_config.refresh_pref_name])
+ );
+
+ chart_obj.setOtherData('counter', chart_config.counter);
+
+ self.chart_store[chart_config.chart_name] = {
+ 'chart_obj' : chart_obj,
+ 'refresh_on': curr_epoch,
+ 'refresh_rate': self.preferences[chart_config.refresh_pref_name],
+ };
+ }
+ });
+
+ self.startChartsPoller(self.chart_store, self.sid, self.did);
+ },
+
+ getStatsUrl: function(sid=-1, did=-1, chart_names=[]) {
+ let base_url = url_for('dashboard.dashboard_stats');
+ base_url += '/' + sid;
+ base_url += (did > 0) ? ('/' + did) : '';
+ base_url += '?chart_names=' + chart_names.join(',');
+ return base_url;
+ },
+
+ updateChart: function(chart_obj, new_data){
+ // Dataset format:
+ // [
+ // { data: [[0, y0], [1, y1]...], label: 'Label 1', [options] },
+ // { data: [[0, y0], [1, y1]...], label: 'Label 2', [options] },
+ // { data: [[0, y0], [1, y1]...], label: 'Label 3', [options] }
+ // ]
+ let dataset = chart_obj.getOtherData('dataset') || [],
+ counter_prev_data = chart_obj.getOtherData('counter_prev_data') || new_data,
+ counter = chart_obj.getOtherData('counter') || false;
+
+ if (dataset.length == 0) {
+ // Create the initial data structure
+ for (let label in new_data) {
+ dataset.push({
+ 'data': [
+ [0, counter ? (new_data[label] - counter_prev_data[label]) : new_data[label]],
+ ],
+ 'label': label,
+ });
+ }
+ } else {
+ Object.keys(new_data).map((label, label_ind) => {
+ // Push new values onto the existing data structure
+ // If this is a counter stat, we need to subtract the previous value
+ if (!counter) {
+ dataset[label_ind]['data'].unshift([0, new_data[label]]);
+ } else {
+ // Store the current value, minus the previous one we stashed.
+ // It's possible the tab has been reloaded, in which case out previous values are gone
+ if (_.isUndefined(counter_prev_data))
+ return;
+
+ dataset[label_ind]['data'].unshift([0, new_data[label] - counter_prev_data[label]]);
}
- // Remove uneeded elements
- for (x = 0; x < dataset.length; x++) {
- // Remove old data points
- if (dataset[x]['data'].length > 101) {
- dataset[x]['data'].pop();
- }
+ // Reset the time index to get a proper scrolling display
+ for (var time_ind = 0; time_ind < dataset[label_ind]['data'].length; time_ind++) {
+ dataset[label_ind]['data'][time_ind][0] = time_ind;
}
+ });
+ counter_prev_data = new_data;
+ }
- if (chartObj.isInPage()) {
- if (chartObj.isVisible()) {
- chartObj.draw(dataset);
- }
- } else {
- return;
+ // Remove old data points
+ for (let label_ind = 0; label_ind < dataset.length; label_ind++) {
+ if (dataset[label_ind]['data'].length > 101) {
+ dataset[label_ind]['data'].pop();
+ }
+ }
+
+ chart_obj.setOtherData('dataset', dataset);
+ chart_obj.setOtherData('counter_prev_data', counter_prev_data);
+
+ if (chart_obj.isInPage()) {
+ if (chart_obj.isVisible()) {
+ chart_obj.draw(dataset);
+ }
+ } else {
+ return;
+ }
+ },
+
+ stopChartsPoller: function() {
+ clearInterval(this.charts_poller_int_id);
+ },
+
+ startChartsPoller: function(chart_store, sid, did) {
+ let self = this;
+ /* polling will the greatest common divisor of the refresh rates*/
+ let poll_interval = commonUtils.getGCD(
+ Object.values(chart_store).map(item => item.refresh_rate)
+ );
+ const WAIT_COUNTER = 3;
+ let last_poll_wait_counter = WAIT_COUNTER;
+
+ /* Stop if running, only one poller lives */
+ self.stopChartsPoller();
+
+ var thePollingFunc = function() {
+ let curr_epoch = commonUtils.getEpoch();
+ let chart_names_to_get = [];
+
+ for(let chart_name in chart_store) {
+ /* when its time to get the data */
+ if(chart_store[chart_name].refresh_on <= curr_epoch) {
+ /* set the next trigger point */
+ chart_store[chart_name].refresh_on = curr_epoch + chart_store[chart_name].refresh_rate;
+ chart_names_to_get.push(chart_name);
}
+ }
+ /* If none of the chart wants data, don't trouble
+ * If response not received from prev poll, don't trouble !!
+ */
+ if(chart_names_to_get.length == 0 || last_poll_wait_counter > 0) {
+ /* reduce the number of tries, request should be sent if last_poll_wait_counter
+ * completes WAIT_COUNTER times.*/
+ last_poll_wait_counter--;
+ return;
+ }
+
+ var path = self.getStatsUrl(sid, did, chart_names_to_get);
+ $.ajax({
+ url: path,
+ type: 'GET',
+ })
+ .done(function(resp) {
+ last_poll_wait_counter = 0;
+ for(let chart_name in resp) {
+ let chart_obj = chart_store[chart_name].chart_obj;
+ $(chart_obj.getContainer()).removeClass('graph-error');
+ self.updateChart(chart_obj, resp[chart_name]);
+ }
})
.fail(function(xhr) {
+ last_poll_wait_counter = 0;
let err = '';
let msg = '';
let cls = 'info';
@@ -484,66 +567,19 @@ define('pgadmin.dashboard', [
}
}
- $(chartObj.getContainer()).addClass('graph-error');
- $(chartObj.getContainer()).html(
- '<div class="alert alert-' + cls + ' pg-panel-message" role="alert">' + msg + '</div>'
- );
+ for(let chart_name in chart_store) {
+ let chart_obj = chart_store[chart_name].chart_obj;
+ $(chart_obj.getContainer()).addClass('graph-error');
+ $(chart_obj.getContainer()).html(
+ '<div class="alert alert-' + cls + ' pg-panel-message" role="alert">' + msg + '</div>'
+ );
+ }
});
+ last_poll_wait_counter = WAIT_COUNTER;
};
/* Execute once for the first time as setInterval will not do */
- theIntervalFunc();
- return setInterval(theIntervalFunc, refresh * 1000);
- },
-
- // Render a chart
- render_chart: function(
- container, url, options, counter, chartName, prefName
- ) {
-
- // Data format:
- // [
- // { data: [[0, y0], [1, y1]...], label: 'Label 1', [options] },
- // { data: [[0, y0], [1, y1]...], label: 'Label 2', [options] },
- // { data: [[0, y0], [1, y1]...], label: 'Label 3', [options] }
- // ]
-
- let self = this,
- tooltipFormatter = function(refresh, currVal) {
- return(`Seconds ago: ${parseInt(currVal.x * refresh)}</br>
- Value: ${currVal.y}`);
- };
-
- if(self.chartStore[chartName]
- && self.old_preferences[prefName] != self.preferences[prefName]) {
- self.clearChartFromStore(chartName);
- }
-
- if(self.chartStore[chartName]) {
- let chartObj = self.chartStore[chartName].chartObj;
- chartObj.setOptions(options, false);
- chartObj.setTooltipFormatter(
- tooltipFormatter.bind(null, self.preferences[prefName])
- );
- }
-
- if(!self.chartStore[chartName]) {
-
- let chartObj = new charting.Chart(container, options);
-
- chartObj.setTooltipFormatter(
- tooltipFormatter.bind(null, self.preferences[prefName])
- );
-
- self.chartStore[chartName] = {
- 'chartObj' : chartObj,
- 'intervalId' : undefined,
- };
-
- self.chartStore[chartName]['intervalId'] = self.renderChartLoop(
- self.chartStore[chartName]['chartObj'], self.sid, self.did, url,
- counter, self.preferences[prefName]
- );
- }
+ thePollingFunc();
+ self.charts_poller_int_id = setInterval(thePollingFunc, poll_interval * 1000);
},
// Handler function to support the "Add Server" link
@@ -683,14 +719,13 @@ define('pgadmin.dashboard', [
clearChartFromStore: function(chartName) {
var self = this;
if(!chartName){
- _.each(self.chartStore, function(chart, key) {
- clearInterval(chart.intervalId);
- delete self.chartStore[key];
+ self.stopChartsPoller();
+ _.each(self.chart_store, function(chart, key) {
+ delete self.chart_store[key];
});
}
else {
- clearInterval(self.chartStore[chartName].intervalId);
- delete self.chartStore[chartName];
+ delete self.chart_store[chartName];
}
},
@@ -797,26 +832,37 @@ define('pgadmin.dashboard', [
if(self.preferences.show_graphs) {
// Render the graphs
- pgAdmin.Dashboard.render_chart(
- div_sessions, url_for('dashboard.session_stats'), options_line, false,
- 'session_stats', 'session_stats_refresh'
- );
- pgAdmin.Dashboard.render_chart(
- div_tps, url_for('dashboard.tps_stats'), options_line, true,
- 'tps_stats','tps_stats_refresh'
- );
- pgAdmin.Dashboard.render_chart(
- div_ti, url_for('dashboard.ti_stats'), options_line, true,
- 'ti_stats', 'ti_stats_refresh'
- );
- pgAdmin.Dashboard.render_chart(
- div_to, url_for('dashboard.to_stats'), options_line, true,
- 'to_stats','to_stats_refresh'
- );
- pgAdmin.Dashboard.render_chart(
- div_bio, url_for('dashboard.bio_stats'), options_line, true,
- 'bio_stats','bio_stats_refresh'
- );
+ pgAdmin.Dashboard.renderCharts([{
+ chart_name: 'session_stats',
+ container: div_sessions,
+ options: options_line,
+ counter: false,
+ refresh_pref_name: 'session_stats_refresh',
+ }, {
+ chart_name: 'tps_stats',
+ container: div_tps,
+ options: options_line,
+ counter: true,
+ refresh_pref_name: 'tps_stats_refresh',
+ }, {
+ chart_name: 'ti_stats',
+ container: div_ti,
+ options: options_line,
+ counter: true,
+ refresh_pref_name: 'ti_stats_refresh',
+ }, {
+ chart_name: 'to_stats',
+ container: div_to,
+ options: options_line,
+ counter: true,
+ refresh_pref_name: 'to_stats_refresh',
+ }, {
+ chart_name: 'bio_stats',
+ container: div_bio,
+ options: options_line,
+ counter: true,
+ refresh_pref_name: 'bio_stats_refresh',
+ }]);
}
if(!self.preferences.show_graphs && !self.preferences.show_activity) {
diff --git a/web/pgadmin/dashboard/templates/dashboard/sql/default/bio_stats.sql b/web/pgadmin/dashboard/templates/dashboard/sql/default/bio_stats.sql
deleted file mode 100644
index 1427e1a7..00000000
--- a/web/pgadmin/dashboard/templates/dashboard/sql/default/bio_stats.sql
+++ /dev/null
@@ -1,4 +0,0 @@
-/*pga4dash*/
-SELECT
- (SELECT sum(blks_read) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Reads') }}",
- (SELECT sum(blks_hit) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Hits') }}"
diff --git a/web/pgadmin/dashboard/templates/dashboard/sql/default/dashboard_stats.sql b/web/pgadmin/dashboard/templates/dashboard/sql/default/dashboard_stats.sql
new file mode 100644
index 00000000..5660c230
--- /dev/null
+++ b/web/pgadmin/dashboard/templates/dashboard/sql/default/dashboard_stats.sql
@@ -0,0 +1,56 @@
+{% set add_union = false %}
+{% if 'session_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'session_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT count(*) FROM pg_stat_activity{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Total') }}",
+ (SELECT count(*) FROM pg_stat_activity WHERE state = 'active'{% if did %} AND datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Active') }}",
+ (SELECT count(*) FROM pg_stat_activity WHERE state = 'idle'{% if did %} AND datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Idle') }}"
+) t
+{% endif %}
+{% if add_union and 'tps_stats' in chart_names %}
+UNION ALL
+{% endif %}
+{% if 'tps_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'tps_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT sum(xact_commit) + sum(xact_rollback) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Transactions') }}",
+ (SELECT sum(xact_commit) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Commits') }}",
+ (SELECT sum(xact_rollback) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Rollbacks') }}"
+) t
+{% endif %}
+{% if add_union and 'ti_stats' in chart_names %}
+UNION ALL
+{% endif %}
+{% if 'ti_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'ti_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT sum(tup_inserted) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Inserts') }}",
+ (SELECT sum(tup_updated) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Updates') }}",
+ (SELECT sum(tup_deleted) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Deletes') }}"
+) t
+{% endif %}
+{% if add_union and 'to_stats' in chart_names %}
+UNION ALL
+{% endif %}
+{% if 'to_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'to_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT sum(tup_fetched) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Fetched') }}",
+ (SELECT sum(tup_returned) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Returned') }}"
+) t
+{% endif %}
+{% if add_union and 'bio_stats' in chart_names %}
+UNION ALL
+{% endif %}
+{% if 'bio_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'bio_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT sum(blks_read) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Reads') }}",
+ (SELECT sum(blks_hit) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Hits') }}"
+) t
+{% endif %}
diff --git a/web/pgadmin/dashboard/templates/dashboard/sql/default/session_stats.sql b/web/pgadmin/dashboard/templates/dashboard/sql/default/session_stats.sql
deleted file mode 100644
index 3ea6a577..00000000
--- a/web/pgadmin/dashboard/templates/dashboard/sql/default/session_stats.sql
+++ /dev/null
@@ -1,5 +0,0 @@
-/*pga4dash*/
-SELECT
- (SELECT count(*) FROM pg_stat_activity{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Total') }}",
- (SELECT count(*) FROM pg_stat_activity WHERE state = 'active'{% if did %} AND datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Active') }}",
- (SELECT count(*) FROM pg_stat_activity WHERE state = 'idle'{% if did %} AND datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Idle') }}"
diff --git a/web/pgadmin/dashboard/templates/dashboard/sql/default/ti_stats.sql b/web/pgadmin/dashboard/templates/dashboard/sql/default/ti_stats.sql
deleted file mode 100644
index 5f976082..00000000
--- a/web/pgadmin/dashboard/templates/dashboard/sql/default/ti_stats.sql
+++ /dev/null
@@ -1,5 +0,0 @@
-/*pga4dash*/
-SELECT
- (SELECT sum(tup_inserted) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Inserts') }}",
- (SELECT sum(tup_updated) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Updates') }}",
- (SELECT sum(tup_deleted) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Deletes') }}"
diff --git a/web/pgadmin/dashboard/templates/dashboard/sql/default/to_stats.sql b/web/pgadmin/dashboard/templates/dashboard/sql/default/to_stats.sql
deleted file mode 100644
index 2f5a3bef..00000000
--- a/web/pgadmin/dashboard/templates/dashboard/sql/default/to_stats.sql
+++ /dev/null
@@ -1,4 +0,0 @@
-/*pga4dash*/
-SELECT
- (SELECT sum(tup_fetched) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Fetched') }}",
- (SELECT sum(tup_returned) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Returned') }}"
diff --git a/web/pgadmin/dashboard/templates/dashboard/sql/default/tps_stats.sql b/web/pgadmin/dashboard/templates/dashboard/sql/default/tps_stats.sql
deleted file mode 100644
index f95f86f5..00000000
--- a/web/pgadmin/dashboard/templates/dashboard/sql/default/tps_stats.sql
+++ /dev/null
@@ -1,5 +0,0 @@
-/*pga4dash*/
-SELECT
- (SELECT sum(xact_commit) + sum(xact_rollback) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Transactions') }}",
- (SELECT sum(xact_commit) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Commits') }}",
- (SELECT sum(xact_rollback) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Rollbacks') }}"
diff --git a/web/pgadmin/dashboard/templates/dashboard/sql/gpdb_5.0_plus/dashboard_stats.sql b/web/pgadmin/dashboard/templates/dashboard/sql/gpdb_5.0_plus/dashboard_stats.sql
new file mode 100644
index 00000000..7670547c
--- /dev/null
+++ b/web/pgadmin/dashboard/templates/dashboard/sql/gpdb_5.0_plus/dashboard_stats.sql
@@ -0,0 +1,56 @@
+{% set add_union = false %}
+{% if 'session_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'session_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT count(*) FROM pg_stat_activity{% if did %} WHERE datid = {{ did }} {% endif %}) AS "{{ _('Total') }}",
+ (SELECT count(*) FROM pg_stat_activity WHERE current_query NOT LIKE '<IDLE>%'{% if did %} AND datid = {{ did }} {% endif %}) AS "{{ _('Active') }}",
+ (SELECT count(*) FROM pg_stat_activity WHERE current_query LIKE '<IDLE>%'{% if did %} AND datid = {{ did }} {% endif %}) AS "{{ _('Idle') }}"
+) t
+{% endif %}
+{% if add_union and 'tps_stats' in chart_names %}
+UNION ALL
+{% endif %}
+{% if 'tps_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'tps_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT sum(xact_commit) + sum(xact_rollback) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Transactions') }}",
+ (SELECT sum(xact_commit) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Commits') }}",
+ (SELECT sum(xact_rollback) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Rollbacks') }}"
+) t
+{% endif %}
+{% if add_union and 'ti_stats' in chart_names %}
+UNION ALL
+{% endif %}
+{% if 'ti_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'ti_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT sum(tup_inserted) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Inserts') }}",
+ (SELECT sum(tup_updated) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Updates') }}",
+ (SELECT sum(tup_deleted) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Deletes') }}"
+) t
+{% endif %}
+{% if add_union and 'to_stats' in chart_names %}
+UNION ALL
+{% endif %}
+{% if 'to_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'to_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT sum(tup_fetched) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Fetched') }}",
+ (SELECT sum(tup_returned) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Returned') }}"
+) t
+{% endif %}
+{% if add_union and 'bio_stats' in chart_names %}
+UNION ALL
+{% endif %}
+{% if 'bio_stats' in chart_names %}
+{% set add_union = true %}
+SELECT 'bio_stats' AS chart_name, row_to_json(t) AS chart_data
+FROM (SELECT
+ (SELECT sum(blks_read) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Reads') }}",
+ (SELECT sum(blks_hit) FROM pg_stat_database{% if did %} WHERE datname = (SELECT datname FROM pg_database WHERE oid = {{ did }}){% endif %}) AS "{{ _('Hits') }}"
+) t
+{% endif %}
diff --git a/web/pgadmin/dashboard/templates/dashboard/sql/gpdb_5.0_plus/session_stats.sql b/web/pgadmin/dashboard/templates/dashboard/sql/gpdb_5.0_plus/session_stats.sql
deleted file mode 100644
index e3f24e7c..00000000
--- a/web/pgadmin/dashboard/templates/dashboard/sql/gpdb_5.0_plus/session_stats.sql
+++ /dev/null
@@ -1,5 +0,0 @@
-/*pga4dash*/
-SELECT
- (SELECT count(*) FROM pg_stat_activity{% if did %} WHERE datid = {{ did }} {% endif %}) AS "{{ _('Total') }}",
- (SELECT count(*) FROM pg_stat_activity WHERE current_query NOT LIKE '<IDLE>%'{% if did %} AND datid = {{ did }} {% endif %}) AS "{{ _('Active') }}",
- (SELECT count(*) FROM pg_stat_activity WHERE current_query LIKE '<IDLE>%'{% if did %} AND datid = {{ did }} {% endif %}) AS "{{ _('Idle') }}"
diff --git a/web/pgadmin/dashboard/tests/test_dashboard_graphs.py b/web/pgadmin/dashboard/tests/test_dashboard_graphs.py
new file mode 100644
index 00000000..90380c2f
--- /dev/null
+++ b/web/pgadmin/dashboard/tests/test_dashboard_graphs.py
@@ -0,0 +1,119 @@
+##########################################################################
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2019, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##########################################################################
+
+from pgadmin.utils.route import BaseTestGenerator
+from pgadmin.utils import server_utils as server_utils
+import simplejson as json
+
+
+class DashboardGraphsTestCase(BaseTestGenerator):
+ """
+ This class validates the version in range functionality
+ by defining different version scenarios; where dict of
+ parameters describes the scenario appended by test name.
+ """
+
+ scenarios = [(
+ 'TestCase for session_stats graph', dict(
+ sid=1,
+ did=-1,
+ chart_data={
+ 'session_stats': ['Total', 'Active', 'Idle'],
+ }
+ )), (
+ 'TestCase for tps_stats graph', dict(
+ sid=1,
+ did=-1,
+ chart_data={
+ 'tps_stats': ['Transactions', 'Commits', 'Rollbacks'],
+ }
+ )), (
+ 'TestCase for ti_stats graph', dict(
+ sid=1,
+ did=-1,
+ chart_data={
+ 'ti_stats': ['Inserts', 'Updates', 'Deletes'],
+ }
+ )), (
+ 'TestCase for to_stats graph', dict(
+ sid=1,
+ did=-1,
+ chart_data={
+ 'to_stats': ['Fetched', 'Returned'],
+ }
+ )), (
+ 'TestCase for bio_stats graph', dict(
+ sid=1,
+ did=-1,
+ chart_data={
+ 'bio_stats': ['Reads', 'Hits'],
+ }
+ )), (
+ 'TestCase for two graphs', dict(
+ sid=1,
+ did=-1,
+ chart_data={
+ 'session_stats': ['Total', 'Active', 'Idle'],
+ 'bio_stats': ['Reads', 'Hits'],
+ }
+ )), (
+ 'TestCase for five graphs', dict(
+ sid=1,
+ did=-1,
+ chart_data={
+ 'session_stats': ['Total', 'Active', 'Idle'],
+ 'tps_stats': ['Transactions', 'Commits', 'Rollbacks'],
+ 'ti_stats': ['Inserts', 'Updates', 'Deletes'],
+ 'to_stats': ['Fetched', 'Returned'],
+ 'bio_stats': ['Reads', 'Hits'],
+ }
+ )), (
+ 'TestCase for no graph', dict(
+ sid=1,
+ did=-1,
+ chart_data={},
+ ))
+ ]
+
+ def setUp(self):
+ pass
+
+ def getStatsUrl(self, sid=-1, did=-1, chart_names=''):
+ base_url = '/dashboard/dashboard_stats'
+ base_url = base_url + '/' + str(sid)
+ base_url += '/' + str(did) if did > 0 else ''
+ base_url += '?chart_names=' + chart_names
+ return base_url
+
+ def runTest(self):
+ server_response = server_utils.connect_server(self, self.sid)
+ if server_response["info"] == "Server connected.":
+
+ url = self.getStatsUrl(self.sid, self.did,
+ ",".join(self.chart_data.keys()))
+ response = self.tester.get(url)
+ self.assertEquals(response.status_code, 200)
+
+ resp_data = json.loads(response.data)
+
+ # All requested charts received
+ self.assertEquals(len(resp_data.keys()),
+ len(self.chart_data.keys()))
+
+ # All requested charts data received
+ for chart_name, chart_vals in self.chart_data.items():
+ self.assertEquals(set(resp_data[chart_name].keys()),
+ set(chart_vals))
+
+ else:
+ raise Exception("Error while connecting server to add the"
+ " database.")
+
+ def tearDown(self):
+ pass
diff --git a/web/pgadmin/static/js/utils.js b/web/pgadmin/static/js/utils.js
index 27b999f7..6cbb68a5 100644
--- a/web/pgadmin/static/js/utils.js
+++ b/web/pgadmin/static/js/utils.js
@@ -46,3 +46,27 @@ let isString = (str) => (_.isString(str));
export {
isValidData, isFunction, isString,
};
+
+export function getEpoch(inp_date) {
+ let date_obj = inp_date ? inp_date : new Date();
+ return parseInt(date_obj.getTime()/1000);
+}
+
+/* Eucladian GCD */
+export function getGCD(inp_arr) {
+ let gcd_for_two = (a, b) => {
+ return a == 0?b:gcd_for_two(b % a, a);
+ };
+
+ let inp_len = inp_arr.length;
+ if(inp_len <= 2) {
+ return gcd_for_two(inp_arr[0], inp_arr[1]);
+ }
+
+ let result = inp_arr[0];
+ for(let i=1; i<inp_len; i++) {
+ result = gcd_for_two(inp_arr[i], result);
+ }
+
+ return result;
+}
\ No newline at end of file
diff --git a/web/regression/javascript/dashboard/charting_spec.js b/web/regression/javascript/dashboard/charting_spec.js
index 8afd7a84..9a0d8709 100644
--- a/web/regression/javascript/dashboard/charting_spec.js
+++ b/web/regression/javascript/dashboard/charting_spec.js
@@ -73,6 +73,10 @@ describe('In charting related testcases', ()=> {
expect(chartObj.getOtherData('some_val')).toEqual(1);
});
+ it('Check if other data returns undefined for not set', ()=>{
+ expect(chartObj.getOtherData('some_val_not_set')).toBe(undefined);
+ });
+
it('Check if isVisible returns correct', ()=>{
let dimSpy = spyOn(chartObj, 'getContainerDimensions');
diff --git a/web/regression/javascript/pgadmin_utils_spec.js b/web/regression/javascript/pgadmin_utils_spec.js
new file mode 100644
index 00000000..341376ad
--- /dev/null
+++ b/web/regression/javascript/pgadmin_utils_spec.js
@@ -0,0 +1,35 @@
+/////////////////////////////////////////////////////////////
+//
+// pgAdmin 4 - PostgreSQL Tools
+//
+// Copyright (C) 2013 - 2019, The pgAdmin Development Team
+// This software is released under the PostgreSQL Licence
+//
+//////////////////////////////////////////////////////////////
+
+import { getEpoch, getGCD } from 'sources/utils';
+
+describe('getEpoch', function () {
+ it('should return non zero', function () {
+ expect(getEpoch()).toBeGreaterThan(0);
+ });
+
+ it('should return epoch for a date passed', function () {
+ let someDate = new Date(2019,1,1,10,20,30,40),
+ someDateEpoch = 1548996630;
+
+ expect(getEpoch(new Date(someDate))).toEqual(someDateEpoch);
+ });
+});
+
+describe('getGCD', function () {
+ it('for two numbers', function () {
+ let nos = [5, 10];
+ expect(getGCD(nos)).toEqual(5);
+ });
+
+ it('for more than two numbers', function () {
+ let nos = [9, 24, 33];
+ expect(getGCD(nos)).toEqual(3);
+ });
+});
view thread (12+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: [pgAdmin4][RM3941] Dashboard graphs needs optimizations
In-Reply-To: <CAM9w-_kj8qVkOO7VNt3qumUHK21OvkAgE=T+YdwOEDJ07Ue-8A@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox