pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedBest Prometheus exporter?
7+ messages / 3 participants
[nested] [flat]
* Best Prometheus exporter?
@ 2021-11-08 18:49 Victor Sudakov <vas@sibptus.ru>
2021-11-08 20:29 ` Re: Best Prometheus exporter? Vijaykumar Jain <vijaykumarjain.github@gmail.com>
2021-11-09 05:52 ` Re: Best Prometheus exporter? Ruohang Feng <rh@vonng.com>
0 siblings, 2 replies; 7+ messages in thread
From: Victor Sudakov @ 2021-11-08 18:49 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
Dear Colleagues,
What good Prometheus exporter for Postgres can you recommend (from your own experience)?
--
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Best Prometheus exporter?
2021-11-08 18:49 Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
@ 2021-11-08 20:29 ` Vijaykumar Jain <vijaykumarjain.github@gmail.com>
2021-11-09 10:31 ` Re: Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
1 sibling, 1 reply; 7+ messages in thread
From: Vijaykumar Jain @ 2021-11-08 20:29 UTC (permalink / raw)
To: Victor Sudakov <vas@sibptus.ru>; +Cc: Pgsql-admin <pgsql-admin@lists.postgresql.org>
On Tue, 9 Nov 2021 at 00:19, Victor Sudakov <vas@sibptus.ru> wrote:
>
> What good Prometheus exporter for Postgres can you recommend (from your
> own experience)?
I have not got a lot of exposure with prometheus, but was able to quickly
set up using below,
but I was aware of monitoring and alerting otherwise , so it was easy. the
below were combined with node_exporter (specially storage io metrics)
prometheus-community/postgres_exporter: A PostgreSQL metric exporter for
Prometheus (github.com)
<https://github.com/prometheus-community/postgres_exporter;
should be a good start.
PostgreSQL Database dashboard for Grafana | Grafana Labs
<https://grafana.com/grafana/dashboards/9628; would be the corresponding
grafana dashboard.
and alerting queries for this exporter
Awesome Prometheus alerts | Collection of alerting rules (grep.to)
<https://awesome-prometheus-alerts.grep.to/rules.html#postgresql-1;
these are very basics to start with, feel free to improvise.
the below utils are also very much handy.
NikolayS/postgres_dba: The missing set of useful tools for Postgres DBAs
and all engineers (github.com) <https://github.com/NikolayS/postgres_dba;
dataegret/pg-utils: Useful PostgreSQL utilities (github.com)
<https://github.com/dataegret/pg-utils;
pg_stat_monitor/USER_GUIDE.md at master · percona/pg_stat_monitor
(github.com)
<https://github.com/percona/pg_stat_monitor/blob/master/docs/USER_GUIDE.md;
(new but i like the option to check query plan via sql query)
<https://github.com/prometheus-community/postgres_exporter>and the most
important wrt logs.
pgBadger Documentation (darold.net)
<https://pgbadger.darold.net/documentation.html;
<https://pgbadger.darold.net/documentation.html;
I shared much more than asked, feel free to discard what is not needed.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Best Prometheus exporter?
2021-11-08 18:49 Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
2021-11-08 20:29 ` Re: Best Prometheus exporter? Vijaykumar Jain <vijaykumarjain.github@gmail.com>
@ 2021-11-09 10:31 ` Victor Sudakov <vas@sibptus.ru>
0 siblings, 0 replies; 7+ messages in thread
From: Victor Sudakov @ 2021-11-09 10:31 UTC (permalink / raw)
To: Pgsql-admin <pgsql-admin@lists.postgresql.org>; +Cc: Vijaykumar Jain <vijaykumarjain.github@gmail.com>
Hello Vijaykumar!
Thank you for your useful links. I've just installed
prometheus-community/postgres_exporter and it is working. It is
also useful that it comes with a ready Grafana dashboard. I've installed
Grafana dashboard "9628" and found it not very useful (in comparison
e.g. with the standard Zabbix PostgreSQL template), but maybe I've not
invested enough time studying it.
I have not looked at the alerter yet.
Vijaykumar Jain wrote:
> On Tue, 9 Nov 2021 at 00:19, Victor Sudakov <vas@sibptus.ru> wrote:
>
> >
> > What good Prometheus exporter for Postgres can you recommend (from your
> > own experience)?
>
>
> I have not got a lot of exposure with prometheus, but was able to quickly
> set up using below,
> but I was aware of monitoring and alerting otherwise , so it was easy. the
> below were combined with node_exporter (specially storage io metrics)
>
> prometheus-community/postgres_exporter: A PostgreSQL metric exporter for
> Prometheus (github.com)
> <https://github.com/prometheus-community/postgres_exporter;
> should be a good start.
> PostgreSQL Database dashboard for Grafana | Grafana Labs
> <https://grafana.com/grafana/dashboards/9628; would be the corresponding
> grafana dashboard.
>
> and alerting queries for this exporter
> Awesome Prometheus alerts | Collection of alerting rules (grep.to)
> <https://awesome-prometheus-alerts.grep.to/rules.html#postgresql-1;
>
> these are very basics to start with, feel free to improvise.
>
> the below utils are also very much handy.
>
> NikolayS/postgres_dba: The missing set of useful tools for Postgres DBAs
> and all engineers (github.com) <https://github.com/NikolayS/postgres_dba;
> dataegret/pg-utils: Useful PostgreSQL utilities (github.com)
> <https://github.com/dataegret/pg-utils;
> pg_stat_monitor/USER_GUIDE.md at master · percona/pg_stat_monitor
> (github.com)
> <https://github.com/percona/pg_stat_monitor/blob/master/docs/USER_GUIDE.md;
> (new but i like the option to check query plan via sql query)
>
> <https://github.com/prometheus-community/postgres_exporter>and the most
> important wrt logs.
> pgBadger Documentation (darold.net)
> <https://pgbadger.darold.net/documentation.html;
> <https://pgbadger.darold.net/documentation.html;
> I shared much more than asked, feel free to discard what is not needed.
--
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Best Prometheus exporter?
2021-11-08 18:49 Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
@ 2021-11-09 05:52 ` Ruohang Feng <rh@vonng.com>
2021-11-09 09:55 ` Re: Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
1 sibling, 1 reply; 7+ messages in thread
From: Ruohang Feng @ 2021-11-09 05:52 UTC (permalink / raw)
To: Victor Sudakov <vas@sibptus.ru>; +Cc: pgsql-admin@lists.postgresql.org
I would recommend https://github.com/Vonng/pg_exporter for some reasons:
Support both Postgres & Pgbouncer
Flexible: Almost all metrics are defined in customizable configuration files in SQL style.
Fine-grained execution control: Tags Filter, Facts Filter, Version Filter, Timeout, Cache, etc..
Dynamic Planning: you could define multiple branches for a collecto. Queries matches server version & fact & tags will be actually executed.
Configurable caching policy & query timeout
Rich metrics about pg_exporter itself.
Auto discovery multi-database in the same instance
Tested and verified in real world production environment for 3+ years (200+ Nodes)
The most important part is: it has a complete solution for monitoring & dashboards:
https://github.com/Vonng/pigsty
https://pigsty.cc <https://pigsty.cc/;
Dashboards screenshots: https://github.com/Vonng/pigsty/raw/master/docs/_media/overview-monitor.jpg
> 2021年11月9日 02:49，Victor Sudakov <vas@sibptus.ru> 写道：
>
> Dear Colleagues,
>
> What good Prometheus exporter for Postgres can you recommend (from your own experience)?
>
> --
> Victor Sudakov VAS4-RIPE
> http://vas.tomsk.ru/
> 2:5005/49@fidonet
>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Best Prometheus exporter?
2021-11-08 18:49 Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
2021-11-09 05:52 ` Re: Best Prometheus exporter? Ruohang Feng <rh@vonng.com>
@ 2021-11-09 09:55 ` Victor Sudakov <vas@sibptus.ru>
2021-11-10 06:13 ` Re: Best Prometheus exporter? Ruohang Feng <rh@vonng.com>
0 siblings, 1 reply; 7+ messages in thread
From: Victor Sudakov @ 2021-11-09 09:55 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org; +Cc: Ruohang Feng <rh@vonng.com>
Hello Ruohang!
I've installed pg_exporter (very simple setup) and indeed I'm
impressed. All the metrics are customizable and the default set of
metrics in the stock config is impressive.
You are the author so can I ask a few questions?
1. Are you providing an official docker image, or a single binary I
could just ADD to a docker image, without the additional RUN steps for
curl'ing and unpacking? An official docker image would be best of
course.
2. I don't think I'm presently going to use the whole of Pigsty, we
are already using some in-house scripts over Patroni, but a collection
of Grafana dashboards using the pg_exporter metrics is something I'd
love having. Is it possible to get them separately from Pigsty?
The exporter itsef is grand!
Ruohang Feng wrote:
> I would recommend https://github.com/Vonng/pg_exporter for some reasons:
>
> Support both Postgres & Pgbouncer
> Flexible: Almost all metrics are defined in customizable configuration files in SQL style.
> Fine-grained execution control: Tags Filter, Facts Filter, Version Filter, Timeout, Cache, etc..
> Dynamic Planning: you could define multiple branches for a collecto. Queries matches server version & fact & tags will be actually executed.
> Configurable caching policy & query timeout
> Rich metrics about pg_exporter itself.
> Auto discovery multi-database in the same instance
> Tested and verified in real world production environment for 3+ years (200+ Nodes)
>
>
> The most important part is: it has a complete solution for monitoring & dashboards:
>
> https://github.com/Vonng/pigsty
> https://pigsty.cc <https://pigsty.cc/;
>
>
> Dashboards screenshots: https://github.com/Vonng/pigsty/raw/master/docs/_media/overview-monitor.jpg
>
>
>
> > 2021年11月9日 02:49，Victor Sudakov <vas@sibptus.ru> 写道：
> >
> > Dear Colleagues,
> >
> > What good Prometheus exporter for Postgres can you recommend (from your own experience)?
> >
> > --
> > Victor Sudakov VAS4-RIPE
> > http://vas.tomsk.ru/
> > 2:5005/49@fidonet
> >
>
--
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Best Prometheus exporter?
2021-11-08 18:49 Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
2021-11-09 05:52 ` Re: Best Prometheus exporter? Ruohang Feng <rh@vonng.com>
2021-11-09 09:55 ` Re: Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
@ 2021-11-10 06:13 ` Ruohang Feng <rh@vonng.com>
2021-11-11 10:15 ` Re: Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
0 siblings, 1 reply; 7+ messages in thread
From: Ruohang Feng @ 2021-11-10 06:13 UTC (permalink / raw)
To: Victor Sudakov <vas@sibptus.ru>; +Cc: pgsql-admin@lists.postgresql.org
Hi Victor:
1.
There is no official docker image. But there's a dockerfile: https://github.com/Vonng/pg_exporter/blob/master/Dockerfile
It is a single binary that could be just ADD to a docker image, and receive parameters via ENVIRONMENT.
Maybe I would create an image in dockerhub in next minor release.
2.
You can setup monitoring staff manually with following resources:
Grafana Dashboards: https://github.com/Vonng/pigsty/tree/master/roles/grafana/files/dashboards/pgsql
Prometheus Config: https://github.com/Vonng/pigsty/blob/master/roles/prometheus/files/example/prometheus-static.yml
Prometheus Rules: https://github.com/Vonng/pigsty/blob/master/roles/prometheus/files/rules/pgsql-rules.yml
Prometheus Targets: https://github.com/Vonng/pigsty/blob/master/roles/register/tasks/prometheus.yml
PG Exporter Config: https://github.com/Vonng/pigsty/blob/master/roles/monitor/files/pg_exporter.yml
Another option is only using the monitoring part of Pigsty which could setup everything automatically.
Install pigsty on a standalone machine, and it can be used for monitoring existing external postgres instances (with a viable PGURL only).
http://demo.pigsty.cc/d/home
Regards~
> 2021年11月9日 17:55，Victor Sudakov <vas@sibptus.ru> 写道：
>
> Hello Ruohang!
>
> I've installed pg_exporter (very simple setup) and indeed I'm
> impressed. All the metrics are customizable and the default set of
> metrics in the stock config is impressive.
>
> You are the author so can I ask a few questions?
>
> 1. Are you providing an official docker image, or a single binary I
> could just ADD to a docker image, without the additional RUN steps for
> curl'ing and unpacking? An official docker image would be best of
> course.
>
> 2. I don't think I'm presently going to use the whole of Pigsty, we
> are already using some in-house scripts over Patroni, but a collection
> of Grafana dashboards using the pg_exporter metrics is something I'd
> love having. Is it possible to get them separately from Pigsty?
>
> The exporter itsef is grand!
>
> Ruohang Feng wrote:
>> I would recommend https://github.com/Vonng/pg_exporter for some reasons:
>>
>> Support both Postgres & Pgbouncer
>> Flexible: Almost all metrics are defined in customizable configuration files in SQL style.
>> Fine-grained execution control: Tags Filter, Facts Filter, Version Filter, Timeout, Cache, etc..
>> Dynamic Planning: you could define multiple branches for a collecto. Queries matches server version & fact & tags will be actually executed.
>> Configurable caching policy & query timeout
>> Rich metrics about pg_exporter itself.
>> Auto discovery multi-database in the same instance
>> Tested and verified in real world production environment for 3+ years (200+ Nodes)
>>
>>
>> The most important part is: it has a complete solution for monitoring & dashboards:
>>
>> https://github.com/Vonng/pigsty
>> https://pigsty.cc <https://pigsty.cc/; <https://pigsty.cc/ <https://pigsty.cc/>;
>>
>>
>> Dashboards screenshots: https://github.com/Vonng/pigsty/raw/master/docs/_media/overview-monitor.jpg
>>
>>
>>
>>> 2021年11月9日 02:49，Victor Sudakov <vas@sibptus.ru> 写道：
>>>
>>> Dear Colleagues,
>>>
>>> What good Prometheus exporter for Postgres can you recommend (from your own experience)?
>>>
>>> --
>>> Victor Sudakov VAS4-RIPE
>>> http://vas.tomsk.ru/
>>> 2:5005/49@fidonet
>>>
>>
>
> --
> Victor Sudakov VAS4-RIPE
> http://vas.tomsk.ru/ <http://vas.tomsk.ru/;
> 2:5005/49@fidonet
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Best Prometheus exporter?
2021-11-08 18:49 Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
2021-11-09 05:52 ` Re: Best Prometheus exporter? Ruohang Feng <rh@vonng.com>
2021-11-09 09:55 ` Re: Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
2021-11-10 06:13 ` Re: Best Prometheus exporter? Ruohang Feng <rh@vonng.com>
@ 2021-11-11 10:15 ` Victor Sudakov <vas@sibptus.ru>
0 siblings, 0 replies; 7+ messages in thread
From: Victor Sudakov @ 2021-11-11 10:15 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org; +Cc: Ruohang Feng <rh@vonng.com>
Ruohang Feng wrote:
> Hi Victor:
Hi Ruohang!
>
> 1.
>
> There is no official docker image. But there's a dockerfile: https://github.com/Vonng/pg_exporter/blob/master/Dockerfile
>
> It is a single binary that could be just ADD to a docker image, and receive parameters via ENVIRONMENT.
>
> Maybe I would create an image in dockerhub in next minor release.
That would be great because I'd be able to use the image directly from
dockerhub in my manifests, much like I use nginx, phpfpm and other
official images, without copying the binary and running `docker build`
somewhere.
> 2.
>
>
> You can setup monitoring staff manually with following resources:
>
> Grafana Dashboards: https://github.com/Vonng/pigsty/tree/master/roles/grafana/files/dashboards/pgsql
Thanks a lot. I have looked at them and decided that I'll probably
start by creating some dashboards from scratch using the metrics from
your excellent exporter.
--
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2021-11-11 10:15 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-11-08 18:49 Best Prometheus exporter? Victor Sudakov <vas@sibptus.ru>
2021-11-08 20:29 ` Vijaykumar Jain <vijaykumarjain.github@gmail.com>
2021-11-09 10:31 ` Victor Sudakov <vas@sibptus.ru>
2021-11-09 05:52 ` Ruohang Feng <rh@vonng.com>
2021-11-09 09:55 ` Victor Sudakov <vas@sibptus.ru>
2021-11-10 06:13 ` Ruohang Feng <rh@vonng.com>
2021-11-11 10:15 ` Victor Sudakov <vas@sibptus.ru>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox