pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedContinuous Performance Test
5+ messages / 2 participants
[nested] [flat]
* Continuous Performance Test
@ 2019-08-14 16:18 Jean Baro <jfbaro@gmail.com>
2019-08-14 17:56 ` Continuous Performance Test Jean Baro <jfbaro@gmail.com>
2019-08-14 20:57 ` Re: Continuous Performance Test Nasby, Jim <nasbyj@amazon.com>
0 siblings, 2 replies; 5+ messages in thread
From: Jean Baro @ 2019-08-14 16:18 UTC (permalink / raw)
To: pgsql-performance
Hello!
I was wondering if there is any external tool or extention to PostgreSQL >
10 (EC2 or RDS) which would:
- Collect the TOP X statements (queries or DML) used in the last Y days
IN PRODUCTION;
- Everytime there is a change in the DATABASE Structure (Migrations,
properties, Tables, Indexes) run these TOP X statements against a
PERFORMANCE TEST DATABASE (Which would be very similar to production, on
both HARDWARE and DATABASE SIZE)
- Collect the metrics for these tests
- Plot the results over time so that the PRODUCT team can be sure how
the performance of the DATABASE is being affected by the chances over time
(also as the number of rows changes). Something like this:
[image: image.png]
Query vs new change to the DATABASE (+ its size). So that we can see which
migration affect the database the most.
Is there such a thing? It doesn't need to be open source nor free. But it
would be good if it is :)
If the answer is "NO", would it be possible to create such a tool and make
it available as OOS? I think so, but I would need some professional help
(and would pay for this).
Thanks
Attachments:
[image/png] image.png (136.0K, ../../CA+fQeenoiMsgHMtKcJxEnccmQzrpXzFjCkJPHqUFaJXf7Oz-vQ@mail.gmail.com/3-image.png)
download | view image
^ permalink raw reply [nested|flat] 5+ messages in thread
* Continuous Performance Test
2019-08-14 16:18 Continuous Performance Test Jean Baro <jfbaro@gmail.com>
@ 2019-08-14 17:56 ` Jean Baro <jfbaro@gmail.com>
1 sibling, 0 replies; 5+ messages in thread
From: Jean Baro @ 2019-08-14 17:56 UTC (permalink / raw)
To: pgsql-performance@lists.postgresql.org
---------- Forwarded message ---------
From: Jean Baro <jfbaro@gmail.com>
Date: Wed, 14 Aug 2019, 13:18
Subject: Continuous Performance Test
To: <pgsql-performance@postgresql.org>
Hello!
I was wondering if there is any external tool or extention to PostgreSQL >
10 (EC2 or RDS) which would:
- Collect the TOP X statements (queries or DML) used in the last Y days
IN PRODUCTION;
- Everytime there is a change in the DATABASE Structure (Migrations,
properties, Tables, Indexes) run these TOP X statements against a
PERFORMANCE TEST DATABASE (Which would be very similar to production, on
both HARDWARE and DATABASE SIZE)
- Collect the metrics for these tests
- Plot the results over time so that the PRODUCT team can be sure how
the performance of the DATABASE is being affected by the chances over time
(also as the number of rows changes). Something like this:
[image: image.png]
Query vs new change to the DATABASE (+ its size). So that we can see which
migration affect the database the most.
Is there such a thing? It doesn't need to be open source nor free. But it
would be good if it is :)
If the answer is "NO", would it be possible to create such a tool and make
it available as OOS? I think so, but I would need some professional help
(and would pay for this).
Thanks
Attachments:
[image/png] image.png (136.0K, ../../CA+fQeekJfyS9Ka=BdJvMc4wct=+Y3cjMwVU678cUMuSQB2HSXg@mail.gmail.com/3-image.png)
download | view image
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Continuous Performance Test
2019-08-14 16:18 Continuous Performance Test Jean Baro <jfbaro@gmail.com>
@ 2019-08-14 20:57 ` Nasby, Jim <nasbyj@amazon.com>
1 sibling, 0 replies; 5+ messages in thread
From: Nasby, Jim @ 2019-08-14 20:57 UTC (permalink / raw)
To: Jean Baro <jfbaro@gmail.com>; pgsql-performance
I’m not aware of such a tool, but it would certainly be useful.
Instead of trying to tie this directly to production changes (at which point it’s already too late), I encourage you to tie it into your QA/deployment process. If you have a reliable benchmark (a *great* accomplishment!) then you should run that as part of your pre-deployment process. Obviously that won’t reflect ad-hoc changes to your production environment, but frankly you don’t want ad-hoc in production except in an emergency (and even then it’s best to have a formal production change deployment process so you can track exactly what happened when).
It does occur to me that it would be handy to be able to flag CloudWatch, Performance Insights, and Enhanced Monitoring when a deployment is done. That would make it easier to see the actual impact of a deployment on production. Let me know if you’re interested in doing a PFR for that.
From: Jean Baro <jfbaro@gmail.com>
Date: Wednesday, August 14, 2019 at 11:19 AM
To: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Subject: Continuous Performance Test
Hello!
I was wondering if there is any external tool or extention to PostgreSQL > 10 (EC2 or RDS) which would:
* Collect the TOP X statements (queries or DML) used in the last Y days IN PRODUCTION;
* Everytime there is a change in the DATABASE Structure (Migrations, properties, Tables, Indexes) run these TOP X statements against a PERFORMANCE TEST DATABASE (Which would be very similar to production, on both HARDWARE and DATABASE SIZE)
* Collect the metrics for these tests
* Plot the results over time so that the PRODUCT team can be sure how the performance of the DATABASE is being affected by the chances over time (also as the number of rows changes). Something like this:
[cid:image001.png@01D552B8.E888D7C0]
Query vs new change to the DATABASE (+ its size). So that we can see which migration affect the database the most.
Is there such a thing? It doesn't need to be open source nor free. But it would be good if it is :)
If the answer is "NO", would it be possible to create such a tool and make it available as OOS? I think so, but I would need some professional help (and would pay for this).
Thanks
Attachments:
[image/png] image001.png (136.0K, ../../E392BF90-9BAE-4C81-A9EE-E107F5F3B89B@amazon.com/3-image001.png)
download | view image
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: [UNVERIFIED SENDER] Re: Continuous Performance Test
@ 2019-08-14 20:59 Nasby, Jim <nasbyj@amazon.com>
2019-08-16 01:33 ` Re: [UNVERIFIED SENDER] Re: Continuous Performance Test Jean Baro <jfbaro@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Nasby, Jim @ 2019-08-14 20:59 UTC (permalink / raw)
To: Jean Baro <jfbaro@gmail.com>; pgsql-performance
Ooops, looks like I need to add email rules for this community list! You can ignore my AWS-specific comments in the last paragraph. 😊
From: "Nasby, Jim" <nasbyj@amazon.com>
Date: Wednesday, August 14, 2019 at 3:57 PM
To: Jean Baro <jfbaro@gmail.com>, "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Subject: [UNVERIFIED SENDER] Re: Continuous Performance Test
I’m not aware of such a tool, but it would certainly be useful.
Instead of trying to tie this directly to production changes (at which point it’s already too late), I encourage you to tie it into your QA/deployment process. If you have a reliable benchmark (a *great* accomplishment!) then you should run that as part of your pre-deployment process. Obviously that won’t reflect ad-hoc changes to your production environment, but frankly you don’t want ad-hoc in production except in an emergency (and even then it’s best to have a formal production change deployment process so you can track exactly what happened when).
It does occur to me that it would be handy to be able to flag CloudWatch, Performance Insights, and Enhanced Monitoring when a deployment is done. That would make it easier to see the actual impact of a deployment on production. Let me know if you’re interested in doing a PFR for that.
From: Jean Baro <jfbaro@gmail.com>
Date: Wednesday, August 14, 2019 at 11:19 AM
To: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Subject: Continuous Performance Test
Hello!
I was wondering if there is any external tool or extention to PostgreSQL > 10 (EC2 or RDS) which would:
* Collect the TOP X statements (queries or DML) used in the last Y days IN PRODUCTION;
* Everytime there is a change in the DATABASE Structure (Migrations, properties, Tables, Indexes) run these TOP X statements against a PERFORMANCE TEST DATABASE (Which would be very similar to production, on both HARDWARE and DATABASE SIZE)
* Collect the metrics for these tests
* Plot the results over time so that the PRODUCT team can be sure how the performance of the DATABASE is being affected by the chances over time (also as the number of rows changes). Something like this:
[cid:image001.png@01D552B9.4A594750]
Query vs new change to the DATABASE (+ its size). So that we can see which migration affect the database the most.
Is there such a thing? It doesn't need to be open source nor free. But it would be good if it is :)
If the answer is "NO", would it be possible to create such a tool and make it available as OOS? I think so, but I would need some professional help (and would pay for this).
Thanks
Attachments:
[image/png] image001.png (136.0K, ../../2DD4DF38-6DCA-40D2-A1BF-DF39179FDACE@amazon.com/3-image001.png)
download | view image
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: [UNVERIFIED SENDER] Re: Continuous Performance Test
2019-08-14 20:59 Re: [UNVERIFIED SENDER] Re: Continuous Performance Test Nasby, Jim <nasbyj@amazon.com>
@ 2019-08-16 01:33 ` Jean Baro <jfbaro@gmail.com>
0 siblings, 0 replies; 5+ messages in thread
From: Jean Baro @ 2019-08-16 01:33 UTC (permalink / raw)
To: Nasby, Jim <nasbyj@amazon.com>; +Cc: pgsql-performance
No problem!
If I find anything Ill let you know.
Thanks
On Wed, 14 Aug 2019, 17:59 Nasby, Jim, <nasbyj@amazon.com> wrote:
> Ooops, looks like I need to add email rules for this community list! You
> can ignore my AWS-specific comments in the last paragraph. 😊
>
>
>
> *From: *"Nasby, Jim" <nasbyj@amazon.com>
> *Date: *Wednesday, August 14, 2019 at 3:57 PM
> *To: *Jean Baro <jfbaro@gmail.com>, "pgsql-performance@postgresql.org" <
> pgsql-performance@postgresql.org>
> *Subject: *[UNVERIFIED SENDER] Re: Continuous Performance Test
>
>
>
> I’m not aware of such a tool, but it would certainly be useful.
>
>
>
> Instead of trying to tie this directly to production changes (at which
> point it’s already too late), I encourage you to tie it into your
> QA/deployment process. If you have a reliable benchmark (a **great**
> accomplishment!) then you should run that as part of your pre-deployment
> process. Obviously that won’t reflect ad-hoc changes to your production
> environment, but frankly you don’t want ad-hoc in production except in an
> emergency (and even then it’s best to have a formal production change
> deployment process so you can track exactly what happened when).
>
>
>
> It does occur to me that it would be handy to be able to flag CloudWatch,
> Performance Insights, and Enhanced Monitoring when a deployment is done.
> That would make it easier to see the actual impact of a deployment on
> production. Let me know if you’re interested in doing a PFR for that.
>
>
>
> *From: *Jean Baro <jfbaro@gmail.com>
> *Date: *Wednesday, August 14, 2019 at 11:19 AM
> *To: *"pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org
> >
> *Subject: *Continuous Performance Test
>
>
>
> Hello!
>
>
>
> I was wondering if there is any external tool or extention to PostgreSQL >
> 10 (EC2 or RDS) which would:
>
> - Collect the TOP X statements (queries or DML) used in the last Y
> days IN PRODUCTION;
> - Everytime there is a change in the DATABASE Structure (Migrations,
> properties, Tables, Indexes) run these TOP X statements against a
> PERFORMANCE TEST DATABASE (Which would be very similar to production, on
> both HARDWARE and DATABASE SIZE)
> - Collect the metrics for these tests
> - Plot the results over time so that the PRODUCT team can be sure how
> the performance of the DATABASE is being affected by the chances over time
> (also as the number of rows changes). Something like this:
>
> Query vs new change to the DATABASE (+ its size). So that we can see which
> migration affect the database the most.
>
>
>
> Is there such a thing? It doesn't need to be open source nor free. But it
> would be good if it is :)
>
>
>
> If the answer is "NO", would it be possible to create such a tool and make
> it available as OOS? I think so, but I would need some professional help
> (and would pay for this).
>
>
>
> Thanks
>
Attachments:
[image/png] image001.png (136.0K, ../../CA+fQeemWOLywzUdMcwjrrkrtZoXmxVfzLRWGkFFFOFY3L3FTpw@mail.gmail.com/3-image001.png)
download | view image
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2019-08-16 01:33 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-08-14 16:18 Continuous Performance Test Jean Baro <jfbaro@gmail.com>
2019-08-14 17:56 ` Jean Baro <jfbaro@gmail.com>
2019-08-14 20:57 ` Nasby, Jim <nasbyj@amazon.com>
2019-08-14 20:59 Re: [UNVERIFIED SENDER] Re: Continuous Performance Test Nasby, Jim <nasbyj@amazon.com>
2019-08-16 01:33 ` Jean Baro <jfbaro@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox