pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Gavin Flower <GavinFlower@archidevsys.co.nz>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: michael@sqlexec.com <michael@sqlexec.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: MusicBrainz postgres performance issues
Date: Mon, 16 Mar 2015 13:22:52 +1300
Message-ID: <5506225C.1070304@archidevsys.co.nz> (raw)
In-Reply-To: <55061EC8.7030807@2ndquadrant.com>
References: <B008F2EE-45D7-4535-9163-56A8CDCA553C@musicbrainz.org>
	<CAB6McgXxZdD6yuYRLg4tgDdhqzXKLs63aV+S_nOfZ9arSXUAhA@mail.gmail.com>
	<9E71CCBE-4CDF-482E-878D-6C16027B6283@musicbrainz.org>
	<20150315135022.GC19792@alap3.anarazel.de>
	<CAOR=d=0YwcnVi3kGD21W7jYjE7DcWjp0v0GjncH-c2-oJEP5Rw@mail.gmail.com>
	<CAOR=d=1Axcbj2VcUO=5tM1nv6RjuA0D+av5+Mt7RTztgTxaMnA@mail.gmail.com>
	<20150315172048.GG9324@alap3.anarazel.de>
	<CAG95seXv4EKGhDV04rkk_ARgvz8OQUJy3Xbx5Rw16Wxm3+1j5Q@mail.gmail.com>
	<20150315174646.GH9324@alap3.anarazel.de>
	<CAOR=d=2rC8dYr=tY7JQTNK+eszx2J1pRHuqYFHPbqxVDXiN1sQ@mail.gmail.com>
	<20150315224756.GE29732@awork2.anarazel.de>
	<550618C2.3030802@2ndquadrant.com>
	<55061BEB.5070209@sqlexec.com>
	<55061EC8.7030807@2ndquadrant.com>
List-Unsubscribe:  <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>

On 16/03/15 13:07, Tomas Vondra wrote:
> On 16.3.2015 00:55, michael@sqlexec.com wrote:
>> Why is 500 connections "insane". We got 32 CPU with 96GB and 3000
>> max connections, and we are doing fine, even when hitting our max
>> concurrent connection peaks around 4500. At a previous site, we were
>> using 2000 max connections on 24 CPU and 64GB RAM, with about 1500
>> max concurrent connections. So I wouldn't be too hasty in saying more
>> than 500 is asking for trouble. Just as long as you got your kernel
>> resources set high enough to sustain it (SHMMAX, SHMALL, SEMMNI, and
>> ulimits), and RAM for work_mem.
[...]
> Also, if part of the query required a certain amount of memory for part
> of the plan, it now holds that memory for much longer too. That only
> increases the change of OOM issues.
>
[...]

Also you could get a situation where a small number of queries & their 
data, relevant indexes, and working memory etc can all just fit into 
RAM, but the extra queries suddenly reduce the RAM so that even these 
queries spill to disk, plus the time required to process the extra 
queries.  So a nicely behaved system could suddenly get a lot worse.  
Even before you consider additional lock contention and other nasty things!

It all depends...


Cheers,
Gavin


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance



view thread (41+ messages)  latest in thread

Message-ID: <5506225C.1070304@archidevsys.co.nz>
Permalink:  ../5506225C.1070304@archidevsys.co.nz/
Also on:    postgresql.org/message-id/5506225C.1070304@archidevsys.co.nz

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: GavinFlower@archidevsys.co.nz, tomas.vondra@2ndquadrant.com, michael@sqlexec.com
  Subject: Re: MusicBrainz postgres performance issues
  In-Reply-To: <5506225C.1070304@archidevsys.co.nz>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox