Feedback-ID: id4a34324:Fastmail
Date: Thu, 28 May 2026 15:42:13 -0400
From: Andres Freund <andres@anarazel.de>
To: =?utf-8?Q?=C3=81lvaro?= Herrera <alvherre@kurilemu.de>
Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>,
	Jelte Fennema-Nio <postgres@jeltef.nl>,
 Thomas Munro <thomas.munro@gmail.com>,
	pgsql-hackers@postgresql.org, Zsolt Parragi <zsolt.parragi@percona.com>,
	Peter Eisentraut <peter@eisentraut.org>
Subject: Re: Heads Up: cirrus-ci is shutting down June 1st
Message-ID: <3wulk62iw2c5n3bxnkdpumm4updcbw3id3ahwkq2566q7m7jbj@qkbdz4rrukbw>
References: <ahh972Lord8jglyO@alvherre.pgsql>
 <ahiLmvtaFWnTsFVu@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ahiLmvtaFWnTsFVu@alvherre.pgsql>
Archived-At: 
 <https://www.postgresql.org/message-id/3wulk62iw2c5n3bxnkdpumm4updcbw3id3ahwkq2566q7m7jbj%40qkbdz4rrukbw>
Precedence: bulk

Hi,

On 2026-05-28 20:42:41 +0200, Álvaro Herrera wrote:
> On 2026-May-28, Álvaro Herrera wrote:
> 
> > Hi, I just pushed this to github to see how it would behave.  In case
> > anyone is curious, the run is here,
> > 
> > https://github.com/alvherre/postgres/actions/runs/26591496806
> > 
> > Overall I get the impression that it's much slower than Cirrus.  [...]


> Macos took 10 minutes just for the macports install!  Cirrus completed the
> same build in 18:32, https://cirrus-ci.com/build/4817054382948352

FWIW, the macports install should be down to ~5s on subsequent runs. Doing
that uncached was also rather slow on cirrus.


> > If I read the Github docs correctly, I get 2000 run minutes for free
> > each month.  That would mean I can run at most some ... 15 runs per
> > month?  That sounds quite limiting.

My understanding is that the 2000 minutes is the limit for *private*
repositories: [1]
"Use of the standard GitHub-hosted runners is free and unlimited on public
repositories."

The concurrency limits are also pretty generous: [2], 20 jobs of 4 cores each.

If you look at the limits of other providers, they are much much lower. Like
400 CPU minutes for gitlab (so about 100 minutes of 4 core VMs).


On 2026-05-28 20:11:45 +0200, Álvaro Herrera wrote:
> Hi, I just pushed this to github to see how it would behave.  In case
> anyone is curious, the run is here,
> 
> https://github.com/alvherre/postgres/actions/runs/26591496806
> 
> Overall I get the impression that it's much slower than Cirrus.  20
> minutes in, only the two "Linux - Meson" build finished, in 14 and 17
> minutes respectively.

It's definitely slower. I've not fully analyzed why, my suspicion is that we
end up being rather terribly IO bound - we used bigger and faster disks on
cirrus than we have access to with github hosted runners (there are large
runners with more storage, but that's not free).

A full testrun on master creates about 36GB of data directories. If individual
tests are fast, that's often not *that* bad, because the tests are over before
linux decides to flush out the data, and then linux never needs to write that
data back, because we remove the data directories immediately. But once you
get to the point that several tests take more than 30s (the default time after
which linux writes dirty data back) or enough dirty data accumulates (20% of
memory IIRC), you have a lot of IO.

My buildfarm host, which hosts quite a few animals, got a new disk within the
last year. Here's what smartctl says about disk IO:

Data Units Read:                    43,513,034 [22.2 TB]
Data Units Written:                 6,062,401,949 [3.10 PB]

A nice indication of how much our tests end up writing...


I think we're going to have to fix that on our end to some degree. 36GB of
data being written each test run is just not reasonable. We spin up quite a
few separate data directories for tests that take well under a second. That's
just a very unfavorable ratio. In [3] I wrote

> I think we need to combine about half the modules in src/test/modules, the
> current course is absurd:
> 
> 16:  37
> 17:  46
> 18:  49
> dev: 62


Greetings,

Andres Freund

[1] https://docs.github.com/en/actions/reference/runners/github-hosted-runners#standard-github-hosted-runners-for-public-repositories
[2] https://docs.github.com/en/actions/reference/limits#job-concurrency-limits-for-github-hosted-runners
[3] https://postgr.es/m/hp4xznm7dqt4ediyhezqysf22eljvu3mucbzsgvgehc6j2hk5v%40laslwlkyixfg