Subject: PostgreSQL - HA Cluster / Etcd Issue

From: SOzcn <[email-masked]>
Date: Wed, 20 Aug 2025 10:40:22 +0300

Hello,

In my database architecture, I use Etcd, Patroni, HAProxy, and Keepalived.

Although many things in this architecture work as expected, Etcd encounters
the error "memberID:5960773920904691185 alarm:NOSPACE" once every 2 or 3
months. The root cause is that the database size reaches the specified
limit and Etcd does not clean it up through rotation. I am aware of this,
but despite reviewing the maintenance documentation, I could not find the
most appropriate parameter values for this rotation. I am sharing an
example of my own Etcd configuration below. I kindly ask you to share your
experience on this matter. I am kinda stuck on this issue.

In particular, is it possible to schedule the parameter that performs this
cleanup with a time setting? For example, can I tell it to run the compact
operation at around 01:00 AM?

Best regards.

[Unit]
Description=etcd service
Documentation=https://github.com/etcd-io/etcd
After=network.target

[Service]
User=etcd
Type=notify
ExecStart=/usr/local/sbin/etcd \
  --name etcd1 \
  --data-dir=/etc/etcd \
  --initial-advertise-peer-urls=http://10.115.208.168:2380 \
  --listen-peer-urls=http://192.168.1.1:2380 \
  --listen-client-urls=http://192.168.1.1:2379,http://127.0.0.1:2379 \
  --advertise-client-urls=192.168.1.1:2379 \
  --initial-cluster-token="etcd-cluster" \
  --initial-cluster="etcd1=http://192.168.1.1:2380,etcd2=
http://192.168.1.2:2380,etcd3=http://3:2380" \
  --initial-cluster-state="new" \
  --log-level="error"

[Install]
WantedBy=multi-user.target

---

From: Scott Ribe <[email-masked]>
Date: Wed, 20 Aug 2025 06:46:40 -0600

This is a PostgreSQL mailing list, and your question has nothing to do with PostgreSQL.

--
Scott Ribe
[email-masked]
https://www.linkedin.com/in/scottribe/

> On Aug 20, 2025, at 1:40 AM, SOzcn <[email-masked]> wrote:
> 
> Hello,
> 
> Although many things in this architecture work as expected, Etcd encounters the error "memberID:5960773920904691185 alarm:NOSPACE" once every 2 or 3 months. The root cause is that the database size reaches the specified limit and Etcd does not clean it up through rotation. I am aware of this, but despite reviewing the maintenance documentation, I could not find the most appropriate parameter values for this rotation. I am sharing an example of my own Etcd configuration below. I kindly ask you to share your experience on this matter. I am kinda stuck on this issue.
> 
> In particular, is it possible to schedule the parameter that performs this cleanup with a time setting? For example, can I tell it to run the compact operation at around 01:00 AM?
> 
> Best regards.
> 
> 
> [Unit]
> Description=etcd service
>   --initial-cluster="etcd1=http://192.168.1.1:2380,etcd2=http://192.168.1.2:2380,etcd3=http://3:2380" \

---

From: SOzcn <[email-masked]>
Date: Wed, 20 Aug 2025 16:52:42 +0300

This is high availability environment and etcd is one management tools of
them . Many PostgreSQL managed by this architec. I advice to you research.
Etcs - Patrono - Postgresql- Haproxy- Keepalived. Anyway if you doesn't
have experience of it, that's ok.

20 Ağu 2025 Çar 15:46 tarihinde Scott Ribe <[email-masked]>
şunu yazdı:

> > Although many things in this architecture work as expected, Etcd
> encounters the error "memberID:5960773920904691185 alarm:NOSPACE" once
> every 2 or 3 months. The root cause is that the database size reaches the
> specified limit and Etcd does not clean it up through rotation. I am aware
> of this, but despite reviewing the maintenance documentation, I could not
> find the most appropriate parameter values for this rotation. I am sharing
> an example of my own Etcd configuration below. I kindly ask you to share
> your experience on this matter. I am kinda stuck on this issue.
> >
> > In particular, is it possible to schedule the parameter that performs
> this cleanup with a time setting? For example, can I tell it to run the
> compact operation at around 01:00 AM?
> >
> > Best regards.
> >
> >
> > [Unit]
> > Description=etcd service

---

From: Mike Artz <[email-masked]>
Date: Wed, 20 Aug 2025 09:03:08 -0500

Are you saying that you already have a compaction job running (e.g.

etcdctl compact $(etcdctl endpoint status --write-out=json | jq -r
'.[0].Status.header.revision - 1000')

or similar), but you’re hoping there’s a parameter in etcd itself to handle
that scheduling?

---

From: Scott Ribe <[email-masked]>
Date: Wed, 20 Aug 2025 10:45:13 -0600

> On Aug 20, 2025, at 7:52 AM, SOzcn <[email-masked]> wrote:
> 
> This is high availability environment and etcd is one management tools of them . Many PostgreSQL managed by this architec. I advice to you research. Etcs - Patrono - Postgresql- Haproxy- Keepalived. Anyway if you doesn't have experience of it, that's ok. 

I am perfectly well aware of all that. However, you have an etcd problem, not a postgres problem. You'd have a much better chance of getting answers to etcd issues on an etcd mailing list.

---

From: Ozgur Kulu <[email-masked]>
Date: Wed, 20 Aug 2025 19:51:36 +0300

Hi there,

Etcd database should be extended from 2 gb to 8gb. On the other hand, you
can use defrag or again created the etcd db.

Have a nice day,

20 Ağu 2025 Çar 10:40 tarihinde SOzcn <[email-masked]> şunu

---

From: SOzcn <[email-masked]>
Date: Thu, 21 Aug 2025 01:17:03 +0300

Hello Ozgur,

Thanks for the reply.

If i up to etcd size from 2 to 8 GB it's make much problem but if defrag is
fix the "no space" issue can you share your experience? By example is it
spent too much rss etc. and are you doing that with cronjob or manuel. As i
know if etcd went to error with "no space" defrag methot also doesn't work.

Have a nice day.

Ozgur Kulu <[email-masked]>, 20 Ağu 2025 Çar, 19:51 tarihinde şunu