public inbox for [email protected]
help / color / mirror / Atom feedFrom: Álvaro Hernández <[email protected]>
To: Jeremy Schneider <[email protected]>
To: [email protected]
Subject: Re: deb package sizes
Date: Thu, 9 Jan 2025 16:43:16 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <20250109005301.3b145092@jeremy-ThinkPad-T430s>
References: <20250109005301.3b145092@jeremy-ThinkPad-T430s>
On 9/1/25 9:53, Jeremy Schneider wrote:
> Hello, I hope I found a good mailing list for this topic?
>
> Recently, I've been spending some time looking at the official Postgres
> docker images. https://hub.docker.com/_/postgres/
Hi Jeremy.
Nitpicking a bit, but I'd call these the "Official Docker Postgres
images". They are official from Docker's perspective. I say this for
general awareness, not everybody understand it's like this.
> With docker images, we like to get the container images to be as
> minimal and small as possible.
Agreed.
> I have spent a little time looking at the
> make-up of the official docker images from a size perspective, which is
> driven by debian package sizes.
In my opinion, "system packages" (deb, rpm, etc) are not
necessarily the best way to compose container images. They are designed
for "systems", and usually contain many files that may not be needed on
a container.
> Before adding any PGDG postgres packages or dependencies, our base OS
> container image is 74MB and includes about 88 debian packages.
Something to consider here is using Distroless
(https://github.com/GoogleContainerTools/distroless) which is a bit of a
misnomer as it really it's based on Debian too.
> We install only 5 PGDG postgres packages: postgresql,
> postgresql-client, postgresql-client-common, postgresql-common and
> libpq5. The "common" packages are tiny, libpq is 1MB, client is 10MB
> and the postgresql package itself is 60MB.
>
> What's more interesting is all of the additional dependencies that the
> postgresql package pulls in: an extra 53 debian packages that are over
> 250MB in total size.
>
> The biggest size contributors are libllvm & libz3 (143MB), libperl &
> perl-modules (45MB total) and libicu (36MB). These three things alone
> make up 64% of the total postgres-specific bytes.
While the results are not too different from your analysis, I'd do
it from the layers that compose the image itself. Here's a simple way to
do it:
$ docker history --no-trunc --format '{{ .Size }} {{ .CreatedBy }}'
postgres |egrep '^[0-9]+(\.[0-9]+)?MB' | cut -b 1-72
330MB RUN /bin/sh -c set -ex; export PYTHONDONTWRITEBYTECODE=1; dpkg
3.61MB RUN /bin/sh -c set -eux; apt-get update; apt-get install -y --n
26.9MB RUN /bin/sh -c set -eux; if [ -f /etc/dpkg/dpkg.cfg.d/docker ];
4.27MB RUN /bin/sh -c set -eux; savedAptMark="$(apt-mark showmanual)";
10.8MB RUN /bin/sh -c set -ex; apt-get update; apt-get install -y --no
85.2MB # debian.sh --arch 'amd64' out/ 'bookworm' '@1734912000'
(see attached a non-truncated version for completeness)
"Base" image is 85MB, Postgres plus dependencies is 330MB (which
you distilled in more detail) and then there's some other 27MB in
locales and 11MB in additional tools.
Also to note is that Docker's official Postgres image compiles from
source packages, not just installs from PGDG (e.g. see
https://github.com/docker-library/postgres/blob/cb049360d9a316e429740d47431e0d6fa129d11a/17/bookworm...).
> I'm wondering if there might be any support for providing a
> "postgresql-slim" package on PGDG which excludes llvm and python? I
> think this might almost cut the total install size in half, and I think
> there might be many users who would value having the option.
>
> Even though ICU is a larger package, I would argue for still
> including it in a "slim" build. Because of the drama around glibc
> collation I view ICU as especially important to make available.
>
> Interested to know others' thoughts about having a slimmer package.
+1
I believe there should be place for slimmer, or even better,
user-configurable Postgres images. Different use cases need different
containers. Postgres on testcontainers use case needs little to no
additional features, while a production setup may require different
additional tools. Similarly, different environments (ICU / not ICU, sets
of locales, parallel query or not) may require different images. Having
choice here would be of great benefit.
Álvaro
--
Alvaro Hernandez
-----------
OnGres
330MB RUN /bin/sh -c set -ex; export PYTHONDONTWRITEBYTECODE=1; dpkgArch="$(dpkg --print-architecture)"; aptRepo="[ signed-by=/usr/local/share/keyrings/postgres.gpg.asc ] http://apt.postgresql.org/pub/repos/apt/ bookworm-pgdg main $PG_MAJOR"; case "$dpkgArch" in amd64 | arm64 | ppc64el | s390x) echo "deb $aptRepo" > /etc/apt/sources.list.d/pgdg.list; apt-get update; ;; *) echo "deb-src $aptRepo" > /etc/apt/sources.list.d/pgdg.list; savedAptMark="$(apt-mark showmanual)"; tempDir="$(mktemp -d)"; cd "$tempDir"; apt-get update; apt-get install -y --no-install-recommends dpkg-dev; echo "deb [ trusted=yes ] file://$tempDir ./" > /etc/apt/sources.list.d/temp.list; _update_repo() { dpkg-scanpackages . > Packages; apt-get -o Acquire::GzipIndexes=false update; }; _update_repo; nproc="$(nproc)"; export DEB_BUILD_OPTIONS="nocheck parallel=$nproc"; apt-get build-dep -y postgresql-common pgdg-keyring; apt-get source --compile postgresql-common pgdg-keyring; _update_repo; apt-get build-dep -y "postgresql-$PG_MAJOR=$PG_VERSION"; apt-get source --compile "postgresql-$PG_MAJOR=$PG_VERSION"; apt-mark showmanual | xargs apt-mark auto > /dev/null; apt-mark manual $savedAptMark; ls -lAFh; _update_repo; grep '^Package: ' Packages; cd /; ;; esac; apt-get install -y --no-install-recommends postgresql-common; sed -ri 's/#(create_main_cluster) .*$/\1 = false/' /etc/postgresql-common/createcluster.conf; apt-get install -y --no-install-recommends "postgresql-$PG_MAJOR=$PG_VERSION" ; rm -rf /var/lib/apt/lists/*; if [ -n "$tempDir" ]; then apt-get purge -y --auto-remove; rm -rf "$tempDir" /etc/apt/sources.list.d/temp.list; fi; find /usr -name '*.pyc' -type f -exec bash -c 'for pyc; do dpkg -S "$pyc" &> /dev/null || rm -vf "$pyc"; done' -- '{}' +; postgres --version # buildkit
3.61MB RUN /bin/sh -c set -eux; apt-get update; apt-get install -y --no-install-recommends libnss-wrapper xz-utils zstd ; rm -rf /var/lib/apt/lists/* # buildkit
26.9MB RUN /bin/sh -c set -eux; if [ -f /etc/dpkg/dpkg.cfg.d/docker ]; then grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; sed -ri '/\/usr\/share\/locale/d' /etc/dpkg/dpkg.cfg.d/docker; ! grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; fi; apt-get update; apt-get install -y --no-install-recommends locales; rm -rf /var/lib/apt/lists/*; echo 'en_US.UTF-8 UTF-8' >> /etc/locale.gen; locale-gen; locale -a | grep 'en_US.utf8' # buildkit
4.27MB RUN /bin/sh -c set -eux; savedAptMark="$(apt-mark showmanual)"; apt-get update; apt-get install -y --no-install-recommends ca-certificates wget; rm -rf /var/lib/apt/lists/*; dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch";; wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc";; export GNUPGHOME="$(mktemp -d)"; gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; gpgconf --kill all; rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; apt-mark auto '.*' > /dev/null; [ -z "$savedAptMark" ] || apt-mark manual $savedAptMark > /dev/null; apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; chmod +x /usr/local/bin/gosu; gosu --version; gosu nobody true # buildkit
10.8MB RUN /bin/sh -c set -ex; apt-get update; apt-get install -y --no-install-recommends gnupg less ; rm -rf /var/lib/apt/lists/* # buildkit
85.2MB # debian.sh --arch 'amd64' out/ 'bookworm' '@1734912000'
Attachments:
[text/plain] postgres-image.layers.size.txt (3.7K, 2-postgres-image.layers.size.txt)
download | inline:
330MB RUN /bin/sh -c set -ex; export PYTHONDONTWRITEBYTECODE=1; dpkgArch="$(dpkg --print-architecture)"; aptRepo="[ signed-by=/usr/local/share/keyrings/postgres.gpg.asc ] http://apt.postgresql.org/pub/repos/apt/ bookworm-pgdg main $PG_MAJOR"; case "$dpkgArch" in amd64 | arm64 | ppc64el | s390x) echo "deb $aptRepo" > /etc/apt/sources.list.d/pgdg.list; apt-get update; ;; *) echo "deb-src $aptRepo" > /etc/apt/sources.list.d/pgdg.list; savedAptMark="$(apt-mark showmanual)"; tempDir="$(mktemp -d)"; cd "$tempDir"; apt-get update; apt-get install -y --no-install-recommends dpkg-dev; echo "deb [ trusted=yes ] file://$tempDir ./" > /etc/apt/sources.list.d/temp.list; _update_repo() { dpkg-scanpackages . > Packages; apt-get -o Acquire::GzipIndexes=false update; }; _update_repo; nproc="$(nproc)"; export DEB_BUILD_OPTIONS="nocheck parallel=$nproc"; apt-get build-dep -y postgresql-common pgdg-keyring; apt-get source --compile postgresql-common pgdg-keyring; _update_repo; apt-get build-dep -y "postgresql-$PG_MAJOR=$PG_VERSION"; apt-get source --compile "postgresql-$PG_MAJOR=$PG_VERSION"; apt-mark showmanual | xargs apt-mark auto > /dev/null; apt-mark manual $savedAptMark; ls -lAFh; _update_repo; grep '^Package: ' Packages; cd /; ;; esac; apt-get install -y --no-install-recommends postgresql-common; sed -ri 's/#(create_main_cluster) .*$/\1 = false/' /etc/postgresql-common/createcluster.conf; apt-get install -y --no-install-recommends "postgresql-$PG_MAJOR=$PG_VERSION" ; rm -rf /var/lib/apt/lists/*; if [ -n "$tempDir" ]; then apt-get purge -y --auto-remove; rm -rf "$tempDir" /etc/apt/sources.list.d/temp.list; fi; find /usr -name '*.pyc' -type f -exec bash -c 'for pyc; do dpkg -S "$pyc" &> /dev/null || rm -vf "$pyc"; done' -- '{}' +; postgres --version # buildkit
3.61MB RUN /bin/sh -c set -eux; apt-get update; apt-get install -y --no-install-recommends libnss-wrapper xz-utils zstd ; rm -rf /var/lib/apt/lists/* # buildkit
26.9MB RUN /bin/sh -c set -eux; if [ -f /etc/dpkg/dpkg.cfg.d/docker ]; then grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; sed -ri '/\/usr\/share\/locale/d' /etc/dpkg/dpkg.cfg.d/docker; ! grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; fi; apt-get update; apt-get install -y --no-install-recommends locales; rm -rf /var/lib/apt/lists/*; echo 'en_US.UTF-8 UTF-8' >> /etc/locale.gen; locale-gen; locale -a | grep 'en_US.utf8' # buildkit
4.27MB RUN /bin/sh -c set -eux; savedAptMark="$(apt-mark showmanual)"; apt-get update; apt-get install -y --no-install-recommends ca-certificates wget; rm -rf /var/lib/apt/lists/*; dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch"; wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc"; export GNUPGHOME="$(mktemp -d)"; gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; gpgconf --kill all; rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; apt-mark auto '.*' > /dev/null; [ -z "$savedAptMark" ] || apt-mark manual $savedAptMark > /dev/null; apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; chmod +x /usr/local/bin/gosu; gosu --version; gosu nobody true # buildkit
10.8MB RUN /bin/sh -c set -ex; apt-get update; apt-get install -y --no-install-recommends gnupg less ; rm -rf /var/lib/apt/lists/* # buildkit
85.2MB # debian.sh --arch 'amd64' out/ 'bookworm' '@1734912000'
view thread (10+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: deb package sizes
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox