public inbox for [email protected]
help / color / mirror / Atom feedFrom: Dave Page <[email protected]>
To: pgadmin-hackers <[email protected]>
Subject: UBI based container patch
Date: Mon, 1 Feb 2021 16:21:45 +0000
Message-ID: <CA+OCxoz0JKkgw5tkiqK1FemLhB3O4FuXu7MOBF0B=6EYyvBqQA@mail.gmail.com> (raw)
* Do not commit this! *
The dev team have discussed moving the pgAdmin container to be based on
Redhat's UBI instead of Alpine Linux. I got some time today to work on
that, and a patch is attached for interest/archives.
Whilst the build process is arguably cleaner with this patch, plus it
would be easier to maintain the versions of the PostgreSQL tools that are
included, unfortunately using UBI takes the size of the container from
~278MB to ~462MB.
So... unless someone finds a huge error in the patch that causes this
bloat, I think we have no choice but to stick with Alpine Linux.
Any thoughts/comments?
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] ubi-container.diff (12.0K, 3-ubi-container.diff)
download | inline diff:
diff --git a/.dockerignore b/.dockerignore
index 4c3d8b0ab..5feb2bae6 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -1,4 +1,6 @@
.git
+docs/en_US/_build/html/_sources
+docs/en_US/_build/html/_static/*.png
web/node_modules
web/*.log
web/regression
diff --git a/Dockerfile b/Dockerfile
index ab04671d8..74ac7e680 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -12,20 +12,12 @@
# and clean up the web/ source code
#########################################################################
-FROM node:14-alpine3.12 AS app-builder
-
-RUN apk add --no-cache \
- autoconf \
- automake \
- bash \
- g++ \
- libc6-compat \
- libjpeg-turbo-dev \
- libpng-dev \
- make \
- nasm \
- git \
- zlib-dev
+FROM registry.redhat.io/ubi8/ubi-minimal AS app-builder
+
+# Install dependencies
+RUN echo -e "[nodejs]\nname=nodejs\nstream=14\nprofiles=\nstate=enabled\n" > /etc/dnf/modules.d/nodejs.module
+RUN microdnf install -y nodejs git
+RUN npm install --global yarn
# Create the /pgadmin4 directory and copy the source into it. Explicitly
# remove the node_modules directory as we'll recreate a clean version, as well
@@ -60,136 +52,65 @@ RUN npm install && \
karma.conf.js \
./pgadmin/static/js/generated/.cache
-#########################################################################
-# Now, create a documentation build container for the Sphinx docs
-#########################################################################
+##########################################################################
+## Now, create the virtual environment
+##########################################################################
-FROM python:3.9-alpine3.12 as docs-builder
+FROM registry.redhat.io/ubi8/ubi-minimal AS env-builder
# Install dependencies
+RUN microdnf install -y gcc make python3-devel python3-pip postgresql-devel krb5-devel
+
COPY requirements.txt /
-RUN apk add --no-cache \
- make \
- build-base \
- openssl-dev \
- libffi-dev \
- postgresql-dev \
- krb5-dev && \
- pip install --no-cache-dir \
- sphinx && \
- pip install --no-cache-dir -r requirements.txt
+RUN python3 -m venv /venv
+RUN /venv/bin/pip3 install --upgrade pip
+RUN /venv/bin/pip3 install -r requirements.txt
+RUN /venv/bin/pip3 install gunicorn
+
+##########################################################################
+## Now, create a documentation build container for the Sphinx docs
+##########################################################################
+
+FROM registry.redhat.io/ubi8/ubi-minimal AS doc-builder
+
+# Install dependencies
+RUN microdnf install -y make python3
+COPY --from=env-builder /venv /venv
+RUN /venv/bin/pip3 install sphinx
# Copy the docs from the local tree. Explicitly remove any existing builds that
-# may be present
+# may be present. We don't use the .dockerignore for this as we need to copy
+# the _build directory later
COPY docs /pgadmin4/docs
COPY web /pgadmin4/web
RUN rm -rf /pgadmin4/docs/en_US/_build
# Build the docs
-RUN LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 make -C /pgadmin4/docs/en_US -f Makefile.sphinx html
-
-#########################################################################
-# Create additional builders to get all of the PostgreSQL utilities
-#########################################################################
-
-FROM postgres:9.5-alpine as pg95-builder
-FROM postgres:9.6-alpine as pg96-builder
-FROM postgres:10-alpine as pg10-builder
-FROM postgres:11-alpine as pg11-builder
-FROM postgres:12-alpine as pg12-builder
-FROM postgres:13-alpine as pg13-builder
-
-FROM alpine:3.11 as tool-builder
-
-# Copy the PG binaries
-COPY --from=pg95-builder /usr/local/bin/pg_dump /usr/local/pgsql/pgsql-9.5/
-COPY --from=pg95-builder /usr/local/bin/pg_dumpall /usr/local/pgsql/pgsql-9.5/
-COPY --from=pg95-builder /usr/local/bin/pg_restore /usr/local/pgsql/pgsql-9.5/
-COPY --from=pg95-builder /usr/local/bin/psql /usr/local/pgsql/pgsql-9.5/
-
-COPY --from=pg96-builder /usr/local/bin/pg_dump /usr/local/pgsql/pgsql-9.6/
-COPY --from=pg96-builder /usr/local/bin/pg_dumpall /usr/local/pgsql/pgsql-9.6/
-COPY --from=pg96-builder /usr/local/bin/pg_restore /usr/local/pgsql/pgsql-9.6/
-COPY --from=pg96-builder /usr/local/bin/psql /usr/local/pgsql/pgsql-9.6/
-
-COPY --from=pg10-builder /usr/local/bin/pg_dump /usr/local/pgsql/pgsql-10/
-COPY --from=pg10-builder /usr/local/bin/pg_dumpall /usr/local/pgsql/pgsql-10/
-COPY --from=pg10-builder /usr/local/bin/pg_restore /usr/local/pgsql/pgsql-10/
-COPY --from=pg10-builder /usr/local/bin/psql /usr/local/pgsql/pgsql-10/
-
-COPY --from=pg11-builder /usr/local/bin/pg_dump /usr/local/pgsql/pgsql-11/
-COPY --from=pg11-builder /usr/local/bin/pg_dumpall /usr/local/pgsql/pgsql-11/
-COPY --from=pg11-builder /usr/local/bin/pg_restore /usr/local/pgsql/pgsql-11/
-COPY --from=pg11-builder /usr/local/bin/psql /usr/local/pgsql/pgsql-11/
-
-COPY --from=pg12-builder /usr/local/bin/pg_dump /usr/local/pgsql/pgsql-12/
-COPY --from=pg12-builder /usr/local/bin/pg_dumpall /usr/local/pgsql/pgsql-12/
-COPY --from=pg12-builder /usr/local/bin/pg_restore /usr/local/pgsql/pgsql-12/
-COPY --from=pg12-builder /usr/local/bin/psql /usr/local/pgsql/pgsql-12/
-
-COPY --from=pg13-builder /usr/local/bin/pg_dump /usr/local/pgsql/pgsql-13/
-COPY --from=pg13-builder /usr/local/bin/pg_dumpall /usr/local/pgsql/pgsql-13/
-COPY --from=pg13-builder /usr/local/bin/pg_restore /usr/local/pgsql/pgsql-13/
-COPY --from=pg13-builder /usr/local/bin/psql /usr/local/pgsql/pgsql-13/
+RUN source /venv/bin/activate && \
+ LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 make -C /pgadmin4/docs/en_US -f Makefile.sphinx html
#########################################################################
# Assemble everything into the final container.
#########################################################################
-FROM python:3.9-alpine3.12
-
-COPY --from=tool-builder /usr/local/pgsql /usr/local/
+FROM registry.redhat.io/ubi8/ubi-minimal
WORKDIR /pgadmin4
ENV PYTHONPATH=/pgadmin4
# Copy in the code and docs
COPY --from=app-builder /pgadmin4/web /pgadmin4
-COPY --from=docs-builder /pgadmin4/docs/en_US/_build/html/ /pgadmin4/docs
-COPY requirements.txt /pgadmin4/requirements.txt
-
-# License files
-COPY LICENSE /pgadmin4/LICENSE
-COPY DEPENDENCIES /pgadmin4/DEPENDENCIES
-
-# Install build-dependencies, build & install C extensions and purge deps in
-# one RUN step
-RUN apk add --no-cache --virtual \
- build-deps \
- build-base \
- postgresql-dev \
- libffi-dev \
- krb5-dev \
- e2fsprogs-dev \
- krb5-server-ldap \
- linux-headers && \
- apk add \
- postfix \
- postgresql-client \
- postgresql-libs \
- krb5-libs \
- shadow \
- sudo \
- libcap && \
- pip install --upgrade pip && \
- pip install --no-cache-dir -r requirements.txt && \
- pip install --no-cache-dir gunicorn==19.9.0 && \
- apk del --no-cache build-deps && \
- echo "pgadmin ALL = NOPASSWD: /usr/sbin/postfix start" > /etc/sudoers.d/postfix
-
-# We need the v13 libpq
-COPY --from=pg13-builder /usr/local/lib/libpq.so.5.13 /usr/lib/
-RUN ln -sf /usr/lib/libpq.so.5.13 /usr/lib/libpq.so.5
-
-# Copy the runner script
+COPY --from=doc-builder /pgadmin4/docs/en_US/_build/html/ /pgadmin4/docs
+COPY --from=env-builder /venv /venv
COPY pkg/docker/run_pgadmin.py /pgadmin4
COPY pkg/docker/gunicorn_config.py /pgadmin4
COPY pkg/docker/entrypoint.sh /entrypoint.sh
-# Precompile and optimize python code to save time and space on startup
-RUN python -O -m compileall -x node_modules /pgadmin4
-
-RUN groupadd -g 5050 pgadmin && \
+# Perform all the OS-level setup. Do this in one RUN command to minimise the
+# number of layers
+RUN rpm -i https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm && \
+ microdnf install -y postgresql13 postgresql12 postgresql11 postgresql10 postgresql96 postfix python3 sudo && \
+ groupadd -g 5050 pgadmin && \
useradd -r -u 5050 -g pgadmin pgadmin && \
mkdir -p /var/lib/pgadmin && \
chown pgadmin:pgadmin /var/lib/pgadmin && \
@@ -197,11 +118,14 @@ RUN groupadd -g 5050 pgadmin && \
chown pgadmin:pgadmin /var/log/pgadmin && \
touch /pgadmin4/config_distro.py && \
chown pgadmin:pgadmin /pgadmin4/config_distro.py && \
- setcap CAP_NET_BIND_SERVICE=+eip /usr/local/bin/python3.9
+ setcap CAP_NET_BIND_SERVICE=+eip /usr/libexec/platform-python3.6 && \
+ sed -i 's/inet_protocols = .*/inet_protocols = ipv4/g' /etc/postfix/main.cf && \
+ echo "pgadmin ALL = NOPASSWD: /usr/sbin/postfix start" > /etc/sudoers.d/postfix && \
+ microdnf clean all
USER pgadmin
# Finish up
VOLUME /var/lib/pgadmin
EXPOSE 80 443
-ENTRYPOINT ["/entrypoint.sh"]
+ENTRYPOINT ["/entrypoint.sh"]
\ No newline at end of file
diff --git a/pkg/docker/entrypoint.sh b/pkg/docker/entrypoint.sh
index 2b33bcfab..87d0747e8 100755
--- a/pkg/docker/entrypoint.sh
+++ b/pkg/docker/entrypoint.sh
@@ -32,7 +32,7 @@ if [ ! -f /var/lib/pgadmin/pgadmin4.db ]; then
# Initialize DB before starting Gunicorn
# Importing pgadmin4 (from this script) is enough
- python run_pgadmin.py
+ /venv/bin/python run_pgadmin.py
export PGADMIN_SERVER_JSON_FILE=${PGADMIN_SERVER_JSON_FILE:-/pgadmin4/servers.json}
# Pre-load any required servers
@@ -40,9 +40,9 @@ if [ ! -f /var/lib/pgadmin/pgadmin4.db ]; then
# When running in Desktop mode, no user is created
# so we have to import servers anonymously
if [ "${PGADMIN_CONFIG_SERVER_MODE}" = "False" ]; then
- /usr/local/bin/python /pgadmin4/setup.py --load-servers "${PGADMIN_SERVER_JSON_FILE}"
+ /venv/bin/python /pgadmin4/setup.py --load-servers "${PGADMIN_SERVER_JSON_FILE}"
else
- /usr/local/bin/python /pgadmin4/setup.py --load-servers "${PGADMIN_SERVER_JSON_FILE}" --user ${PGADMIN_DEFAULT_EMAIL}
+ /venv/bin/python /pgadmin4/setup.py --load-servers "${PGADMIN_SERVER_JSON_FILE}" --user ${PGADMIN_DEFAULT_EMAIL}
fi
fi
fi
@@ -54,13 +54,13 @@ fi
# Get the session timeout from the pgAdmin config. We'll use this (in seconds)
# to define the Gunicorn worker timeout
-TIMEOUT=$(cd /pgadmin4 && python -c 'import config; print(config.SESSION_EXPIRATION_TIME * 60 * 60 * 24)')
+TIMEOUT=$(cd /pgadmin4 && /venv/bin/python -c 'import config; print(config.SESSION_EXPIRATION_TIME * 60 * 60 * 24)')
# NOTE: currently pgadmin can run only with 1 worker due to sessions implementation
# Using --threads to have multi-threaded single-process worker
if [ ! -z ${PGADMIN_ENABLE_TLS} ]; then
- exec gunicorn --timeout ${TIMEOUT} --bind ${PGADMIN_LISTEN_ADDRESS:-[::]}:${PGADMIN_LISTEN_PORT:-443} -w 1 --threads ${GUNICORN_THREADS:-25} --access-logfile ${GUNICORN_ACCESS_LOGFILE:--} --keyfile /certs/server.key --certfile /certs/server.cert -c gunicorn_config.py run_pgadmin:app
+ exec /venv/bin/gunicorn --timeout ${TIMEOUT} --bind ${PGADMIN_LISTEN_ADDRESS:-[::]}:${PGADMIN_LISTEN_PORT:-443} -w 1 --threads ${GUNICORN_THREADS:-25} --access-logfile ${GUNICORN_ACCESS_LOGFILE:--} --keyfile /certs/server.key --certfile /certs/server.cert -c gunicorn_config.py run_pgadmin:app
else
- exec gunicorn --timeout ${TIMEOUT} --bind ${PGADMIN_LISTEN_ADDRESS:-[::]}:${PGADMIN_LISTEN_PORT:-80} -w 1 --threads ${GUNICORN_THREADS:-25} --access-logfile ${GUNICORN_ACCESS_LOGFILE:--} -c gunicorn_config.py run_pgadmin:app
+ exec /venv/bin/gunicorn --timeout ${TIMEOUT} --bind ${PGADMIN_LISTEN_ADDRESS:-[::]}:${PGADMIN_LISTEN_PORT:-80} -w 1 --threads ${GUNICORN_THREADS:-25} --access-logfile ${GUNICORN_ACCESS_LOGFILE:--} -c gunicorn_config.py run_pgadmin:app
fi
diff --git a/web/config.py b/web/config.py
index 8fd8b0391..84792f58c 100644
--- a/web/config.py
+++ b/web/config.py
@@ -350,7 +350,7 @@ SESSION_COOKIE_NAME = 'pga4_session'
# These settings are used when running in web server mode for confirming
# and resetting passwords etc.
# See: http://pythonhosted.org/Flask-Mail/ for more info
-MAIL_SERVER = 'localhost'
+MAIL_SERVER = '127.0.0.1'
MAIL_PORT = 25
MAIL_USE_SSL = False
MAIL_USE_TLS = False
view thread (2+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: UBI based container patch
In-Reply-To: <CA+OCxoz0JKkgw5tkiqK1FemLhB3O4FuXu7MOBF0B=6EYyvBqQA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox