Date: Sun, 13 Aug 2023 11:32:08 +0200 (CEST)
From: Fabien COELHO <coelho@cri.ensmp.fr>
Reply-To: Fabien COELHO <coelho@cri.ensmp.fr>
To: PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>
Subject: pgbench with libevent?
Message-ID: <f6a5e724-e2d2-af78-4a6-4547994761b1@minesparis.psl.eu>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="1381411976-1327593738-1691919133=:138234"
Archived-At: <https://www.postgresql.org/message-id/f6a5e724-e2d2-af78-4a6-4547994761b1%40minesparis.psl.eu>
Precedence: bulk

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--1381411976-1327593738-1691919133=:138234
Content-Type: text/plain; format=flowed; charset=UTF-8
Content-Transfer-Encoding: 8BIT


Hello devs,

Pgbench is managing clients I/Os manually with select or poll. Much of 
this could be managed by libevent.

Pros:

1. libevent is portable, stable, and widely used (eg Chromium, Memcached, PgBouncer).

2. libevent implements more I/O wait methods, which may be more efficient on some platforms
    (eg FreeBSD kqueue, Windows wepoll in libevent 2.2 alpha), and hides portability issues.

3. it would remove significant portions of unattractive pgbench code, esp. in threadRun,
    and around socket/poll abstraction and portability layer.

4. depending on the number of event loops, the client load could be shared more evenly.
    currently a thread only manages its own clients, some client I/Os may be waiting to be
    processed while other threads could be available to process them.

Cons:

1. it adds a libevent dependency to postgres. This may be a no go from the start.

2. this is a significant refactoring project which implies a new internal architecture and adds
    new code to process and generate appropriate events.

3. libevent ability to function efficiently in a highly multithreaded environment
    is unclear. Should there be one event queue which generate a shared work queue?
    or several event queues, one per thread (which would remove the sharing pro)?
    or something in between? Some experiments and configuratibility may be desirable.
    This may also have an impact on pgbench user interface and output depending on the result,
    eg there may be specialized event and worker threads, some statistics may be slightly
    different, new options may be needed…

4. libevent development seems slugish, last bugfix was published 3 years ago, version
    2.2 has been baking for years, but the development seems lively (+100 contributors).

Neutral?

1. BSD 3 clauses license.

Is pros > cons, or not? Other thoughts, pros, cons?

-- 
Fabien.
--1381411976-1327593738-1691919133=:138234--