Date: Mon, 5 Dec 2016 11:42:43 -0600
From: Nico Williams <nico@cryptonector.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>,
	Christian Convey <christian.convey@gmail.com>,
	Petr Jelinek <petr@2ndquadrant.com>,
	Pavel Stehule <pavel.stehule@gmail.com>,
	David Fetter <david@fetter.org>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Tackling JsonPath support
Message-ID: <20161205174242.GB32541@localhost>
References: 
 <CAPfS4ZwgHT5FkRm8LE0ucG97e36ziBm0Pc-GkyGC15ksTwVFpw@mail.gmail.com>
 <CAFj8pRAtvZAtA2OBiyZyjf+AC24g6oSuKsbA8uWGZFZOoq=hDw@mail.gmail.com>
 <CAPfS4ZzcAdX6S3389rO-xfULvYuvD6VCACoxNJK=sUmhsnj0ow@mail.gmail.com>
 <CAFj8pRC8FFX9=+yi_im8bBJP0GJrv92hSuoFUH0nft62QLk0oQ@mail.gmail.com>
 <80228378-f5ba-1291-872b-9812e103e37c@2ndquadrant.com>
 <CAPfS4Zyjh4F9OWTsc4EJKh=1NR_qU+7GLZr0BC4bq9KULEf75Q@mail.gmail.com>
 <CA+TgmoZ28byiVjXTHO=B3jbOWcVk7dVySvJGc7wBF0ecPjOBhg@mail.gmail.com>
 <20161202213223.GJ24797@localhost>
 <CA+TgmoZBcdKO=v9rHmf=KhNjgPvCL_PUmKwZOHJTUgcuP1hBrg@mail.gmail.com>
 <18951.1480956777@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <18951.1480956777@sss.pgh.pa.us>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On Mon, Dec 05, 2016 at 11:52:57AM -0500, Tom Lane wrote:
> Another point here is that packagers such as Red Hat strenuously dislike
> such source-code-level wrapping of other projects, because that means that
> they have to rebuild multiple packages to fix any bugs found in the
> wrapped code.  If I were still packaging Postgres for Red Hat, and such
> a distribution landed in my inbox, the very first thing I'd be looking
> to do is rip out the borrowed code and replace it with a runtime
> shared-library dependency on the upstream project's official library.

I'm quite aware :(  I used to work at Sun on Solaris.  We too hated
duplication.  OpenSSL was a particularly problematic case...

There is a real tension between the release trains of many distinct open
source projects and those of their consumers, and those of distros/OSes.

Some projects, such as SQLite3, explicitly recommend copying their
source or statically linking them into dependents; distros/vendors never
like this.

My best advice on this (PG might benefit from it), informed by years of
experience dealing with this, is that there's no perfect answer, but
that nonetheless library developers should always follow these best
practices so as to help those who end up having to deal with multiple
versions of those libraries:

 - prefer dynamic linking (because dynamic linking semantics are
   superior to static linking semantics)

 - make libraries self-initialize and self-finalize!  (pthread_once()
   and Win32's ExecuteOnce* are your friends, as are atexit()/on_exit(),
   pthread_key_create(), and DllMain() on Windows)

   (otherwise calling your library from another library gets tricky)

 - make it so that as long as you change SONAMEs you can have multiple
   versions of the library loaded in one process, specifically:

    - don't use POSIX file locking (but the new non-POSIX OFD locks are
      OK) (or use them for files that wouldn't be shared across multiple
      versions in one process)

      (e.g., SQLite3 uses POSIX file locking safely, but it's not likely
      that two consumers of different SQLite3 versions in one process
      would access the same DB files, so it kinda works)

    - be backwards- and forwards-compatible as to any config file
      formats and other state that will be shared by multiple versions

 - generally: mind backwards compatibility, both source and binary, so
   as to make it easy to upgrade

    - this means applying good API design best practices that I won't go
      into here

 - write thread-safe code, and preferably fork-safe code too

For example, I've seen OpenSSL built with different SONAMEs to support
multiple versions of OpenSSL coexisting in a single program/process.
That actually works.

> Having said that ... we have a *really bad* track record of deciding which
> outside projects we want to depend on, or maybe we've just outlived a lot
> of them.  Aside from Robert's examples, there's uuid-ossp and libxml2,
> which are external code but have caused us headaches anyway.  So I think
> there's a lot to be said for avoiding dependencies on libraries that may
> or may not still be getting actively maintained ten years from now.

I'm not at all surprised.

One codebase I help develop and maintain, Heimdal, includes SQLite3 and
libeditline, and parts of Heimdal should really be separate projects
(e.g., its ASN.1 compiler and library, and several supporting libraries
like libroken (a portability layer)) because they could be useful to
others outside Heimdal.  Finding the right balance is not trivial.

Nico
-- 


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers