Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mia3C-0005Fo-Qm for pgsql-www@arkaria.postgresql.org; Thu, 04 Nov 2021 10:32:54 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1mia3B-0003cS-Pf for pgsql-www@arkaria.postgresql.org; Thu, 04 Nov 2021 10:32:53 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mia3B-0003cJ-JQ for pgsql-www@lists.postgresql.org; Thu, 04 Nov 2021 10:32:53 +0000 Received: from mail-lf1-x133.google.com ([2a00:1450:4864:20::133]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1mia38-0003dR-Op for pgsql-www@lists.postgresql.org; Thu, 04 Nov 2021 10:32:53 +0000 Received: by mail-lf1-x133.google.com with SMTP id bi35so10981264lfb.9 for ; Thu, 04 Nov 2021 03:32:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hagander-net.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=h6KBCnARIYyJNfwDNfOFDkWS/0VJ5brt+y+xudDk4Zo=; b=Vlt4HKVkFWpRr3fowv1cCn1DsDHPnwbonX+9nD+W6OGwQpttZY6TMLoL6s7GJ7coO0 9bPyQIkiQQk2rwD9Za2osElJ3AJcYxorNB7q5ewuMA/NZU8aER0gNoH8+lgjylZNGGsN w88UHn/dpKiB4o2kh8cq8fGBwMu0zCy/FOKNmT5wSIRQeaAp59GT37HE6IW27RYD5BQL IDG6C2gF/c5uzzNgUaw5rzZ5bjHJSFSm8/TAUEc4GV63BRqVbD1y19V6qtecd3aoxlDW FuhP4C9wUKXcFuQ9lA7qYpWpaoG8AXtcMWERCr1Y73wT6srPTA5z+HgEZ2gv6znUzoFj SLeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=h6KBCnARIYyJNfwDNfOFDkWS/0VJ5brt+y+xudDk4Zo=; b=PrvWP1l3nlXFckRIN6xYcuuPHozEdtuHmH9dBYIqN62mH3NixR5HR9oR2ITS/UxKz9 Y7iwJZ1vg3zZpTF4I1OglXx3O72LtOGVZY8rwaiyPzbgOFMl3YodmYT8WrXEy+zyf58J IX/7aiOVLQEp0QtkDj1Qwt5fH0prYIc0qfSse6ctUIXR7P+UIXnTcADntwSHUYJ05PQ9 0q2k2nbOFRY4vszkaGYKrBxjJsl37Ame/imX49kW6k6t+fI7syS+YL/cjO/Hgn1HbcL+ vJ8MCt3i2WwtMWOoP8bfi0husAaURnZl9w/f4KJytoET3h7gYouJjLXIvupDEUN9wqds QGiA== X-Gm-Message-State: AOAM532twzFjKXgr+Bg8TFIyRQHDY4PUZgRX8M6nPeVpcdOCZcNyMGgO iRS91V7H2VqVjHx2tqMOQPaFM/t4AHu+/OsFJQYs/UixWWc= X-Google-Smtp-Source: ABdhPJz4li2cgvJeL43BenD+u9loeCWRlsuxYr+fiXGrdUwRu8tNqyeBTiIcWz2Gf+G8ei17Cc3uvZh10yIacvmLTt0= X-Received: by 2002:a19:5004:: with SMTP id e4mr31740120lfb.497.1636021970178; Thu, 04 Nov 2021 03:32:50 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Magnus Hagander Date: Thu, 4 Nov 2021 11:32:37 +0100 Message-ID: Subject: Re: [PATCH] pgarchives: parser: handle messages in which Message-ID is missing To: =?UTF-8?Q?C=C3=A9lestin_Matte?= Cc: pgsql-www@lists.postgresql.org Content-Type: multipart/alternative; boundary="00000000000060b12d05cff40c9b" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --00000000000060b12d05cff40c9b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Nov 4, 2021 at 11:31 AM Magnus Hagander wrote= : > > > On Wed, Nov 3, 2021 at 6:02 PM C=C3=A9lestin Matte > wrote: > >> Hello, >> >> As surprising as it may seem, Message-ID is actually not a mandatory >> email field [1]. While most MTAs do add this field, some might not, and >> this will cause load_message.py to crash. >> As a solution to this, when this field is missing, this patch: >> - attempts to find a "Sent-Message-ID" header and use it as the >> Message-ID (a case I encountered when trying to import an old mbox) >> - generates a new Message-ID if none exists, following (a simpler versio= n >> of) [2]. >> > > I don't think this should be the responsibility of pglister. As you say, > "most MTAs do add this field" -- and the solution is to configure the MTA > to do this. We already rely on the MTA to get a lot of other important > things right. > Sorry, I mean pgarchives here of course, not pglister :) --=20 Magnus Hagander Me: https://www.hagander.net/ Work: https://www.redpill-linpro.com/ --00000000000060b12d05cff40c9b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Nov 4, 2021 at 11:31 AM Magnu= s Hagander <magnus@hagander.net> wrote:


On Wed, Nov 3, 2021 at 6:02 PM C=C3=A9l= estin Matte <celestin.matte@cmatte.me> wrote:
Hello,

As surprising as it may seem, Message-ID is actually not a mandatory email = field [1]. While most MTAs do add this field, some might not, and this will= cause load_message.py to crash.
As a solution to this, when this field is missing, this patch:
- attempts to find a "Sent-Message-ID" header and use it as the M= essage-ID (a case I encountered when trying to import an old mbox)
- generates a new Message-ID if none exists, following (a simpler version o= f) [2].

I don't think this should b= e the responsibility of pglister. As you say, "most MTAs do add this f= ield" -- and the solution is to configure the MTA to do this. We alrea= dy rely on the MTA to get a lot of other important things right.

Sorry, I mean pgarchives here of co= urse, not pglister :)
=C2=A0

--
=C2=A0Magnus = Hagander
=C2=A0Me: https://www.hagander.net/
=C2=A0Work: https://www.redpill-linpro.com/
<= /div>
--00000000000060b12d05cff40c9b--