Subject: Re: [PATCH] pgarchives: parser: handle messages in which Message-ID
 is missing
To: Magnus Hagander <magnus@hagander.net>,
 Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: =?UTF-8?Q?C=c3=a9lestin_Matte?= <celestin.matte@cmatte.me>,
 pgsql-www@lists.postgresql.org
References: <53316703-db69-7067-c82e-47a598711595@cmatte.me>
 <202111041947.x5su2sdjdx74@alvherre.pgsql>
 <CABUevEyhYyx4DJ_7kENZry2wwZYz9R4J_oh8f0cKkUY+3Q8Kfw@mail.gmail.com>
From: Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
Message-ID: <61fb4078-6b27-9921-7f7d-0345978d8300@kaltenbrunner.cc>
Date: Fri, 5 Nov 2021 21:44:33 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: 
 <CABUevEyhYyx4DJ_7kENZry2wwZYz9R4J_oh8f0cKkUY+3Q8Kfw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/61fb4078-6b27-9921-7f7d-0345978d8300%40kaltenbrunner.cc>
Precedence: bulk

On 11/4/21 10:07 PM, Magnus Hagander wrote:
> 
> 
> On Thu, Nov 4, 2021 at 8:47 PM Alvaro Herrera <alvherre@alvh.no-ip.org 
> <mailto:alvherre@alvh.no-ip.org>> wrote:
> 
>     On 2021-Nov-04, Célestin Matte wrote:
> 
>      > > I don't think this should be the responsibility of pglister. As you
>      > > say, "most MTAs do add this field" -- and the solution is to
>      > > configure the MTA to do this. We already rely on the MTA to get a
>      > > lot of other important things right.
>      >
>      > But then these messages will get delivered by pglister but pgarchives
>      > will fail to archive them, although they do not actually break
>      > requirements. Shouldn't we follow the RFC here?
> 
> 
> I agree that the scenario is a problem, per below.  I don't agree that 
> making up an id is a solution to that problem.
> 
> 
>     Maybe pglister should refuse to deliver messages that don't contain
>     a Message-Id.
> 
> 
> It should. I actually thought it did already, but apparently it does 
> not. I guess we've only ever used it under properly configured MTAs :)
> 
> Have you actually come across any case where a *proper* non-spam message 
> is sent without a message-id and passes through actual mailservers on 
> the way?
> 
> Looking through the approximately 1.4 million mails in the postgres list 
> archives, not a single one has a message-id generated by the archives 
> server MTA (which is configured to generate it). Not a single one by our 
> inbound relay servers. And exactly one by the pglister server -- which 
> turns out to be a bounce that ended up in the archives because of a 
> misconfiguration back in 2018 that's not visible in the public archives.

as mentioned down-thread by Justin Clift we have been plain rejecting 
mails without a message-id on the postgresql.org inbound relays since 
March 27th 2012(!) according to our repo and the number of rejects due 
to that rule is actually not-insignificant (approximately 200-400/day 
with the majority being for a very small number of bounce generating 
senders) but the number of complaints is also approaching (almost) zero.

So the reason why pglister is not seeing them a lot is because we dont 
accept them upstream, not that they dont exist in the wild...

Though the ones in the wild seem to be "not very useful"...


Stefan

Stefan