Date: Sun, 24 Jan 2021 11:53:26 -0600
From: Justin Pryzby <pryzby@telsasoft.com>
To: Magnus Hagander <magnus@hagander.net>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-www@postgresql.org
Subject: Re: missing ML messages
Message-ID: <20210124175326.GA30308@telsasoft.com>
References: <20210123025941.GA30285@telsasoft.com>
 <1287447.1611371258@sss.pgh.pa.us>
 <20210123031431.GI27167@telsasoft.com>
 <CABUevEyYop_zjaCggv-FNd9HCRyb4D2xmj5+Wf16Xqp_PHFU=g@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CABUevEyYop_zjaCggv-FNd9HCRyb4D2xmj5+Wf16Xqp_PHFU=g@mail.gmail.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Precedence: bulk

On Sun, Jan 24, 2021 at 02:37:33PM +0100, Magnus Hagander wrote:
> On Sat, Jan 23, 2021 at 4:14 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> >
> > On Fri, Jan 22, 2021 at 10:07:38PM -0500, Tom Lane wrote:
> > > Justin Pryzby <pryzby@telsasoft.com> writes:
> > > > I noticed that one of my messages never made it to the HTTP page:
> > > > 20210119190720.GL8560@telsasoft.com
> > >
> > > I see it in the archives, and in my local mail folder too.
> > > https://www.postgresql.org/message-id/20210119190720.GL8560@telsasoft.com
> >
> > Ok...I must have entered it wrong somehow.
> >
> > So the problem is that it doesn't show up here:
> > https://www.postgresql.org/message-id/flat/20170907194236.4cefce96%40wp.localdomain
> >
> > Is it because I used two IDs in reply-to headers?
> >
> > In-Reply-To: <CAFiTN-updTo3aB+7vsECHOHpjB23Av5OfgZQrt90Q33gFK-oZQ@mail.gmail.com>
> >         <CAFiTN-utJPbTLQ9i10wT_zmHX=un+RQMB1B1xbkTgrh971vqjw@mail.gmail.com>
> >
> > In mutt, I would've 't'agged two messages, and then ';'-'r' to reply and quote both.
> 
> Yeah, I think that's it. We only parse a single value in the in-reply-to.
> 
> Normally this shouldn't be a problem, because normally the messageid
> is also listed in References,  but your message doesn't appear to
> contain *any* References header. Which should normally be a list of
> all the previous messages in the thread.
> 
> Have you done something in your config to specifically turn off adding
> of References headers? It seems strange we haven't seen this before...

I suspect I *have* seen it before, but in a marginal context so I never
reported it.

It looks like mutt doesn't use References when there's multiple parents.

headers.c:
|    /* in case the user modifies/removes the In-Reply-To header with
|       $edit_headers set, we remove References: as they're likely invalid;
|       we can simply compare strings as we don't generate References for
|       multiple Message-Ids in IRT anyways */
|    if (sctx->msg->env->in_reply_to &&
|        (!n->in_reply_to || mutt_strcmp (n->in_reply_to->data,
|                                         sctx->msg->env->in_reply_to->data) != 0))
|      mutt_free_list (&sctx->msg->env->references);


send.c:
|  /* if there's more than entry in In-Reply-To (i.e. message has
|     multiple parents), don't generate a References: header as it's
|     discouraged by RfC2822, sect. 3.6.4 */
|  if (ctx->tagged > 0 && env->in_reply_to && env->in_reply_to->next)
|    mutt_free_list (&env->references);


https://tools.ietf.org/html/rfc2822#section-3.6.4
|   The "In-Reply-To:" field will contain the contents of the "Message-
|   ID:" field of the message to which this one is a reply (the "parent
|   message").  If there is more than one parent message, then the "In-
|   Reply-To:" field will contain the contents of all of the parents'
|   "Message-ID:" fields.
|
|   The "References:" field will contain the contents of the parent's
|   "References:" field (if any) followed by the contents of the parent's
|   "Message-ID:" field (if any).  If the parent message does not contain
|   a "References:" field but does have an "In-Reply-To:" field
|   containing a single message identifier, then the "References:" field
|   will contain the contents of the parent's "In-Reply-To:" field
|   followed by the contents of the parent's "Message-ID:" field (if
|   any).  If the parent has none of the "References:", "In-Reply-To:",
|   or "Message-ID:" fields, then the new message will have no
|   "References:" field.
|
|   Note: Some implementations parse the "References:" field to display
|   the "thread of the discussion".  These implementations assume that
|   each new message is a reply to a single parent and hence that they
|   can walk backwards through the "References:" field to find the parent
|   of each message listed there.  Therefore, trying to form a
|   "References:" field for a reply that has multiple parents is
|   discouraged and how to do so is not defined in this document.


So mutt's behavior is deliberate and recommended by standard, so I think this
is up to pglister to (try to) handle IRT with multiple IDs.  It seems to me
like it currently either does nothing, or doesn't split the field, so ends up
trying to us an ID that's actually two catenated IDs.  Maybe it'd be fine (or
at least an improvement) to just use the first ID, or the first one that's
known to pglister.

-- 
Justin