Subject: Problems with pgsql-announce

From: Josh Berkus <[email-masked]>
Date: Mon, 8 Jan 2007 10:55:17 -0800

All,

1) I thougth we'd set up pgsql-announce so that it didn't bounce to the 
sender?   I'm still getting 20-30 bounce messages every time I post to 
-announce.

2) For some reason, I got four duplicate checksum warnings for *one* 
posting to announce.  Example:

Duplicate posting detected
Duplicate Message-ID - <[email-masked]> (Mon Jan  8 
13:46:12 2007)
Duplicate Partial Message Checksum (Mon Jan  8 13:46:12 2007)

I'm certain that I sent the message only once, and it showed up on the list 
promptly.  Huh?

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: "Dan Langille" <[email-masked]>
Date: Mon, 08 Jan 2007 13:57:02 -0500

> sender?   I'm still getting 20-30 bounce messages every time I post to 

That is often not the sending MTAs fault.  It is often the fault of 
the receiving MTA which does not or can not differentiate between 
sender and author, and replies to the author regardless.

-- 
Dan Langille : Software Developer looking for work
my resume: http://www.freebsddiary.org/dan_langille.php
PGCon - The PostgreSQL Conference - http://www.pgcon.org/

---

From: "Greg Sabino Mullane" <[email-masked]>
Date: Mon, 8 Jan 2007 19:35:14 -0000

> sender?   I'm still getting 20-30 bounce messages every time I post to

Could be misconfigured on the sender's end. If you want, send me a list
of the offenders and I'll remove 'em. That goes for anyone else as well.

> posting to announce.  Example:

Yeah, something weird is going on there. From looking at the moderation
queue, it seems everyone is getting three or four duplicates. Maybe
we can simply turn that off, Marc?

- --
Greg Sabino Mullane [email-masked]
PGP Key: 0x14964AC8 200701081430
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

---

From: Magnus Hagander <[email-masked]>
Date: Mon, 08 Jan 2007 20:59:19 +0100

> we can simply turn that off, Marc?

I think we've seen a lot of examples of that "partial message duplicate"
filter blocking things it shouldn't. I know Dave had some problems
before, and at some point it started blocking almost all mail to the
slaves list (because those are very similar *by design*).

Do we have any indication that this helps *at all*? If not, we should
just turn it off on all the lists...

Which reminds me - I saw several emails pop through today after spending
>48 hours in the Maia Mailguard. Does that one actually help noticably
for list traffic, or just for mailboxes? Because it certainly happens
often that it slows down or stops list traffic completely..

//Magnus

---

From: Josh Berkus <[email-masked]>
Date: Mon, 8 Jan 2007 12:02:53 -0800

Greg,

> we can simply turn that off, Marc?

... not if duplicates are actually being received; we don't want 3-4 or 
each message.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: "Greg Sabino Mullane" <[email-masked]>
Date: Mon, 8 Jan 2007 20:48:56 -0000

> ... not if duplicates are actually being received; we don't want 3-4 or

I'm not sure what's going on, but it was working before, and now is not,
so the recent "cure" may be worse than the disease. I'm pretty sure these
are not people *sending* duplicate messages: for example, yesterday's
Weekly News appeared three times in the moderation queue, which it never
has before. Perhaps it was getting flagged as a duplicate before but not
sent to the moderation queue? Somehow I doubt that everyone's mail clients
are misconfigured to be sending duplicate mails, so the problem is on
our end.

Hm...just checked the -announce queue...their are three identical messages
from Josh Berkus about the new releases, all claiming the checksum errors.
Josh, did you ever get those checksum errors bounced back to you before the
change when you posted previously to announce? ISTR this change started with
someone complaining about getting those sent back to them?

- --
Greg Sabino Mullane [email-masked]
PGP Key: 0x14964AC8 200701081540
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

---

From: Josh Berkus <[email-masked]>
Date: Mon, 8 Jan 2007 14:29:29 -0800

Greg,

> Hm...just checked the -announce queue...their are three identical
> messages from Josh Berkus about the new releases, all claiming the
> checksum errors. Josh, did you ever get those checksum errors bounced
> back to you before the change when you posted previously to announce?
> ISTR this change started with someone complaining about getting those
> sent back to them?

Today's announcement is the first time it's happened to me.

-- 
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

---

From: "Marc G. Fournier" <[email-masked]>
Date: Thu, 11 Jan 2007 23:12:49 -0400

> just turn it off on all the lists...

If ppl want me to remove it altogether, I can ... and we'll see what happens 
... you call it :)

> often that it slows down or stops list traffic completely..

'k, nothing shoudl be spending >48 hours in the Maia Mailguard ... all it is is 
amavisd on steroids ... it only quarantines email marked as spam (>5) and I go 
through it periodically to make sure un-quarantine emails to help train the 
bayes filters ... most of what gets 'falsely stuck' tend to be 
subscribe/unsubscribe requests though ... if you see one that has been 
'spending 48 hours', please forward it to me with full headers, as something 
else is wrong ...

Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . [email-masked]                              MSN . [email-masked]
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664

---

From: "Marc G. Fournier" <[email-masked]>
Date: Thu, 11 Jan 2007 23:14:48 -0400

> ... not if duplicates are actually being received; we don't want 3-4 or

Right now, any duplicates are supposed to be shuffled to the moderators queue 
to deal with ... what the filter is *supposed* to deal with is spam, which 
tends to be alot of duplicates of the same message with different headers ...

If I remove the duplicate altogether, I suspect that there won't be much change 
on the lists themselves, but I don't know for sure ...

- ----

---

From: Dave Page <[email-masked]>
Date: Fri, 12 Jan 2007 08:39:58 +0000

Marc G. Fournier wrote:
> 
> 
> --On Monday, January 08, 2007 12:02:53 -0800 Josh Berkus <[email-masked]> 
> wrote:
> 
>> Greg,
> 

Don't think I ever saw one in the queue. Plenty of bounces though...

> on the lists themselves, but I don't know for sure ...

It hasn't made any difference whatsoever to the slaves list, except that
we now get the reports we expect to see. I realise there's a difference
between that and other lists, but it does still get spam.

/D

---

From: Robert Treat <[email-masked]>
Date: Fri, 12 Jan 2007 08:35:07 -0500

On Friday 12 January 2007 03:39, Dave Page wrote:
> > Right now, any duplicates are supposed to be shuffled to the moderators
> > queue to deal with ... what the filter is *supposed* to deal with is
> > spam, which tends to be alot of duplicates of the same message with
> > different headers ...
>
> > If I remove the duplicate altogether, I suspect that there won't be much
> > change on the lists themselves, but I don't know for sure ...
>

We could change some of the more public but lower traffic lists, like 
pgsql-www, first and see how that goes. 

-- 
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

---

From: Magnus Hagander <[email-masked]>
Date: Fri, 12 Jan 2007 15:28:24 +0100

> 'spending 48 hours', please forward it to me with full headers, as something 

I've definitly seen that happen and more than once. DOn't have one
around, but I'll eb sure to collect the headers next time it happens.

//Magnus

---

From: "Greg Sabino Mullane" <[email-masked]>
Date: Fri, 12 Jan 2007 14:45:08 -0000

> Right now, any duplicates are supposed to be shuffled to the
> moderators queue to deal with ... what the filter is *supposed*
> to deal with is spam, which tends to be alot of duplicates of the
> same message with different headers ...

Please turn it off entirely. I can easily identify spam, but
lately I've also had to deal with duplicate legitimate messages,
which means checking to see if the message has really been already
delivered or not, which means in effect I'm acting as a human checksum
checker, which kind of defeats the purpose of the automatic one.
I could simply trust the system and reject all messages marked as
duplicate checksums, but a false positive is what started all these
changes, if I recall correctly. If you want a good test, you could
remove it from -general: it gets many, many duplicates (and spam)
throughout the day, so I'll be able to give you feedback quick
on how the change affects that queue.

Thanks,
- --
Greg Sabino Mullane [email-masked]
PGP Key: 0x14964AC8 200701120941
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

---

From: Tom Lane <[email-masked]>
Date: Fri, 12 Jan 2007 10:08:51 -0500

Magnus Hagander <[email-masked]> writes:
>> most of what gets 'falsely stuck' tend to be 

> around, but I'll eb sure to collect the headers next time it happens.

Something else I've noticed saveral times in the last week is getting a
delayed copy of something that already came through.  These weren't the
usual newbie-sends-another-copy-after-subscribing deal because they came
from established subscribers.  I didn't save copies but will next time.

			regards, tom lane

---

From: Magnus Hagander <[email-masked]>
Date: Fri, 12 Jan 2007 16:18:57 +0100

> from established subscribers.  I didn't save copies but will next time.

I'm glad it's not just me. I've noticed this a couple of times, but
didn't have time to investigate at the time either - I thought it might
be my new MUA on the cellphone acting up ;-)

//Magnus

---

From: "Marc G. Fournier" <[email-masked]>
Date: Fri, 12 Jan 2007 13:26:30 -0400

> from established subscribers.  I didn't save copies but will next time.

Which is why I'm hesitant to remove the duplicate checks ...

Just curious though, next time you see this, if you notice it on time, could 
you send me the full headers?  I know this will be a hard one, since I imagine 
the second comes through as almost a 'deja vu' kind of experience, not 
simultaneously, but if you get 'lucky' enough to have them both at the same 
time, be interseting to see the complete message with full headers to compare 
...

- ----

---

From: "Marc G. Fournier" <[email-masked]>
Date: Sat, 13 Jan 2007 13:52:53 -0400

Try that ... see if there are a bunch of dups hitting the list now ... I can 
turn it back on quickly if there are ...

- --On Friday, January 12, 2007 14:45:08 +0000 Greg Sabino Mullane 
<[email-masked]> wrote:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> -----BEGIN PGP SIGNATURE-----
>
> iD8DBQFFp54gvJuQZxSWSsgRAkCCAJ4lCtdiNr+v4y8mGT0OjXOF73v4ZgCg72BD
> NUWrrNTkv2REIs2l8JsC+zY=
> =W7jB
> -----END PGP SIGNATURE-----
>
>

- ----

---

From: "Greg Sabino Mullane" <[email-masked]>
Date: Sat, 13 Jan 2007 18:26:23 -0000

>> from established subscribers.  I didn't save copies but will next time.

That was probably me as a result of the checksum moderation thing - I tend
to be conservative and approve messages when I am not sure, as it seems
better to have duplicates posted rather than to have nothing at all go
through. I just approved a duplicate checksum post on hackers by Christopher
Browne as a test.

It's still early, but so far the -general changes seem to be working out, Marc.

- --
Greg Sabino Mullane [email-masked]
PGP Key: 0x14964AC8 200701131322
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8