Subject: Re: what is up with the PG mailing lists?

From: "Marc G. Fournier" <[email-masked]>
Date: Thu, 01 Nov 2007 19:05:07 -0300

This is where I hate debugging email issues ... Bruce IMd me about this, I 
looked around and the only problem that I could see was that the caching DNS 
server within the VPS itself was still reporting one scanner, so I restarted 
named, and postfix after to clear out any caches it might have, and mail 
started flowing again ...

The only thing I have in the log file is:

Nov  1 14:22:41 postgresql postfix/smtpd[26360]: E4E039FC8E6: 
client=hub.org[200.46.204.220]
Nov  1 14:22:41 postgresql postfix/cleanup[73513]: E4E039FC8E6: 
message-id=<[email-masked]>
Nov  1 14:23:58 postgresql postfix/qmgr[7655]: E4E039FC8E6: 
from=<[email-masked]>, size=3553, nrcpt=1 (queue active)
Nov  1 16:12:19 postgresql postfix/smtp[51424]: E4E039FC8E6: 
to=<[email-masked]>, 
orig_to=<[email-masked]>, relay=maia.hub.org[200.46.204.184]:10024, 
conn_use=7, delay=6578, delays=78/6496/0.01/3.9, dsn=2.6.0, status=sent (250 
2.6.0 Ok, id=62395-04-7, from MTA: 250 2.0.0 Ok: queued as 9E4B49FC802)
Nov  1 16:12:19 postgresql postfix/qmgr[7655]: E4E039FC8E6: removed

Which shows the 'delay', but, there are no errors on the scanner server, nor 
did I touch either of htem to fix the problem ...

so it looks like the scanner received it and hung trying to send it back, until 
I reset the receiving side, whereby it finished ..

But there is nothing on either side to indicate a problem ... but restarting 
postfix 'unstuck' it ...

- --On Thursday, November 01, 2007 15:23:12 -0400 Tom Lane <[email-masked]> 
wrote:

> "Marc G. Fournier" <[email-masked]> writes:
>> But, again, it would be interesting to look at a specific one, with
>> message-id,  and see what the logs show for reasons of delays ...
>
> Well, the current problems seem to be entirely inside hub.org, and so
> all this banter about DNS etc seems not relevant.  A handy example is
> this same message I'm replying to, which seems to have been hung up
> for 2.5 hours:
>
> Received: from maia-2.hub.org (maia-2.hub.org [200.46.204.187] (may be
> forged)) 	by sss.pgh.pa.us (8.14.1/8.14.1) with ESMTP id lA1JCPC2010552
> 	for <[email-masked]>; Thu, 1 Nov 2007 15:12:26 -0400 (EDT)
> Received: from postgresql.org (postgresql.org [200.46.204.71])
> 	by maia-2.hub.org (Postfix) with ESMTP id 912252C9544
> 	for <[email-masked]>; Thu,  1 Nov 2007 16:12:22 -0300 (ADT)
> Received: from localhost (unknown [200.46.204.184])
> 	by postgresql.org (Postfix) with ESMTP id 9E4B49FC802
> 	for <[email-masked]>; Thu,  1 Nov 2007 16:12:18
> -0300 (ADT) Received: from postgresql.org ([200.46.204.71])
>  by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024)
>  with ESMTP id 62395-04-7 for <[email-masked]>;
>  Thu,  1 Nov 2007 16:12:11 -0300 (ADT)
> Received: from hub.org (hub.org [200.46.204.220])
> 	by postgresql.org (Postfix) with ESMTP id E4E039FC8E6
> 	for <[email-masked]>; Thu,  1 Nov 2007 14:22:40 -0300 (ADT)
> Received: from localhost (unknown [200.46.204.184])
> 	by hub.org (Postfix) with ESMTP id A428DB475C8
> 	for <[email-masked]>; Thu,  1 Nov 2007 13:43:35 -0300 (ADT)
> Received: from hub.org ([200.46.204.220])
>  by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024)
>  with ESMTP id 56221-03; Thu,  1 Nov 2007 13:43:01 -0300 (ADT)
> Received: from fserv.hub.org (blk-137-93-67.eastlink.ca [24.137.93.67])
> 	by hub.org (Postfix) with ESMTP id BB0BCB47581;
> 	Thu,  1 Nov 2007 13:43:02 -0300 (ADT)
> Received: from [192.168.1.2] (unknown [192.168.1.2])
> 	by fserv.hub.org (Postfix) with ESMTP id AD2C34E165;
> 	Thu,  1 Nov 2007 13:43:02 -0300 (ADT)
> Date: Thu, 01 Nov 2007 13:42:31 -0300
> From: "Marc G. Fournier" <[email-masked]>
> To: "Joshua D. Drake" <[email-masked]>
> cc: Magnus Hagander <[email-masked]>,
>         Andrew Sullivan <[email-masked]>, [email-masked]
> Subject: Re: [pgsql-www] what is up with the PG mailing lists?
> Message-ID: <[email-masked]>
> In-Reply-To: <20071101092806.3b1fa452@scratch>
> References: <[email-masked]>
>  	<[email-masked]>	<[email-masked]>
>  	<[email-masked]>	<[email-masked]>
>  	<[email-masked]>	<20071101080959.49f3087b@scratch>
>  	<[email-masked]>	<[email-masked]>
>  	<[email-masked]>	<[email-masked]>
>  	<[email-masked]>
> <20071101092806.3b1fa452@scratch> X-Mailer: Mulberry/4.0.8 (Linux/x86)
> MIME-Version: 1.0
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
> X-Virus-Scanned: Maia Mailguard 1.0.1
> X-Virus-Scanned: Maia Mailguard 1.0.1
> X-Mailing-List: pgsql-www
> List-Archive: <http://archives.postgresql.org/pgsql-www>
> List-Help: <mailto:[email-masked]?body=help>
> List-ID: <pgsql-www.postgresql.org>
> List-Owner: <mailto:[email-masked]>
> List-Post: <mailto:[email-masked]>
> List-Subscribe: <mailto:[email-masked]?body=sub%20pgsql-www>
> List-Unsubscribe: <mailto:[email-masked]?body=unsub%20pgsql-www>
> Precedence: bulk
> Sender: [email-masked]
>
> 			regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>        choose an index scan if your joining column's datatypes do not
>        match

- ----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . [email-masked]                              MSN . [email-masked]
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664

---

From: Magnus Hagander <[email-masked]>
Date: Fri, 02 Nov 2007 08:32:38 +0100

> started flowing again ...

You know, that DNS hangy thing reminds me a *lot* of the problems we had
on wwwmaster prior to the move to tribble. Now, I don't think the move
to tribble itself fixed it, but we did upgrade FreeBSD at that time, no?
Does this machine by any chance have the same FreeBSD release as the
*old* wwwmaster?

> But there is nothing on either side to indicate a problem ... but restarting 

We used to have monitoring on the mailqueue length on svr1, that could
catch these things. actually, I see that we still have monitoring, but
we've disabled notifications for it. Perhaps we should re-enable that to
keep track ofthings properly?
(Won't work for the notifies sent through -slaves though, since they'll
also get caught in the backlog if that happens)

BTW, I'm seeing much better times this morning. Just did another run,
this is the data for the past 9 hours:
                            path                            | count | avg
------------------------------------------------------------+-------+-----
 postgresql.org -> mx2.hub.org                              |    21 | 169
 mx2.hub.org -> svr2.hagander.net                           |    21 |   1
 svr2.hagander.net -> svr2.hagander.net                     |    21 |   0
 postgresql.org -> localhost (mx1.hub.org [200.46.204.184]) |    13 |  18
 localhost (mx1.hub.org [200.46.204.184]) -> postgresql.org |    13 |  13
 localhost (mx1.hub.org [200.46.204.183]) -> postgresql.org |     8 |   6
 postgresql.org -> localhost (mx1.hub.org [200.46.204.183]) |     8 |   5
 sss.pgh.pa.us (8.14.1/8.14.1) -> postgresql.org            |     4 |  15
 ug-out-1314.google.com -> postgresql.org                   |     3 |   1
 wdscexfe01.sc.wdc.com -> postgresql.org                    |     2 |  17

169 seconds is a lot more reasonable. It's still a bit of time, but I
assume that's when it does the spamscanning? (sice it delivers to my
machine in no more than a second after that)

//Magnus

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 2 Nov 2007 17:13:17 -0400

> keep track ofthings properly?

This seems like an obvious thing to do, from where I sit.  It's going
to be rather hard to debug problems that one doesn't notice.

> machine in no more than a second after that)

I'd expect spamscanning to be a significant overhead, yes.

A

-- 
Andrew Sullivan  | [email-masked]
The very definition of "news" is "something that hardly ever happens."	
		--Bruce Schneier