pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
SQL logging (log_min_duration_statement) not working properly
10+ messages / 4 participants
[nested] [flat]

* SQL logging (log_min_duration_statement) not working properly
@ 2005-05-18 13:35 Pallav Kalva <pkalva@deg.cc>
  2005-05-18 14:39 ` Re: SQL logging (log_min_duration_statement) not working Scott Marlowe <smarlowe@g2switchworks.com>
  2005-05-18 15:18 ` Re: SQL logging (log_min_duration_statement) not working properly Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 2 replies; 10+ messages in thread

From: Pallav Kalva @ 2005-05-18 13:35 UTC (permalink / raw)
  To: pgsql-admin

Hi Everybody,


    I am having problems with logging long running SQL in the log file. 
I have set log_min_duration_statement = 5000 (5secs) and it is not 
logging sql statements that took over 5 secs and this is only from 
queries generated by Hibernate . here is a sample query from hibernate :

-----------------------------------------------------------------------------------------
select listingadd1_.addressid as col_0_0_
from listing.listing listing0_, listing.address listingadd1_
where ((select count(taxrolls2_.fkaddressid)
       from listing.taxroll taxrolls2_
       where listingadd1_.addressid=taxrolls2_.fkaddressid)=0 )
       and(listing0_.fkbestaddressid=listingadd1_.addressid )
       and(listing0_.entrydate>$1 )
order by  listingadd1_.addressid asc
--------------------------------------------------------------------------------------------

  This above query take more than 5 secs to run and it wont get logged , 
where as all other queries which are not generated by Hibernate are 
logged. Is this expected behavior ?  am I missing something else ?  
Please advice.

Thanks!
Pallav




^ permalink  raw  reply  [nested|flat] 10+ messages in thread

* Re: SQL logging (log_min_duration_statement) not working
  2005-05-18 13:35 SQL logging (log_min_duration_statement) not working properly Pallav Kalva <pkalva@deg.cc>
@ 2005-05-18 14:39 ` Scott Marlowe <smarlowe@g2switchworks.com>
  2005-05-18 14:49   ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
  1 sibling, 1 reply; 10+ messages in thread

From: Scott Marlowe @ 2005-05-18 14:39 UTC (permalink / raw)
  To: Pallav Kalva <pkalva@deg.cc>; +Cc: pgsql-admin

On Wed, 2005-05-18 at 08:35, Pallav Kalva wrote:
> Hi Everybody,
> 
> 
>     I am having problems with logging long running SQL in the log file. 
> I have set log_min_duration_statement = 5000 (5secs) and it is not 
> logging sql statements that took over 5 secs and this is only from 
> queries generated by Hibernate . here is a sample query from hibernate :
> 
> -----------------------------------------------------------------------------------------
> select listingadd1_.addressid as col_0_0_
> from listing.listing listing0_, listing.address listingadd1_
> where ((select count(taxrolls2_.fkaddressid)
>        from listing.taxroll taxrolls2_
>        where listingadd1_.addressid=taxrolls2_.fkaddressid)=0 )
>        and(listing0_.fkbestaddressid=listingadd1_.addressid )
>        and(listing0_.entrydate>$1 )
> order by  listingadd1_.addressid asc
> --------------------------------------------------------------------------------------------
> 
>   This above query take more than 5 secs to run and it wont get logged , 
> where as all other queries which are not generated by Hibernate are 
> logged. Is this expected behavior ?  am I missing something else ?  
> Please advice.

Are you sure it's taking more than 5 seconds to run, not that it's
taking <5 seconds to run and >5 seconds to return the data.  I'm pretty
sure the time it takes to ship out the data isn't counted in the time
used to decide whether to log the query or not by the postmaster.

I.e. what does explain analyze QUERY HERE say about the execution time?



^ permalink  raw  reply  [nested|flat] 10+ messages in thread

* Re: SQL logging (log_min_duration_statement) not working
  2005-05-18 13:35 SQL logging (log_min_duration_statement) not working properly Pallav Kalva <pkalva@deg.cc>
  2005-05-18 14:39 ` Re: SQL logging (log_min_duration_statement) not working Scott Marlowe <smarlowe@g2switchworks.com>
@ 2005-05-18 14:49   ` Pallav Kalva <pkalva@deg.cc>
  2005-05-18 15:51     ` Re: SQL logging (log_min_duration_statement) not working Scott Marlowe <smarlowe@g2switchworks.com>
  0 siblings, 1 reply; 10+ messages in thread

From: Pallav Kalva @ 2005-05-18 14:49 UTC (permalink / raw)
  To: Scott Marlowe <smarlowe@g2switchworks.com>; +Cc: pgsql-admin

Scott Marlowe wrote:

>On Wed, 2005-05-18 at 08:35, Pallav Kalva wrote:
>  
>
>>Hi Everybody,
>>
>>
>>    I am having problems with logging long running SQL in the log file. 
>>I have set log_min_duration_statement = 5000 (5secs) and it is not 
>>logging sql statements that took over 5 secs and this is only from 
>>queries generated by Hibernate . here is a sample query from hibernate :
>>
>>-----------------------------------------------------------------------------------------
>>select listingadd1_.addressid as col_0_0_
>>from listing.listing listing0_, listing.address listingadd1_
>>where ((select count(taxrolls2_.fkaddressid)
>>       from listing.taxroll taxrolls2_
>>       where listingadd1_.addressid=taxrolls2_.fkaddressid)=0 )
>>       and(listing0_.fkbestaddressid=listingadd1_.addressid )
>>       and(listing0_.entrydate>$1 )
>>order by  listingadd1_.addressid asc
>>--------------------------------------------------------------------------------------------
>>
>>  This above query take more than 5 secs to run and it wont get logged , 
>>where as all other queries which are not generated by Hibernate are 
>>logged. Is this expected behavior ?  am I missing something else ?  
>>Please advice.
>>    
>>
>
>Are you sure it's taking more than 5 seconds to run, not that it's
>taking <5 seconds to run and >5 seconds to return the data.  I'm pretty
>sure the time it takes to ship out the data isn't counted in the time
>used to decide whether to log the query or not by the postmaster.
>
>I.e. what does explain analyze QUERY HERE say about the execution time?
>
>  
>
Hi Scott,

     Thanks! for the reply. The query for sure runs for more than 5 
secs, I  passed the same value again and ran the query in the psql with 
explain analyze .
 
    Here is the explain analyze output :

explain analyze
select listingadd1_.addressid as col_0_0_
from listing.listing listing0_, listing.address listingadd1_
where ((select count(taxrolls2_.fkaddressid)
        from listing.taxroll taxrolls2_
        where listingadd1_.addressid=taxrolls2_.fkaddressid)=0 )
        and(listing0_.fkbestaddressid=listingadd1_.addressid )
        and(listing0_.entrydate>'2004-12-15' )
order by  listingadd1_.addressid asc;
                                                                                  
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=17503528.41..17503530.10 rows=678 width=4) (actual 
time=37279.353..37750.532 rows=398459 loops=1)
   Sort Key: listingadd1_.addressid
   ->  Nested Loop  (cost=0.00..17503496.52 rows=678 width=4) (actual 
time=5.185..31575.629 rows=398459 loops=1)
         ->  Seq Scan on listing listing0_  (cost=0.00..10312.25 
rows=135580 width=4) (actual time=4.885..1834.806 rows=406740 loops=1)
               Filter: (entrydate > '2004-12-15 00:00:00'::timestamp 
without time zone)
         ->  Index Scan using pk_address_addressid on address 
listingadd1_  (cost=0.00..129.01 rows=1 width=4) (actual 
time=0.065..0.067 rows=1 loops=406740)
               Index Cond: ("outer".fkbestaddressid = 
listingadd1_.addressid)
               Filter: ((subplan) = 0)
               SubPlan
                 ->  Aggregate  (cost=123.00..123.00 rows=1 width=4) 
(actual time=0.015..0.016 rows=1 loops=406740)
                       ->  Index Scan using idx_taxroll_fkaddressid on 
taxroll taxrolls2_  (cost=0.00..122.89 rows=42 width=4) (actual 
time=0.009..0.009 rows=0 loops=406740)
                             Index Cond: ($0 = fkaddressid)
 Total runtime: 38026.606 ms
(13 rows)




^ permalink  raw  reply  [nested|flat] 10+ messages in thread

* Re: SQL logging (log_min_duration_statement) not working
  2005-05-18 13:35 SQL logging (log_min_duration_statement) not working properly Pallav Kalva <pkalva@deg.cc>
  2005-05-18 14:39 ` Re: SQL logging (log_min_duration_statement) not working Scott Marlowe <smarlowe@g2switchworks.com>
  2005-05-18 14:49   ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
@ 2005-05-18 15:51     ` Scott Marlowe <smarlowe@g2switchworks.com>
  2005-05-18 18:06       ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
  0 siblings, 1 reply; 10+ messages in thread

From: Scott Marlowe @ 2005-05-18 15:51 UTC (permalink / raw)
  To: Pallav Kalva <pkalva@deg.cc>; +Cc: pgsql-admin

On Wed, 2005-05-18 at 09:49, Pallav Kalva wrote:
> Scott Marlowe wrote:
> 
> >On Wed, 2005-05-18 at 08:35, Pallav Kalva wrote:
> >  
> >
> >>Hi Everybody,
> >>
> >>
> >>    I am having problems with logging long running SQL in the log file. 
> >>I have set log_min_duration_statement = 5000 (5secs) and it is not 
> >>logging sql statements that took over 5 secs and this is only from 
> >>queries generated by Hibernate . here is a sample query from hibernate :
> >>
> >>-----------------------------------------------------------------------------------------
> >>select listingadd1_.addressid as col_0_0_
> >>from listing.listing listing0_, listing.address listingadd1_
> >>where ((select count(taxrolls2_.fkaddressid)
> >>       from listing.taxroll taxrolls2_
> >>       where listingadd1_.addressid=taxrolls2_.fkaddressid)=0 )
> >>       and(listing0_.fkbestaddressid=listingadd1_.addressid )
> >>       and(listing0_.entrydate>$1 )
> >>order by  listingadd1_.addressid asc
> >>--------------------------------------------------------------------------------------------
> >>
> >>  This above query take more than 5 secs to run and it wont get logged , 
> >>where as all other queries which are not generated by Hibernate are 
> >>logged. Is this expected behavior ?  am I missing something else ?  
> >>Please advice.
> >>    
> >>
> >
> >Are you sure it's taking more than 5 seconds to run, not that it's
> >taking <5 seconds to run and >5 seconds to return the data.  I'm pretty
> >sure the time it takes to ship out the data isn't counted in the time
> >used to decide whether to log the query or not by the postmaster.
> >
> >I.e. what does explain analyze QUERY HERE say about the execution time?
> >
> >  
> >
> Hi Scott,
> 
>      Thanks! for the reply. The query for sure runs for more than 5 
> secs, I  passed the same value again and ran the query in the psql with 
> explain analyze .
>  
>     Here is the explain analyze output :

SNIP

>    ->  Nested Loop  (cost=0.00..17503496.52 rows=678 width=4) (actual 
> time=5.185..31575.629 rows=398459 loops=1)

SNIP

>  Total runtime: 38026.606 ms
> (13 rows)

OK, two problems here, I assume.  One is your logging is dodgy right
now.  Is postgresql logging anything else?  Are you sure your logs are
going where you think they are, etc?

Secondly, it looks from the nested loop part of your explain analyze
that there's a part where the query planner THINKS there's gonna be 678
rows, but actually gets 398459, that's a miscalculation by a factor of
about 600.  Since there's no functions or anything I recognize as being
a common cause of such a mis-estimation, I'd guess that either you need
to analyze, or if you have analyzed, you may need to alter the stats
target for the column entrydate.



^ permalink  raw  reply  [nested|flat] 10+ messages in thread

* Re: SQL logging (log_min_duration_statement) not working
  2005-05-18 13:35 SQL logging (log_min_duration_statement) not working properly Pallav Kalva <pkalva@deg.cc>
  2005-05-18 14:39 ` Re: SQL logging (log_min_duration_statement) not working Scott Marlowe <smarlowe@g2switchworks.com>
  2005-05-18 14:49   ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
  2005-05-18 15:51     ` Re: SQL logging (log_min_duration_statement) not working Scott Marlowe <smarlowe@g2switchworks.com>
@ 2005-05-18 18:06       ` Pallav Kalva <pkalva@deg.cc>
  0 siblings, 0 replies; 10+ messages in thread

From: Pallav Kalva @ 2005-05-18 18:06 UTC (permalink / raw)
  To: Scott Marlowe <smarlowe@g2switchworks.com>; +Cc: pgsql-admin

Scott Marlowe wrote:

>On Wed, 2005-05-18 at 09:49, Pallav Kalva wrote:
>  
>
>>Scott Marlowe wrote:
>>
>>    
>>
>>>On Wed, 2005-05-18 at 08:35, Pallav Kalva wrote:
>>> 
>>>
>>>      
>>>
>>>>Hi Everybody,
>>>>
>>>>
>>>>   I am having problems with logging long running SQL in the log file. 
>>>>I have set log_min_duration_statement = 5000 (5secs) and it is not 
>>>>logging sql statements that took over 5 secs and this is only from 
>>>>queries generated by Hibernate . here is a sample query from hibernate :
>>>>
>>>>-----------------------------------------------------------------------------------------
>>>>select listingadd1_.addressid as col_0_0_
>>>>        
>>>>
>>>>from listing.listing listing0_, listing.address listingadd1_
>>>      
>>>
>>>>where ((select count(taxrolls2_.fkaddressid)
>>>>      from listing.taxroll taxrolls2_
>>>>      where listingadd1_.addressid=taxrolls2_.fkaddressid)=0 )
>>>>      and(listing0_.fkbestaddressid=listingadd1_.addressid )
>>>>      and(listing0_.entrydate>$1 )
>>>>order by  listingadd1_.addressid asc
>>>>--------------------------------------------------------------------------------------------
>>>>
>>>> This above query take more than 5 secs to run and it wont get logged , 
>>>>where as all other queries which are not generated by Hibernate are 
>>>>logged. Is this expected behavior ?  am I missing something else ?  
>>>>Please advice.
>>>>   
>>>>
>>>>        
>>>>
>>>Are you sure it's taking more than 5 seconds to run, not that it's
>>>taking <5 seconds to run and >5 seconds to return the data.  I'm pretty
>>>sure the time it takes to ship out the data isn't counted in the time
>>>used to decide whether to log the query or not by the postmaster.
>>>
>>>I.e. what does explain analyze QUERY HERE say about the execution time?
>>>
>>> 
>>>
>>>      
>>>
>>Hi Scott,
>>
>>     Thanks! for the reply. The query for sure runs for more than 5 
>>secs, I  passed the same value again and ran the query in the psql with 
>>explain analyze .
>> 
>>    Here is the explain analyze output :
>>    
>>
>
>SNIP
>
>  
>
>>   ->  Nested Loop  (cost=0.00..17503496.52 rows=678 width=4) (actual 
>>time=5.185..31575.629 rows=398459 loops=1)
>>    
>>
>
>SNIP
>
>  
>
>> Total runtime: 38026.606 ms
>>(13 rows)
>>    
>>
>
>OK, two problems here, I assume.  One is your logging is dodgy right
>now.  Is postgresql logging anything else?  Are you sure your logs are
>going where you think they are, etc?
>

Postgres is logging everything else , even the queries which take more 
than 5 sec which are
executed from psql not the Hibernate queries. The main concern for me 
was to check if the bad
queries which are executed from Hibernate are logged or not.

>
>Secondly, it looks from the nested loop part of your explain analyze
>that there's a part where the query planner THINKS there's gonna be 678
>rows, but actually gets 398459, that's a miscalculation by a factor of
>about 600.  Since there's no functions or anything I recognize as being
>a common cause of such a mis-estimation, I'd guess that either you need
>to analyze, or if you have analyzed, you may need to alter the stats
>target for the column entrydate.
>
>---------------------------(end of broadcast)---------------------------
>TIP 6: Have you searched our list archives?
>
>               http://archives.postgresql.org
>
>  
>





^ permalink  raw  reply  [nested|flat] 10+ messages in thread

* Re: SQL logging (log_min_duration_statement) not working properly
  2005-05-18 13:35 SQL logging (log_min_duration_statement) not working properly Pallav Kalva <pkalva@deg.cc>
@ 2005-05-18 15:18 ` Tom Lane <tgl@sss.pgh.pa.us>
  2005-05-18 15:25   ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
  1 sibling, 1 reply; 10+ messages in thread

From: Tom Lane @ 2005-05-18 15:18 UTC (permalink / raw)
  To: Pallav Kalva <pkalva@deg.cc>; +Cc: pgsql-admin

Pallav Kalva <pkalva@deg.cc> writes:
>     I am having problems with logging long running SQL in the log file. 
> I have set log_min_duration_statement = 5000 (5secs) and it is not 
> logging sql statements that took over 5 secs and this is only from 
> queries generated by Hibernate .

Did you set log_min_duration_statement in a way that would actually
affect Hibernate (ie, change postgresql.conf and sighup the postmaster)?

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 10+ messages in thread

* Re: SQL logging (log_min_duration_statement) not working
  2005-05-18 13:35 SQL logging (log_min_duration_statement) not working properly Pallav Kalva <pkalva@deg.cc>
  2005-05-18 15:18 ` Re: SQL logging (log_min_duration_statement) not working properly Tom Lane <tgl@sss.pgh.pa.us>
@ 2005-05-18 15:25   ` Pallav Kalva <pkalva@deg.cc>
  2005-05-18 15:41     ` Re: SQL logging (log_min_duration_statement) not working properly Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 10+ messages in thread

From: Pallav Kalva @ 2005-05-18 15:25 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-admin

Tom Lane wrote:

>Pallav Kalva <pkalva@deg.cc> writes:
>  
>
>>    I am having problems with logging long running SQL in the log file. 
>>I have set log_min_duration_statement = 5000 (5secs) and it is not 
>>logging sql statements that took over 5 secs and this is only from 
>>queries generated by Hibernate .
>>    
>>
>
>Did you set log_min_duration_statement in a way that would actually
>affect Hibernate (ie, change postgresql.conf and sighup the postmaster)?
>
>			regards, tom lane
>
>  
>
I changed the log_min_duration_statement from 60000 to 5000 and then 
ran  --> pg_ctl  reload -D <data cluster location> and then i ran the 
queries from hibernate. These are the only things which I did on the 
postgres side.




^ permalink  raw  reply  [nested|flat] 10+ messages in thread

* Re: SQL logging (log_min_duration_statement) not working properly
  2005-05-18 13:35 SQL logging (log_min_duration_statement) not working properly Pallav Kalva <pkalva@deg.cc>
  2005-05-18 15:18 ` Re: SQL logging (log_min_duration_statement) not working properly Tom Lane <tgl@sss.pgh.pa.us>
  2005-05-18 15:25   ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
@ 2005-05-18 15:41     ` Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 10+ messages in thread

From: Tom Lane @ 2005-05-18 15:41 UTC (permalink / raw)
  To: Pallav Kalva <pkalva@deg.cc>; +Cc: pgsql-admin

Pallav Kalva <pkalva@deg.cc> writes:
>>> I am having problems with logging long running SQL in the log file. 
>>> I have set log_min_duration_statement = 5000 (5secs) and it is not 
>>> logging sql statements that took over 5 secs and this is only from 
>>> queries generated by Hibernate .

Theory B is that Hibernate is a Java program and is using a recent
version of the JDBC driver.  JDBC now uses Parse/Bind/Execute messages
instead of the old-style protocol, and we haven't put support for
log_min_duration_statement into that code path yet ...

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 10+ messages in thread

* Re: SQL logging (log_min_duration_statement) not working
@ 2005-05-18 16:40 Anjan Dave <adave@vantage.com>
  2005-05-18 18:11 ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
  0 siblings, 1 reply; 10+ messages in thread

From: Anjan Dave @ 2005-05-18 16:40 UTC (permalink / raw)
  To: Scott Marlowe <smarlowe@g2switchworks.com>; Pallav Kalva <pkalva@deg.cc>; +Cc: pgsql-admin

For logging the queries taking 5+ seconds, try this:

In postgresql.conf:
syslog = 0    # range 0-2; 0=stdout; 1=both; 2=syslog
syslog_facility = 'LOCAL0'
syslog_ident = 'postgres'

log_min_duration_statement = 5000

Then, in your pg startup script (etc/init.d/Postgresql), you want to
have something like PGLOG variable...and change the startup command so
stuff goes into this file:

PGLOG=/var/log/pglogs
export PGLOG
  $SU -l postgres -c "$PGENGINE/initdb --pgdata=$PGDATA" >> $PGLOG 2>&1
< /dev/null
 $SU -l postgres -c "$PGENGINE/postmaster -p ${PGPORT} -D '${PGDATA}'
${PGOPTS} &" >> $PGLOG 2>&1 < /dev/null

Note, you need to stop|start the database for all this, and do copy your
original init.d/Postgresql and the data/Postgresql.conf files first.


Thanks,
Anjan


-----Original Message-----
From: Scott Marlowe [mailto:smarlowe@g2switchworks.com] 
Sent: Wednesday, May 18, 2005 11:52 AM
To: Pallav Kalva
Cc: pgsql-admin@postgresql.org
Subject: Re: [ADMIN] SQL logging (log_min_duration_statement) not
working

On Wed, 2005-05-18 at 09:49, Pallav Kalva wrote:
> Scott Marlowe wrote:
> 
> >On Wed, 2005-05-18 at 08:35, Pallav Kalva wrote:
> >  
> >
> >>Hi Everybody,
> >>
> >>
> >>    I am having problems with logging long running SQL in the log
file. 
> >>I have set log_min_duration_statement = 5000 (5secs) and it is not 
> >>logging sql statements that took over 5 secs and this is only from 
> >>queries generated by Hibernate . here is a sample query from
hibernate :
> >>
>
>>----------------------------------------------------------------------
-------------------
> >>select listingadd1_.addressid as col_0_0_
> >>from listing.listing listing0_, listing.address listingadd1_
> >>where ((select count(taxrolls2_.fkaddressid)
> >>       from listing.taxroll taxrolls2_
> >>       where listingadd1_.addressid=taxrolls2_.fkaddressid)=0 )
> >>       and(listing0_.fkbestaddressid=listingadd1_.addressid )
> >>       and(listing0_.entrydate>$1 )
> >>order by  listingadd1_.addressid asc
>
>>----------------------------------------------------------------------
----------------------
> >>
> >>  This above query take more than 5 secs to run and it wont get
logged , 
> >>where as all other queries which are not generated by Hibernate are 
> >>logged. Is this expected behavior ?  am I missing something else ?  
> >>Please advice.
> >>    
> >>
> >
> >Are you sure it's taking more than 5 seconds to run, not that it's
> >taking <5 seconds to run and >5 seconds to return the data.  I'm
pretty
> >sure the time it takes to ship out the data isn't counted in the time
> >used to decide whether to log the query or not by the postmaster.
> >
> >I.e. what does explain analyze QUERY HERE say about the execution
time?
> >
> >  
> >
> Hi Scott,
> 
>      Thanks! for the reply. The query for sure runs for more than 5 
> secs, I  passed the same value again and ran the query in the psql
with 
> explain analyze .
>  
>     Here is the explain analyze output :

SNIP

>    ->  Nested Loop  (cost=0.00..17503496.52 rows=678 width=4) (actual 
> time=5.185..31575.629 rows=398459 loops=1)

SNIP

>  Total runtime: 38026.606 ms
> (13 rows)

OK, two problems here, I assume.  One is your logging is dodgy right
now.  Is postgresql logging anything else?  Are you sure your logs are
going where you think they are, etc?

Secondly, it looks from the nested loop part of your explain analyze
that there's a part where the query planner THINKS there's gonna be 678
rows, but actually gets 398459, that's a miscalculation by a factor of
about 600.  Since there's no functions or anything I recognize as being
a common cause of such a mis-estimation, I'd guess that either you need
to analyze, or if you have analyzed, you may need to alter the stats
target for the column entrydate.

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

               http://archives.postgresql.org




^ permalink  raw  reply  [nested|flat] 10+ messages in thread

* Re: SQL logging (log_min_duration_statement) not working
  2005-05-18 16:40 Re: SQL logging (log_min_duration_statement) not working Anjan Dave <adave@vantage.com>
@ 2005-05-18 18:11 ` Pallav Kalva <pkalva@deg.cc>
  0 siblings, 0 replies; 10+ messages in thread

From: Pallav Kalva @ 2005-05-18 18:11 UTC (permalink / raw)
  To: Anjan Dave <adave@vantage.com>; +Cc: Scott Marlowe <smarlowe@g2switchworks.com>; pgsql-admin

Hi Anjan,

    I used to follow this procedure while using Postgres 7.4 version, we 
are now using Postgres 8.0.2 and I dont have give the log file path name 
everytime I start the postgres , I use these paramaters in the 
postgresql.conf

redirect_stderr = true     
log_directory = '/var/lib/pgsql8/pg_log'  
log_filename = 'postgresql-%a.log'


Pallav


Anjan Dave wrote:

>For logging the queries taking 5+ seconds, try this:
>
>In postgresql.conf:
>syslog = 0    # range 0-2; 0=stdout; 1=both; 2=syslog
>syslog_facility = 'LOCAL0'
>syslog_ident = 'postgres'
>
>log_min_duration_statement = 5000
>
>Then, in your pg startup script (etc/init.d/Postgresql), you want to
>have something like PGLOG variable...and change the startup command so
>stuff goes into this file:
>
>PGLOG=/var/log/pglogs
>export PGLOG
>  $SU -l postgres -c "$PGENGINE/initdb --pgdata=$PGDATA" >> $PGLOG 2>&1
>< /dev/null
> $SU -l postgres -c "$PGENGINE/postmaster -p ${PGPORT} -D '${PGDATA}'
>${PGOPTS} &" >> $PGLOG 2>&1 < /dev/null
>
>Note, you need to stop|start the database for all this, and do copy your
>original init.d/Postgresql and the data/Postgresql.conf files first.
>
>
>Thanks,
>Anjan
>
>
>-----Original Message-----
>From: Scott Marlowe [mailto:smarlowe@g2switchworks.com] 
>Sent: Wednesday, May 18, 2005 11:52 AM
>To: Pallav Kalva
>Cc: pgsql-admin@postgresql.org
>Subject: Re: [ADMIN] SQL logging (log_min_duration_statement) not
>working
>
>On Wed, 2005-05-18 at 09:49, Pallav Kalva wrote:
>  
>
>>Scott Marlowe wrote:
>>
>>    
>>
>>>On Wed, 2005-05-18 at 08:35, Pallav Kalva wrote:
>>> 
>>>
>>>      
>>>
>>>>Hi Everybody,
>>>>
>>>>
>>>>   I am having problems with logging long running SQL in the log
>>>>        
>>>>
>file. 
>  
>
>>>>I have set log_min_duration_statement = 5000 (5secs) and it is not 
>>>>logging sql statements that took over 5 secs and this is only from 
>>>>queries generated by Hibernate . here is a sample query from
>>>>        
>>>>
>hibernate :
>  
>
>>>----------------------------------------------------------------------
>>>      
>>>
>-------------------
>  
>
>>>>select listingadd1_.addressid as col_0_0_
>>>>        
>>>>
>>>>from listing.listing listing0_, listing.address listingadd1_
>>>      
>>>
>>>>where ((select count(taxrolls2_.fkaddressid)
>>>>      from listing.taxroll taxrolls2_
>>>>      where listingadd1_.addressid=taxrolls2_.fkaddressid)=0 )
>>>>      and(listing0_.fkbestaddressid=listingadd1_.addressid )
>>>>      and(listing0_.entrydate>$1 )
>>>>order by  listingadd1_.addressid asc
>>>>        
>>>>
>>>----------------------------------------------------------------------
>>>      
>>>
>----------------------
>  
>
>>>> This above query take more than 5 secs to run and it wont get
>>>>        
>>>>
>logged , 
>  
>
>>>>where as all other queries which are not generated by Hibernate are 
>>>>logged. Is this expected behavior ?  am I missing something else ?  
>>>>Please advice.
>>>>   
>>>>
>>>>        
>>>>
>>>Are you sure it's taking more than 5 seconds to run, not that it's
>>>taking <5 seconds to run and >5 seconds to return the data.  I'm
>>>      
>>>
>pretty
>  
>
>>>sure the time it takes to ship out the data isn't counted in the time
>>>used to decide whether to log the query or not by the postmaster.
>>>
>>>I.e. what does explain analyze QUERY HERE say about the execution
>>>      
>>>
>time?
>  
>
>>> 
>>>
>>>      
>>>
>>Hi Scott,
>>
>>     Thanks! for the reply. The query for sure runs for more than 5 
>>secs, I  passed the same value again and ran the query in the psql
>>    
>>
>with 
>  
>
>>explain analyze .
>> 
>>    Here is the explain analyze output :
>>    
>>
>
>SNIP
>
>  
>
>>   ->  Nested Loop  (cost=0.00..17503496.52 rows=678 width=4) (actual 
>>time=5.185..31575.629 rows=398459 loops=1)
>>    
>>
>
>SNIP
>
>  
>
>> Total runtime: 38026.606 ms
>>(13 rows)
>>    
>>
>
>OK, two problems here, I assume.  One is your logging is dodgy right
>now.  Is postgresql logging anything else?  Are you sure your logs are
>going where you think they are, etc?
>
>Secondly, it looks from the nested loop part of your explain analyze
>that there's a part where the query planner THINKS there's gonna be 678
>rows, but actually gets 398459, that's a miscalculation by a factor of
>about 600.  Since there's no functions or anything I recognize as being
>a common cause of such a mis-estimation, I'd guess that either you need
>to analyze, or if you have analyzed, you may need to alter the stats
>target for the column entrydate.
>
>---------------------------(end of broadcast)---------------------------
>TIP 6: Have you searched our list archives?
>
>               http://archives.postgresql.org
>
>
>---------------------------(end of broadcast)---------------------------
>TIP 5: Have you checked our extensive FAQ?
>
>               http://www.postgresql.org/docs/faq
>
>  
>





^ permalink  raw  reply  [nested|flat] 10+ messages in thread


end of thread, other threads:[~2005-05-18 18:11 UTC | newest]

Thread overview: 10+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-05-18 13:35 SQL logging (log_min_duration_statement) not working properly Pallav Kalva <pkalva@deg.cc>
2005-05-18 14:39 ` Re: SQL logging (log_min_duration_statement) not working Scott Marlowe <smarlowe@g2switchworks.com>
2005-05-18 14:49   ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
2005-05-18 15:51     ` Re: SQL logging (log_min_duration_statement) not working Scott Marlowe <smarlowe@g2switchworks.com>
2005-05-18 18:06       ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
2005-05-18 15:18 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-05-18 15:25   ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>
2005-05-18 15:41     ` Tom Lane <tgl@sss.pgh.pa.us>
2005-05-18 16:40 Re: SQL logging (log_min_duration_statement) not working Anjan Dave <adave@vantage.com>
2005-05-18 18:11 ` Re: SQL logging (log_min_duration_statement) not working Pallav Kalva <pkalva@deg.cc>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox