public inbox for [email protected]help / color / mirror / Atom feed
Re: Direct SSL connection with ALPN and HBA rules 30+ messages / 7 participants [nested] [flat]
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-04-25 23:23 Jacob Champion <[email protected]> 0 siblings, 1 reply; 30+ messages in thread From: Jacob Champion @ 2024-04-25 23:23 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Thu, Apr 25, 2024 at 2:50 PM Heikki Linnakangas <[email protected]> wrote: > > I think that comes down to the debate upthread, and whether you think > > it's a performance tweak or a security feature. My take on it is, > > `direct` mode is performance, and `requiredirect` is security. > > Agreed, although the the security benefits from `requiredirect` are > pretty vague. It reduces the attack surface, but there are no known > issues with the 'postgres' or 'direct' negotiation either. I think reduction in attack surface is a concrete security benefit, not a vague one. True, I don't know of any exploits today, but that seems almost tautological -- if there were known exploits in our upgrade handshake, I assume we'd be working to fix them ASAP? > Perhaps 'requiredirect' should be renamed to 'directonly'? If it's agreed that we don't want to require a stronger sslmode for that sslnegotiation setting, then that would probably be an improvement. But who is the target user for `sslnegotiation=directonly`, in your opinion? Would they ever have a reason to use a weak sslmode? > >> I'm not sure about this either. The 'gssencmode' option is already > >> quite weird in that it seems to override the "require"d priority of > >> "sslmode=require", which it IMO really shouldn't. > > Yeah, that combination is weird. I think we should forbid it. But that's > separate from sslnegotiation. Separate but related, IMO. If we were all hypothetically okay with gssencmode ignoring `sslmode=require`, then it's hard for me to claim that `sslnegotiation=requiredirect` should behave differently. On the other hand, if we're not okay with that and we'd like to change it, it's easier for me to argue that `requiredirect` should also be stricter from the get-go. Thanks, --Jacob ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-04-29 08:38 Heikki Linnakangas <[email protected]> parent: Jacob Champion <[email protected]> 0 siblings, 2 replies; 30+ messages in thread From: Heikki Linnakangas @ 2024-04-29 08:38 UTC (permalink / raw) To: Jacob Champion <[email protected]>; +Cc: Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On 26/04/2024 02:23, Jacob Champion wrote: > On Thu, Apr 25, 2024 at 2:50 PM Heikki Linnakangas <[email protected]> wrote: >>> I think that comes down to the debate upthread, and whether you think >>> it's a performance tweak or a security feature. My take on it is, >>> `direct` mode is performance, and `requiredirect` is security. >> >> Agreed, although the the security benefits from `requiredirect` are >> pretty vague. It reduces the attack surface, but there are no known >> issues with the 'postgres' or 'direct' negotiation either. > > I think reduction in attack surface is a concrete security benefit, > not a vague one. True, I don't know of any exploits today, but that > seems almost tautological -- if there were known exploits in our > upgrade handshake, I assume we'd be working to fix them ASAP? Sure, we'd try to fix them ASAP. But we've had the SSLRequest negotiation since time immemorial. If a new vulnerability is found, it's unlikely that we'd need to disable the SSLRequest negotiation altogether to fix it. We'd be in serious trouble with back-branches in that case. There's no sudden need to have a kill-switch for it. Taking that to the extreme, you could argue for a kill-switch for every feature, just in case there's a vulnerability in them. I agree that authentication is more sensitive so reducing the surface of that is more reasonable. And but nevertheless. (This discussion is moot though, because we do have the sslnegotiation=requiredirect mode, so you can disable the SSLRequest negotiation.) >> Perhaps 'requiredirect' should be renamed to 'directonly'? > > If it's agreed that we don't want to require a stronger sslmode for > that sslnegotiation setting, then that would probably be an > improvement. But who is the target user for > `sslnegotiation=directonly`, in your opinion? Would they ever have a > reason to use a weak sslmode? It's unlikely, I agree. A user who's sophisticated enough to use sslnegotiation=directonly would probably also want sslmode=require and require_auth=scram-sha256 and channel_binding=require. Or sslmode=verify-full. But in principle they're orthogonal. If you only have v17 servers in your environment, so you know all servers support direct negotiation if they support SSL at all, but a mix of servers with and without SSL, sslnegotiation=directonly would reduce roundtrips with sslmode=prefer. Making requiredirect to imply sslmode=require, or error out unless you also set sslmode=require, feels like a cavalier way of forcing SSL. We should have a serious discussion on making sslmode=require the default instead. That would be a more direct way of nudging people to use SSL. It would cause a lot of breakage, but it would also be a big improvement to security. Consider how sslnegotiation=requiredirect/directonly would feel, if we made sslmode=require the default. If you explicitly set "sslmode=prefer" or "sslmode=disable", it would be annoying if you would also need to remove "sslnegotiation=requiredirect" from your connection string. I'm leaning towards renaming sslnegotiation=requiredirect to sslnegotiation=directonly at this point. >>>> I'm not sure about this either. The 'gssencmode' option is already >>>> quite weird in that it seems to override the "require"d priority of >>>> "sslmode=require", which it IMO really shouldn't. >> >> Yeah, that combination is weird. I think we should forbid it. But that's >> separate from sslnegotiation. > > Separate but related, IMO. If we were all hypothetically okay with > gssencmode ignoring `sslmode=require`, then it's hard for me to claim > that `sslnegotiation=requiredirect` should behave differently. On the > other hand, if we're not okay with that and we'd like to change it, > it's easier for me to argue that `requiredirect` should also be > stricter from the get-go. I think the best way forward for those is something like a new "require_proto" parameter that you suggested upthread. Perhaps call it "encryption", with options "none", "ssl", "gss" so that you can provide multiple options and libpq will try them in the order specified. For example: encryption=none encryption=ssl, none # like sslmode=prefer encryption=gss encryption=gss, ssl # try GSS first, then SSL encryption=ssl, gss # try SSL first, then GSS This would make gssencmode and sslmode=disable/allow/prefer/require settings obsolete. sslmode would stil be needed to distinguish between verify-ca/verify-full though. But sslnegotiation would be still orthogonal to that. -- Heikki Linnakangas Neon (https://neon.tech) ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-04-29 12:38 Robert Haas <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 0 replies; 30+ messages in thread From: Robert Haas @ 2024-04-29 12:38 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jacob Champion <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, Apr 29, 2024 at 4:38 AM Heikki Linnakangas <[email protected]> wrote: > Making requiredirect to imply sslmode=require, or error out unless you > also set sslmode=require, feels like a cavalier way of forcing SSL. We > should have a serious discussion on making sslmode=require the default > instead. That would be a more direct way of nudging people to use SSL. > It would cause a lot of breakage, but it would also be a big improvement > to security. > > Consider how sslnegotiation=requiredirect/directonly would feel, if we > made sslmode=require the default. If you explicitly set "sslmode=prefer" > or "sslmode=disable", it would be annoying if you would also need to > remove "sslnegotiation=requiredirect" from your connection string. I think making sslmode=require the default is pretty unworkable, unless we also had a way of automatically setting up SSL as part of initdb or something. Otherwise, we'd have to add sslmode=disable to a million places just to get the regression tests to work, and every test cluster anyone spins up locally would break in annoying ways, too. I had been thinking we might want to change the default to sslmode=disable and remove allow and prefer, but maybe automating a basic SSL setup is better. Either way, we should move toward a world where you either ask for SSL and get it, or don't ask for it and don't get it. Being halfway in between is bad. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-04-29 18:43 Jacob Champion <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 1 reply; 30+ messages in thread From: Jacob Champion @ 2024-04-29 18:43 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, Apr 29, 2024 at 1:38 AM Heikki Linnakangas <[email protected]> wrote: > Sure, we'd try to fix them ASAP. But we've had the SSLRequest > negotiation since time immemorial. If a new vulnerability is found, it's > unlikely that we'd need to disable the SSLRequest negotiation altogether > to fix it. We'd be in serious trouble with back-branches in that case. > There's no sudden need to have a kill-switch for it. I'm not really arguing that you'd need the kill switch to fix a problem in the code. (At least, I'm not arguing that in this thread; I reserve the right to argue that in the future. :D) But between the point of time that a vulnerability is announced and a user has upgraded, it's really nice to have a switch as a mitigation. Even better if it's server-side, because then the DBA can protect all their clients without requiring action on their part. > Taking that to the extreme, you could argue for a kill-switch for every > feature, just in case there's a vulnerability in them. I agree that > authentication is more sensitive so reducing the surface of that is more > reasonable. And but nevertheless. I mean... that would be extreme, yeah. I don't think anyone's proposed that. > If you only > have v17 servers in your environment, so you know all servers support > direct negotiation if they support SSL at all, but a mix of servers with > and without SSL, sslnegotiation=directonly would reduce roundtrips with > sslmode=prefer. But if you're in that situation, what does the use of directonly give you over `sslnegotiation=direct`? You already know that servers support direct, so there's no additional performance penalty from the less strict mode. > Making requiredirect to imply sslmode=require, or error out unless you > also set sslmode=require, feels like a cavalier way of forcing SSL. We > should have a serious discussion on making sslmode=require the default > instead. That would be a more direct way of nudging people to use SSL. > It would cause a lot of breakage, but it would also be a big improvement > to security. > > Consider how sslnegotiation=requiredirect/directonly would feel, if we > made sslmode=require the default. If you explicitly set "sslmode=prefer" > or "sslmode=disable", it would be annoying if you would also need to > remove "sslnegotiation=requiredirect" from your connection string. That's similar to how sslrootcert=system already works. To me, it feels great, because I don't have to worry about nonsensical combinations (with the exception of GSS, which we've touched on above). libpq complains loudly if I try to shoot myself in the foot, and if I'm using sslrootcert=system then it's a pretty clear signal that I care more about security than the temporary inconvenience of editing my connection string for one weird server that doesn't use SSL for some reason. > I think the best way forward for those is something like a new > "require_proto" parameter that you suggested upthread. Perhaps call it > "encryption", with options "none", "ssl", "gss" so that you can provide > multiple options and libpq will try them in the order specified. For > example: > > encryption=none > encryption=ssl, none # like sslmode=prefer > encryption=gss > encryption=gss, ssl # try GSS first, then SSL > encryption=ssl, gss # try SSL first, then GSS > > This would make gssencmode and sslmode=disable/allow/prefer/require > settings obsolete. sslmode would stil be needed to distinguish between > verify-ca/verify-full though. But sslnegotiation would be still > orthogonal to that. I will give this some more thought. Thanks, --Jacob ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-04-29 19:06 Heikki Linnakangas <[email protected]> parent: Jacob Champion <[email protected]> 0 siblings, 2 replies; 30+ messages in thread From: Heikki Linnakangas @ 2024-04-29 19:06 UTC (permalink / raw) To: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; +Cc: Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On 29/04/2024 21:43, Jacob Champion wrote: > On Mon, Apr 29, 2024 at 1:38 AM Heikki Linnakangas <[email protected]> wrote: >> If you only >> have v17 servers in your environment, so you know all servers support >> direct negotiation if they support SSL at all, but a mix of servers with >> and without SSL, sslnegotiation=directonly would reduce roundtrips with >> sslmode=prefer. > > But if you're in that situation, what does the use of directonly give > you over `sslnegotiation=direct`? You already know that servers > support direct, so there's no additional performance penalty from the > less strict mode. Well, by that argument we don't need requiredirect/directonly at all. This goes back to whether it's a security feature or a performance feature. There is a small benefit with sslmode=prefer if you connect to a server that doesn't support SSL, though. With sslnegotiation=direct, if the server rejects the direct SSL connection, the client will reconnect and try SSL with SSLRequest. The server will respond with 'N', and the client will proceed without encryption. sslnegotiation=directonly removes that SSLRequest attempt, eliminating one roundtrip. >> Making requiredirect to imply sslmode=require, or error out unless you >> also set sslmode=require, feels like a cavalier way of forcing SSL. We >> should have a serious discussion on making sslmode=require the default >> instead. That would be a more direct way of nudging people to use SSL. >> It would cause a lot of breakage, but it would also be a big improvement >> to security. >> >> Consider how sslnegotiation=requiredirect/directonly would feel, if we >> made sslmode=require the default. If you explicitly set "sslmode=prefer" >> or "sslmode=disable", it would be annoying if you would also need to >> remove "sslnegotiation=requiredirect" from your connection string. > > That's similar to how sslrootcert=system already works. To me, it > feels great, because I don't have to worry about nonsensical > combinations (with the exception of GSS, which we've touched on > above). libpq complains loudly if I try to shoot myself in the foot, > and if I'm using sslrootcert=system then it's a pretty clear signal > that I care more about security than the temporary inconvenience of > editing my connection string for one weird server that doesn't use SSL > for some reason. Oh I was not aware sslrootcert=system works like that. That's a bit surprising, none of the other ssl-related settings imply or require that SSL is actually used. Did we intend to set a precedence for new settings with that? (adding Daniel in case he has an opinion) -- Heikki Linnakangas Neon (https://neon.tech) ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-04-29 19:32 Jacob Champion <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 2 replies; 30+ messages in thread From: Jacob Champion @ 2024-04-29 19:32 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, Apr 29, 2024 at 12:06 PM Heikki Linnakangas <[email protected]> wrote: > On 29/04/2024 21:43, Jacob Champion wrote: > > But if you're in that situation, what does the use of directonly give > > you over `sslnegotiation=direct`? You already know that servers > > support direct, so there's no additional performance penalty from the > > less strict mode. > > Well, by that argument we don't need requiredirect/directonly at all. > This goes back to whether it's a security feature or a performance feature. That's what I've been trying to argue, yeah. If it's not a security feature... why's it there? > There is a small benefit with sslmode=prefer if you connect to a server > that doesn't support SSL, though. With sslnegotiation=direct, if the > server rejects the direct SSL connection, the client will reconnect and > try SSL with SSLRequest. The server will respond with 'N', and the > client will proceed without encryption. sslnegotiation=directonly > removes that SSLRequest attempt, eliminating one roundtrip. Okay, agreed that in this case, there is a performance benefit. It's not enough to convince me, honestly, but are there any other cases I missed as well? > Oh I was not aware sslrootcert=system works like that. That's a bit > surprising, none of the other ssl-related settings imply or require that > SSL is actually used. For sslrootcert=system in particular, the danger of accidentally weak sslmodes is pretty high, especially for verify-ca mode. (It goes back to that other argument -- there should be, effectively, zero users who both opt in to the public CA system, and are also okay with silently falling back and not using it.) > Did we intend to set a precedence for new settings > with that? (I'll let committers answer whether they intended that or not -- I was just bringing up that we already have a setting that works like that, and I really like how it works in practice. But it's probably unsurprising that I like it.) --Jacob ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-04-29 19:34 Jacob Champion <[email protected]> parent: Jacob Champion <[email protected]> 1 sibling, 0 replies; 30+ messages in thread From: Jacob Champion @ 2024-04-29 19:34 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, Apr 29, 2024 at 12:32 PM Jacob Champion <[email protected]> wrote: > > On Mon, Apr 29, 2024 at 12:06 PM Heikki Linnakangas <[email protected]> wrote: > > On 29/04/2024 21:43, Jacob Champion wrote: > > > But if you're in that situation, what does the use of directonly give > > > you over `sslnegotiation=direct`? You already know that servers > > > support direct, so there's no additional performance penalty from the > > > less strict mode. > > > > Well, by that argument we don't need requiredirect/directonly at all. > > This goes back to whether it's a security feature or a performance feature. > > That's what I've been trying to argue, yeah. If it's not a security > feature... why's it there? Er, I should clarify this. I _want_ requiredirect. I just want it to be a security feature. --Jacob ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-04-30 10:10 Daniel Gustafsson <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 0 replies; 30+ messages in thread From: Daniel Gustafsson @ 2024-04-30 10:10 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jacob Champion <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> > On 29 Apr 2024, at 21:06, Heikki Linnakangas <[email protected]> wrote: > Oh I was not aware sslrootcert=system works like that. That's a bit surprising, none of the other ssl-related settings imply or require that SSL is actually used. Did we intend to set a precedence for new settings with that? It was very much intentional, and documented, an sslmode other than verify-full makes little sense when combined with sslrootcert=system. It wasn't intended to set a precedence (though there is probably a fair bit of things we can do, getting this right is hard enough as it is), rather it was footgun prevention. -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-10 13:50 Heikki Linnakangas <[email protected]> parent: Jacob Champion <[email protected]> 1 sibling, 2 replies; 30+ messages in thread From: Heikki Linnakangas @ 2024-05-10 13:50 UTC (permalink / raw) To: Jacob Champion <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On 29/04/2024 22:32, Jacob Champion wrote: > On Mon, Apr 29, 2024 at 12:06 PM Heikki Linnakangas <[email protected]> wrote: >> There is a small benefit with sslmode=prefer if you connect to a server >> that doesn't support SSL, though. With sslnegotiation=direct, if the >> server rejects the direct SSL connection, the client will reconnect and >> try SSL with SSLRequest. The server will respond with 'N', and the >> client will proceed without encryption. sslnegotiation=directonly >> removes that SSLRequest attempt, eliminating one roundtrip. > > Okay, agreed that in this case, there is a performance benefit. It's > not enough to convince me, honestly, but are there any other cases I > missed as well? I realized one case that hasn't been discussed so far: If you use the combination of "sslmode=prefer sslnegotiation=requiredirect" to connect to a pre-v17 server that has SSL enabled but does not support direct SSL connections, you will fall back to a plaintext connection instead. That's almost certainly not what you wanted. I'm coming around to your opinion that we should not allow that combination. Stepping back to summarize my thoughts, there are now three things I don't like about the status quo: 1. As noted above, the sslmode=prefer and sslnegotiation=requiredirect combination is somewhat dangerous, as you might unintentionally fall back to plaintext authentication when connecting to a pre-v17 server. 2. There is an asymmetry between "postgres" and "direct" option names. "postgres" means "try only traditional negotiation", while "direct" means "try direct first, and fall back to traditional negotiation if it fails". That is apparent only if you know that the "requiredirect" mode also exists. 3. The "require" word in "requiredirect" suggests that it's somehow more strict or more secure, similar to sslmode=require. However, I don't consider direct SSL connections to be a security feature. New proposal: - Remove the "try both" mode completely, and rename "requiredirect" to just "direct". So there would be just two modes: "postgres" and "direct". On reflection, the automatic fallback mode doesn't seem very useful. It would make sense as the default, because then you would get the benefits automatically in most cases but still be compatible with old servers. But if it's not the default, you have to fiddle with libpq settings anyway to enable it, and then you might as well use the "requiredirect" mode when you know the server supports it. There isn't anything wrong with it as such, but given how much confusion there's been on how this all works, I'd prefer to cut this back to the bare minimum now. We can add it back in the future, and perhaps make it the default at the same time. This addresses points 2. and 3. above. and: - Only allow sslnegotiation=direct with sslmode=require or higher. This is what you, Jacob, wanted to do all along, and addresses point 1. Thoughts? -- Heikki Linnakangas Neon (https://neon.tech) ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-11 20:45 Jelte Fennema-Nio <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 1 reply; 30+ messages in thread From: Jelte Fennema-Nio @ 2024-05-11 20:45 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Fri, 10 May 2024 at 15:50, Heikki Linnakangas <[email protected]> wrote: > New proposal: > > - Remove the "try both" mode completely, and rename "requiredirect" to > just "direct". So there would be just two modes: "postgres" and > "direct". On reflection, the automatic fallback mode doesn't seem very > useful. It would make sense as the default, because then you would get > the benefits automatically in most cases but still be compatible with > old servers. But if it's not the default, you have to fiddle with libpq > settings anyway to enable it, and then you might as well use the > "requiredirect" mode when you know the server supports it. There isn't > anything wrong with it as such, but given how much confusion there's > been on how this all works, I'd prefer to cut this back to the bare > minimum now. We can add it back in the future, and perhaps make it the > default at the same time. This addresses points 2. and 3. above. > > and: > > - Only allow sslnegotiation=direct with sslmode=require or higher. This > is what you, Jacob, wanted to do all along, and addresses point 1. > > Thoughts? Sounds mostly good to me. But I think we'd want to automatically increase sslmode to require if it is unset, but sslnegotation is set to direct. Similar to how we bump sslmode to verify-full if sslrootcert is set to system, but sslmode is unset. i.e. it seems unnecessary/unwanted to throw an error if the connection string only contains sslnegotiation=direct ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-12 21:39 Heikki Linnakangas <[email protected]> parent: Jelte Fennema-Nio <[email protected]> 0 siblings, 1 reply; 30+ messages in thread From: Heikki Linnakangas @ 2024-05-12 21:39 UTC (permalink / raw) To: Jelte Fennema-Nio <[email protected]>; +Cc: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On 11/05/2024 23:45, Jelte Fennema-Nio wrote: > On Fri, 10 May 2024 at 15:50, Heikki Linnakangas <[email protected]> wrote: >> New proposal: >> >> - Remove the "try both" mode completely, and rename "requiredirect" to >> just "direct". So there would be just two modes: "postgres" and >> "direct". On reflection, the automatic fallback mode doesn't seem very >> useful. It would make sense as the default, because then you would get >> the benefits automatically in most cases but still be compatible with >> old servers. But if it's not the default, you have to fiddle with libpq >> settings anyway to enable it, and then you might as well use the >> "requiredirect" mode when you know the server supports it. There isn't >> anything wrong with it as such, but given how much confusion there's >> been on how this all works, I'd prefer to cut this back to the bare >> minimum now. We can add it back in the future, and perhaps make it the >> default at the same time. This addresses points 2. and 3. above. >> >> and: >> >> - Only allow sslnegotiation=direct with sslmode=require or higher. This >> is what you, Jacob, wanted to do all along, and addresses point 1. >> >> Thoughts? > > Sounds mostly good to me. But I think we'd want to automatically > increase sslmode to require if it is unset, but sslnegotation is set > to direct. Similar to how we bump sslmode to verify-full if > sslrootcert is set to system, but sslmode is unset. i.e. it seems > unnecessary/unwanted to throw an error if the connection string only > contains sslnegotiation=direct I find that error-prone. For example: 1. Try to connect to a server with direct negotiation: psql "host=foobar dbname=mydb sslnegotiation=direct" 2. It fails. Maybe it was an old server? Let's change it to sslnegotiation=postgres. 3. Now it succeeds. Great! You might miss that by changing sslnegotiation to 'postgres', or by removing it altogether, you not only made it compatible with older server versions, but you also allowed falling back to a plaintext connection. Maybe you're fine with that, but maybe not. I'd like to nudge people to use sslmode=require, not rely on implicit stuff like this just to make connection strings a little shorter. I'm not a fan of sslrootcert=system implying sslmode=verify-full either, for the same reasons. But at least "sslrootcert" is a clearly security-related setting, so removing it might give you a pause, whereas sslnegotition is about performance and compatibility. In v18, I'd like to make sslmode=require the default. Or maybe introduce a new setting like "encryption=ssl|gss|none", defaulting to 'ssl'. If we want to encourage encryption, that's the right way to do it. (I'd still recommend everyone to use an explicit sslmode=require in their connection strings for many years, though, because you might be using an older client without realizing it.) -- Heikki Linnakangas Neon (https://neon.tech) ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 09:50 Jelte Fennema-Nio <[email protected]> parent: Heikki Linnakangas <[email protected]> 0 siblings, 1 reply; 30+ messages in thread From: Jelte Fennema-Nio @ 2024-05-13 09:50 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Sun, 12 May 2024 at 23:39, Heikki Linnakangas <[email protected]> wrote: > You might miss that by changing sslnegotiation to 'postgres', or by > removing it altogether, you not only made it compatible with older > server versions, but you also allowed falling back to a plaintext > connection. Maybe you're fine with that, but maybe not. I'd like to > nudge people to use sslmode=require, not rely on implicit stuff like > this just to make connection strings a little shorter. I understand your worry, but I'm not sure that this is actually much of a security issue in practice. sslmode=prefer and sslmode=require are the same amount of insecure imho (i.e. extremely insecure). The only reason sslmode=prefer would connect as non-ssl to a server that supports ssl is if an attacker has write access to the network in the middle (i.e. eavesdropping ability alone is not enough). Once an attacker has this level of network access, it's trivial for this attacker to read any data sent to Postgres by intercepting the TLS handshake and doing TLS termination with some arbitrary cert (any cert is trusted by sslmode=require). So the only actual case where this is a security issue I can think of is when an attacker has only eavesdropping ability on the network. And somehow the Postgres server that the client tries to connect to is configured incorrectly, so that no ssl is set up at all. Then a client would drop to plaintext, when connecting to this server instead of erroring, and the attacker could now read the traffic. But I don't really see this scenario end up any differently when requiring people to enter sslmode=require. The only action a user can take to connect to a server that does not have ssl support is to remove sslmode=require from the connection string. Except if they are also the server operator, in which case they could enable ssl on the server. But if ssl is not set up, then it was probably never set up, and thus providing sslnegotiation=direct would be to test if ssl works. But, if you disagree I'm fine with erroring on plain sslnegotiation=direct > In v18, I'd like to make sslmode=require the default. Or maybe introduce > a new setting like "encryption=ssl|gss|none", defaulting to 'ssl'. If we > want to encourage encryption, that's the right way to do it. (I'd still > recommend everyone to use an explicit sslmode=require in their > connection strings for many years, though, because you might be using an > older client without realizing it.) I'm definitely a huge proponent of making the connection defaults more secure. But as described above sslmode=require is still extremely insecure and I don't think we gain anything substantial by making it the default. I think the only useful secure default would be to use sslmode=verify-full (with probably some automatic fallback to sslmode=prefer when connecting to hardcoded IPs or localhost). Which probably means that sslrootcert=system should also be made the default. Which would mean that ~/.postgresql/root.crt would not be the default anymore, which I personally think is fine but others likely disagree. ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 11:07 Heikki Linnakangas <[email protected]> parent: Jelte Fennema-Nio <[email protected]> 0 siblings, 3 replies; 30+ messages in thread From: Heikki Linnakangas @ 2024-05-13 11:07 UTC (permalink / raw) To: Jelte Fennema-Nio <[email protected]>; +Cc: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On 13/05/2024 12:50, Jelte Fennema-Nio wrote: > On Sun, 12 May 2024 at 23:39, Heikki Linnakangas <[email protected]> wrote: >> In v18, I'd like to make sslmode=require the default. Or maybe introduce >> a new setting like "encryption=ssl|gss|none", defaulting to 'ssl'. If we >> want to encourage encryption, that's the right way to do it. (I'd still >> recommend everyone to use an explicit sslmode=require in their >> connection strings for many years, though, because you might be using an >> older client without realizing it.) > > I'm definitely a huge proponent of making the connection defaults more > secure. But as described above sslmode=require is still extremely > insecure and I don't think we gain anything substantial by making it > the default. I think the only useful secure default would be to use > sslmode=verify-full (with probably some automatic fallback to > sslmode=prefer when connecting to hardcoded IPs or localhost). Which > probably means that sslrootcert=system should also be made the > default. Which would mean that ~/.postgresql/root.crt would not be the > default anymore, which I personally think is fine but others likely > disagree. "channel_binding=require sslmode=require" also protects from MITM attacks. I think these options should be designed from the user's point of view, so that the user can specify the risks they're willing to accept, and the details of how that's accomplished are handled by libpq. For example, I'm OK with (tick all that apply): [ ] passive eavesdroppers seeing all the traffic [ ] MITM being able to hijack the session [ ] connecting without verifying the server's identity [ ] divulging the plaintext password to the server [ ] ... The requirements for whether SSL or GSS encryption is required, whether the server's certificate needs to signed with known CA, etc. can be derived from those. For example, if you need protection from eavesdroppers, SSL or GSS encryption must be used. If you need to verify the server's identity, it implies sslmode=verify-CA or channel_binding=true. If you don't want to divulge the password, it implies a suitable require_auth setting. I don't have a concrete proposal yet, but something like that. And the defaults for those are up for debate. psql could perhaps help by listing the above properties at the beginning of the session, something like: psql (16.2) WARNING: Connection is not encrypted. WARNING: The server's identity has not been verified Type "help" for help. postgres=# Although for the "divulge plaintext password to server" property, it's too late to print a warning after connecting, because the damage has already been done. A different line of thought is that to keep the attack surface as smal as possible, you should specify very explicitly what exactly you expect to happen in the authentication, and disallow any variance. For example, you expect SCRAM to be used, with a certificate signed by a particular CA, and if the server requests anything else, that's suspicious and the connection is aborted. We should make that possible too, but the above flexible risk-based approach seems good for the defaults. -- Heikki Linnakangas Neon (https://neon.tech) ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 13:37 Heikki Linnakangas <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 2 replies; 30+ messages in thread From: Heikki Linnakangas @ 2024-05-13 13:37 UTC (permalink / raw) To: Jacob Champion <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On 10/05/2024 16:50, Heikki Linnakangas wrote: > New proposal: > > - Remove the "try both" mode completely, and rename "requiredirect" to > just "direct". So there would be just two modes: "postgres" and > "direct". On reflection, the automatic fallback mode doesn't seem very > useful. It would make sense as the default, because then you would get > the benefits automatically in most cases but still be compatible with > old servers. But if it's not the default, you have to fiddle with libpq > settings anyway to enable it, and then you might as well use the > "requiredirect" mode when you know the server supports it. There isn't > anything wrong with it as such, but given how much confusion there's > been on how this all works, I'd prefer to cut this back to the bare > minimum now. We can add it back in the future, and perhaps make it the > default at the same time. This addresses points 2. and 3. above. > > and: > > - Only allow sslnegotiation=direct with sslmode=require or higher. This > is what you, Jacob, wanted to do all along, and addresses point 1. Here's a patch to implement that. -- Heikki Linnakangas Neon (https://neon.tech) Attachments: [text/x-patch] 0001-Remove-option-to-fall-back-from-direct-to-postgres-S.patch (41.4K, ../../[email protected]/2-0001-Remove-option-to-fall-back-from-direct-to-postgres-S.patch) download | inline diff: From 2a7bde8502966a634b62d4109f0cde5fdc685da2 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas <[email protected]> Date: Mon, 13 May 2024 16:36:54 +0300 Subject: [PATCH 1/1] Remove option to fall back from direct to postgres SSL negotiation There were three problems with the sslnegotiation options: 1. The sslmode=prefer and sslnegotiation=requiredirect combination was somewhat dangerous, as you might unintentionally fall back to plaintext authentication when connecting to a pre-v17 server. 2. There was an asymmetry between 'postgres' and 'direct' options. 'postgres' meant "try only traditional negotiation", while 'direct' meant "try direct first, and fall back to traditional negotiation if it fails". That was apparent only if you knew that the 'requiredirect' mode also exists. 3. The "require" word in 'requiredirect' suggests that it's somehow more strict or more secure, similar to sslmode. However, I don't consider direct SSL connections to be a security feature. To address these problems: - Only allow sslnegotiation='direct' if sslmode='require' or stronger. And for the record, Jacob and Robert felt that we should do that (or have sslnegotiation='direct' imply sslmode='require') anyway, regardless of the first issue. - Remove the 'direct' mode that falls back to traditional negotiation, and rename what was called 'requiredirect' to 'direct' instead. In other words, there is no "try both methods" option anymore, 'postgres' now means the traditional negotiation and 'direct' means a direct SSL connection. Discussion: https://www.postgresql.org/message-id/d3b1608a-a1b6-4eda-9ec5-ddb3e4375808%40iki.fi --- doc/src/sgml/libpq.sgml | 49 ++-- src/interfaces/libpq/fe-connect.c | 52 ++-- src/interfaces/libpq/libpq-int.h | 3 +- .../libpq/t/005_negotiate_encryption.pl | 254 ++++++++---------- 4 files changed, 150 insertions(+), 208 deletions(-) diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 1d32c226d8..b32e497b1b 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -1772,15 +1772,18 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname <term><literal>sslnegotiation</literal></term> <listitem> <para> - This option controls whether <productname>PostgreSQL</productname> - will perform its protocol negotiation to request encryption from the - server or will just directly make a standard <acronym>SSL</acronym> - connection. Traditional <productname>PostgreSQL</productname> - protocol negotiation is the default and the most flexible with - different server configurations. If the server is known to support - direct <acronym>SSL</acronym> connections then the latter requires one - fewer round trip reducing connection latency and also allows the use - of protocol agnostic SSL network tools. + This option controls how SSL encryption is negotiated with the server, + if SSL is used. In the default <literal>postgres</literal> mode, the + client first asks the server if SSL is supported. In + <literal>direct</literal> mode, the client starts the standard SSL + handshake directly after establishing the TCP/IP connection. Traditional + <productname>PostgreSQL</productname> protocol negotiation is the most + flexible with different server configurations. If the server is known + to support direct <acronym>SSL</acronym> connections then the latter + requires one fewer round trip reducing connection latency and also + allows the use of protocol agnostic SSL network tools. The direct SSL + option was introduced in <productname>PostgreSQL</productname> version + 17. </para> <variablelist> @@ -1798,32 +1801,14 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname <term><literal>direct</literal></term> <listitem> <para> - first attempt to establish a standard SSL connection and if that - fails reconnect and perform the negotiation. This fallback - process adds significant latency if the initial SSL connection - fails. - </para> - <para> - An exception is if <literal>gssencmode</literal> is set - to <literal>prefer</literal>, but the server rejects GSS encryption. - In that case, SSL is negotiated over the same TCP connection using - <productname>PostgreSQL</productname> protocol negotiation. In - other words, the direct SSL handshake is not used, if a TCP - connection has already been established and can be used for the + start SSL handshake directly after establishing the TCP/IP + connection. This is only allowed with sslmode=require or higher, + because the weaker settings could lead to unintended fallback to + plaintext authentication when the server does not support direct SSL handshake. </para> </listitem> </varlistentry> - - <varlistentry> - <term><literal>requiredirect</literal></term> - <listitem> - <para> - attempt to establish a standard SSL connection and if that fails - return a connection failure immediately. - </para> - </listitem> - </varlistentry> </variablelist> </listitem> </varlistentry> @@ -2064,7 +2049,7 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname connections without having to decrypt the SSL stream. (Note that unless the proxy is aware of the PostgreSQL protocol handshake this would require setting <literal>sslnegotiation</literal> - to <literal>direct</literal> or <literal>requiredirect</literal>.) + to <literal>direct</literal>.) However, <acronym>SNI</acronym> makes the destination host name appear in cleartext in the network traffic, so it might be undesirable in some cases. diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index 81d278c395..4a5a71de4b 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -274,7 +274,7 @@ static const internalPQconninfoOption PQconninfoOptions[] = { offsetof(struct pg_conn, sslmode)}, {"sslnegotiation", "PGSSLNEGOTIATION", DefaultSSLNegotiation, NULL, - "SSL-Negotiation", "", 14, /* sizeof("requiredirect") == 14 */ + "SSL-Negotiation", "", 9, /* sizeof("postgres") == 9 */ offsetof(struct pg_conn, sslnegotiation)}, {"sslcompression", "PGSSLCOMPRESSION", "0", NULL, @@ -1590,8 +1590,7 @@ pqConnectOptions2(PGconn *conn) if (conn->sslnegotiation) { if (strcmp(conn->sslnegotiation, "postgres") != 0 - && strcmp(conn->sslnegotiation, "direct") != 0 - && strcmp(conn->sslnegotiation, "requiredirect") != 0) + && strcmp(conn->sslnegotiation, "direct") != 0) { conn->status = CONNECTION_BAD; libpq_append_conn_error(conn, "invalid %s value: \"%s\"", @@ -1608,6 +1607,25 @@ pqConnectOptions2(PGconn *conn) return false; } #endif + + /* + * Don't allow direct SSL negotiation with sslmode='prefer', because + * that poses a risk of unintentional fallback to plaintext connection + * when connecting to a pre-v17 server that does not support direct + * SSL connections. To keep things simple, don't allow it with + * sslmode='allow' or sslmode='disable' either. If a user goes through + * the trouble of setting sslnegotiation='direct', they probably + * intend to use SSL, and sslmode=disable or allow is probably a user + * user mistake anyway. + */ + if (conn->sslnegotiation[0] == 'd' && + conn->sslmode[0] != 'r' && conn->sslmode[0] != 'v') + { + conn->status = CONNECTION_BAD; + libpq_append_conn_error(conn, "weak sslmode \"%s\" may not be used with sslnegotiation=direct (use \"require\", \"verify-ca\", or \"verify-full\")", + conn->sslmode); + return false; + } } else { @@ -4312,8 +4330,6 @@ init_allowed_encryption_methods(PGconn *conn) if (conn->sslnegotiation[0] == 'p') conn->allowed_enc_methods |= ENC_NEGOTIATED_SSL; else if (conn->sslnegotiation[0] == 'd') - conn->allowed_enc_methods |= ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL; - else if (conn->sslnegotiation[0] == 'r') conn->allowed_enc_methods |= ENC_DIRECT_SSL; } #endif @@ -4376,18 +4392,6 @@ connection_failed(PGconn *conn) Assert((conn->failed_enc_methods & conn->current_enc_method) == 0); conn->failed_enc_methods |= conn->current_enc_method; - /* - * If the server reported an error after the SSL handshake, no point in - * retrying with negotiated vs direct SSL. - */ - if ((conn->current_enc_method & (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL)) != 0 && - conn->ssl_handshake_started) - { - conn->failed_enc_methods |= (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL) & conn->allowed_enc_methods; - } - else - conn->failed_enc_methods |= conn->current_enc_method; - return select_next_encryption_method(conn, false); } @@ -4449,20 +4453,8 @@ select_next_encryption_method(PGconn *conn, bool have_valid_connection) if (conn->sslmode[0] == 'a') SELECT_NEXT_METHOD(ENC_PLAINTEXT); - /* - * If enabled, try direct SSL. Unless we have a valid TCP connection that - * failed negotiating GSSAPI encryption; in that case we prefer to reuse - * the connection with negotiated SSL, instead of reconnecting to do - * direct SSL. The point of sslnegotiation=direct is to avoid the - * roundtrip from the negotiation, but reconnecting would also incur a - * roundtrip. (In sslnegotiation=requiredirect mode, negotiated SSL is not - * in the list of allowed methods and we will reconnect.) - */ - if (have_valid_connection) - SELECT_NEXT_METHOD(ENC_NEGOTIATED_SSL); - - SELECT_NEXT_METHOD(ENC_DIRECT_SSL); SELECT_NEXT_METHOD(ENC_NEGOTIATED_SSL); + SELECT_NEXT_METHOD(ENC_DIRECT_SSL); if (conn->sslmode[0] != 'a') SELECT_NEXT_METHOD(ENC_PLAINTEXT); diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h index 3691e5ee96..10d2709869 100644 --- a/src/interfaces/libpq/libpq-int.h +++ b/src/interfaces/libpq/libpq-int.h @@ -395,8 +395,7 @@ struct pg_conn char *keepalives_count; /* maximum number of TCP keepalive * retransmits */ char *sslmode; /* SSL mode (require,prefer,allow,disable) */ - char *sslnegotiation; /* SSL initiation style - * (postgres,direct,requiredirect) */ + char *sslnegotiation; /* SSL initiation style (postgres,direct) */ char *sslcompression; /* SSL compression (0 or 1) */ char *sslkey; /* client key filename */ char *sslcert; /* client certificate filename */ diff --git a/src/interfaces/libpq/t/005_negotiate_encryption.pl b/src/interfaces/libpq/t/005_negotiate_encryption.pl index 03a9079f3f..36ba8aede8 100644 --- a/src/interfaces/libpq/t/005_negotiate_encryption.pl +++ b/src/interfaces/libpq/t/005_negotiate_encryption.pl @@ -209,7 +209,7 @@ $node->reload; my @all_test_users = ('testuser', 'ssluser', 'nossluser', 'gssuser', 'nogssuser'); my @all_gssencmodes = ('disable', 'prefer', 'require'); my @all_sslmodes = ('disable', 'allow', 'prefer', 'require'); -my @all_sslnegotiations = ('postgres', 'direct', 'requiredirect'); +my @all_sslnegotiations = ('postgres', 'direct'); my $server_config = { server_ssl => 0, @@ -223,23 +223,22 @@ my $test_table; if ($ssl_supported) { $test_table = q{ # USER GSSENCMODE SSLMODE SSLNEGOTIATION EVENTS -> OUTCOME -testuser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain +testuser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslreject, authok -> plain -. . . direct connect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, directsslreject, reconnect, authok -> plain . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail -. prefer disable * connect, authok -> plain -. . allow * connect, authok -> plain +. . . direct connect, directsslreject -> fail +. prefer disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslreject, authok -> plain -. . . direct connect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, directsslreject, reconnect, authok -> plain . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail - }; +. . . direct connect, directsslreject -> fail + +# sslnegotiation=direct is not acccepted unless sslmode=require or stronger +* * disable direct - -> fail +* * allow direct - -> fail +* * prefer direct - -> fail +}; } else { # Compiled without SSL support $test_table = q{ @@ -251,11 +250,10 @@ testuser disable disable postgres connect, authok . . allow postgres connect, authok -> plain . . prefer postgres connect, authok -> plain -# Without SSL support, sslmode=require and sslnegotiation=direct/requiredirect -# are not accepted at all. -. * require * - -> fail -. * * direct - -> fail -. * * requiredirect - -> fail +# Without SSL support, sslmode=require and sslnegotiation=direct are +# not accepted at all +* * require * - -> fail +* * * direct - -> fail }; } @@ -281,34 +279,26 @@ SKIP: $test_table = q{ # USER GSSENCMODE SSLMODE SSLNEGOTIATION EVENTS -> OUTCOME -testuser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain +testuser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl . . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -ssluser . disable * connect, authfail -> fail +ssluser . disable postgres connect, authfail -> fail . . allow postgres connect, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, authfail, reconnect, directsslaccept, authok -> ssl . . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl . . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -nossluser . disable * connect, authok -> plain +nossluser . disable postgres connect, authok -> plain . . allow postgres connect, authok -> plain -. . . direct connect, authok -> plain -. . . requiredirect connect, authok -> plain . . prefer postgres connect, sslaccept, authfail, reconnect, authok -> plain -. . . direct connect, directsslaccept, authfail, reconnect, authok -> plain -. . . requiredirect connect, directsslaccept, authfail, reconnect, authok -> plain . . require postgres connect, sslaccept, authfail -> fail . . require direct connect, directsslaccept, authfail -> fail -. . require requiredirect connect, directsslaccept, authfail -> fail + +# sslnegotiation=direct is not acccepted unless sslmode=require or stronger +* * disable direct - -> fail +* * allow direct - -> fail +* * prefer direct - -> fail }; # Enable SSL in the server @@ -342,62 +332,54 @@ SKIP: $test_table = q{ # USER GSSENCMODE SSLMODE SSLNEGOTIATION EVENTS -> OUTCOME -testuser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain +testuser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslreject, authok -> plain -. . . direct connect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, directsslreject, reconnect, authok -> plain . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail -. prefer * * connect, gssaccept, authok -> gss -. require * * connect, gssaccept, authok -> gss +. . . direct connect, directsslreject -> fail +. prefer * postgres connect, gssaccept, authok -> gss +. prefer require direct connect, gssaccept, authok -> gss +. require * postgres connect, gssaccept, authok -> gss +. . require direct connect, gssaccept, authok -> gss -gssuser disable disable * connect, authfail -> fail +gssuser disable disable postgres connect, authfail -> fail . . allow postgres connect, authfail, reconnect, sslreject -> fail -. . . direct connect, authfail, reconnect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, authfail, reconnect, directsslreject -> fail . . prefer postgres connect, sslreject, authfail -> fail -. . . direct connect, directsslreject, reconnect, sslreject, authfail -> fail -. . . requiredirect connect, directsslreject, reconnect, authfail -> fail . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail -. prefer * * connect, gssaccept, authok -> gss -. require * * connect, gssaccept, authok -> gss +. . . direct connect, directsslreject -> fail +. prefer * postgres connect, gssaccept, authok -> gss +. prefer require direct connect, gssaccept, authok -> gss +. require * postgres connect, gssaccept, authok -> gss +. . require direct connect, gssaccept, authok -> gss -nogssuser disable disable * connect, authok -> plain +nogssuser disable disable postgres connect, authok -> plain . . allow postgres connect, authok -> plain -. . . direct connect, authok -> plain -. . . requiredirect connect, authok -> plain . . prefer postgres connect, sslreject, authok -> plain -. . . direct connect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, directsslreject, reconnect, authok -> plain . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail -. prefer disable * connect, gssaccept, authfail, reconnect, authok -> plain +. . . direct connect, directsslreject -> fail +. prefer disable postgres connect, gssaccept, authfail, reconnect, authok -> plain . . allow postgres connect, gssaccept, authfail, reconnect, authok -> plain -. . . direct connect, gssaccept, authfail, reconnect, authok -> plain -. . . requiredirect connect, gssaccept, authfail, reconnect, authok -> plain . . prefer postgres connect, gssaccept, authfail, reconnect, sslreject, authok -> plain -. . . direct connect, gssaccept, authfail, reconnect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslreject, reconnect, authok -> plain . . require postgres connect, gssaccept, authfail, reconnect, sslreject -> fail -. . . direct connect, gssaccept, authfail, reconnect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslreject -> fail -. require disable * connect, gssaccept, authfail -> fail -. . allow * connect, gssaccept, authfail -> fail -. . prefer * connect, gssaccept, authfail -> fail -. . require * connect, gssaccept, authfail -> fail # If both GSSAPI and sslmode are required, and GSS is not available -> fail +. . . direct connect, gssaccept, authfail, reconnect, directsslreject -> fail +. require disable postgres connect, gssaccept, authfail -> fail +. . allow postgres connect, gssaccept, authfail -> fail +. . prefer postgres connect, gssaccept, authfail -> fail +. . require postgres connect, gssaccept, authfail -> fail # If both GSSAPI and sslmode are required, and GSS is not available -> fail +. . . direct connect, gssaccept, authfail -> fail # If both GSSAPI and sslmode are required, and GSS is not available -> fail + +# sslnegotiation=direct is not acccepted unless sslmode=require or stronger +* * disable direct - -> fail +* * allow direct - -> fail +* * prefer direct - -> fail }; # The expected events and outcomes above assume that SSL support # is enabled. When libpq is compiled without SSL support, all # attempts to connect with sslmode=require or - # sslnegotiation=direct/requiredirect would fail immediately without - # even connecting to the server. Skip those, because we tested - # them earlier already. + # sslnegotiation=direct would fail immediately without even + # connecting to the server. Skip those, because we tested them + # earlier already. my ($sslmodes, $sslnegotiations); if ($ssl_supported != 0) { ($sslmodes, $sslnegotiations) = (\@all_sslmodes, \@all_sslnegotiations); @@ -431,100 +413,84 @@ SKIP: $test_table = q{ # USER GSSENCMODE SSLMODE SSLNEGOTIATION EVENTS -> OUTCOME -testuser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain -. . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -. . require postgres connect, sslaccept, authok -> ssl +testuser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain +. . prefer postgres connect, sslaccept, authok -> ssl +. . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -. prefer disable * connect, gssaccept, authok -> gss -. . allow * connect, gssaccept, authok -> gss -. . prefer * connect, gssaccept, authok -> gss -. . require * connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require -. require disable * connect, gssaccept, authok -> gss -. . allow * connect, gssaccept, authok -> gss -. . prefer * connect, gssaccept, authok -> gss -. . require * connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require - -gssuser disable disable * connect, authfail -> fail +. prefer disable postgres connect, gssaccept, authok -> gss +. . allow postgres connect, gssaccept, authok -> gss +. . prefer postgres connect, gssaccept, authok -> gss +. . require postgres connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require +. . . direct connect, gssaccept, authok -> gss +. require disable postgres connect, gssaccept, authok -> gss +. . allow postgres connect, gssaccept, authok -> gss +. . prefer postgres connect, gssaccept, authok -> gss +. . require postgres connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require +. . . direct connect, gssaccept, authok -> gss + +gssuser disable disable postgres connect, authfail -> fail . . allow postgres connect, authfail, reconnect, sslaccept, authfail -> fail -. . . direct connect, authfail, reconnect, directsslaccept, authfail -> fail -. . . requiredirect connect, authfail, reconnect, directsslaccept, authfail -> fail . . prefer postgres connect, sslaccept, authfail, reconnect, authfail -> fail -. . . direct connect, directsslaccept, authfail, reconnect, authfail -> fail -. . . requiredirect connect, directsslaccept, authfail, reconnect, authfail -> fail . . require postgres connect, sslaccept, authfail -> fail . . . direct connect, directsslaccept, authfail -> fail -. . . requiredirect connect, directsslaccept, authfail -> fail -. prefer disable * connect, gssaccept, authok -> gss -. . allow * connect, gssaccept, authok -> gss -. . prefer * connect, gssaccept, authok -> gss -. . require * connect, gssaccept, authok -> gss # GSS is chosen over SSL, even though sslmode=require -. require disable * connect, gssaccept, authok -> gss -. . allow * connect, gssaccept, authok -> gss -. . prefer * connect, gssaccept, authok -> gss -. . require * connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require - -ssluser disable disable * connect, authfail -> fail +. prefer disable postgres connect, gssaccept, authok -> gss +. . allow postgres connect, gssaccept, authok -> gss +. . prefer postgres connect, gssaccept, authok -> gss +. . require postgres connect, gssaccept, authok -> gss # GSS is chosen over SSL, even though sslmode=require +. . . direct connect, gssaccept, authok -> gss +. require disable postgres connect, gssaccept, authok -> gss +. . allow postgres connect, gssaccept, authok -> gss +. . prefer postgres connect, gssaccept, authok -> gss +. . require postgres connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require +. . . direct connect, gssaccept, authok -> gss + +ssluser disable disable postgres connect, authfail -> fail . . allow postgres connect, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, authfail, reconnect, directsslaccept, authok -> ssl . . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl . . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -. prefer disable * connect, gssaccept, authfail, reconnect, authfail -> fail +. prefer disable postgres connect, gssaccept, authfail, reconnect, authfail -> fail . . allow postgres connect, gssaccept, authfail, reconnect, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, gssaccept, authfail, reconnect, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, authfail, reconnect, directsslaccept, authok -> ssl . . prefer postgres connect, gssaccept, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl . . require postgres connect, gssaccept, authfail, reconnect, sslaccept, authok -> ssl . . . direct connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. require disable * connect, gssaccept, authfail -> fail -. . allow * connect, gssaccept, authfail -> fail -. . prefer * connect, gssaccept, authfail -> fail -. . require * connect, gssaccept, authfail -> fail # If both GSS and SSL are required, the sslmode=require is effectively ignored and GSS is required +. require disable postgres connect, gssaccept, authfail -> fail +. . allow postgres connect, gssaccept, authfail -> fail +. . prefer postgres connect, gssaccept, authfail -> fail +. . require postgres connect, gssaccept, authfail -> fail # If both GSS and SSL are required, the sslmode=require is effectively ignored and GSS is required +. . . direct connect, gssaccept, authfail -> fail -nogssuser disable disable * connect, authok -> plain +nogssuser disable disable postgres connect, authok -> plain . . allow postgres connect, authok -> plain -. . . direct connect, authok -> plain -. . . requiredirect connect, authok -> plain . . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl . . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -. prefer disable * connect, gssaccept, authfail, reconnect, authok -> plain -. . allow * connect, gssaccept, authfail, reconnect, authok -> plain +. prefer disable postgres connect, gssaccept, authfail, reconnect, authok -> plain +. . allow postgres connect, gssaccept, authfail, reconnect, authok -> plain . . prefer postgres connect, gssaccept, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl . . require postgres connect, gssaccept, authfail, reconnect, sslaccept, authok -> ssl . . . direct connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. require disable * connect, gssaccept, authfail -> fail -. . allow * connect, gssaccept, authfail -> fail -. . prefer * connect, gssaccept, authfail -> fail -. . require * connect, gssaccept, authfail -> fail # If both GSS and SSL are required, the sslmode=require is effectively ignored and GSS is required - -nossluser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain +. require disable postgres connect, gssaccept, authfail -> fail +. . allow postgres connect, gssaccept, authfail -> fail +. . prefer postgres connect, gssaccept, authfail -> fail +. . require postgres connect, gssaccept, authfail -> fail # If both GSS and SSL are required, the sslmode=require is effectively ignored and GSS is required +. . . direct connect, gssaccept, authfail -> fail + +nossluser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslaccept, authfail, reconnect, authok -> plain -. . . direct connect, directsslaccept, authfail, reconnect, authok -> plain -. . . requiredirect connect, directsslaccept, authfail, reconnect, authok -> plain . . require postgres connect, sslaccept, authfail -> fail . . . direct connect, directsslaccept, authfail -> fail -. . . requiredirect connect, directsslaccept, authfail -> fail -. prefer * * connect, gssaccept, authok -> gss -. require * * connect, gssaccept, authok -> gss +. prefer * postgres connect, gssaccept, authok -> gss +. . require direct connect, gssaccept, authok -> gss +. require * postgres connect, gssaccept, authok -> gss +. . require direct connect, gssaccept, authok -> gss + +# sslnegotiation=direct is not acccepted unless sslmode=require or stronger +* * disable direct - -> fail +* * allow direct - -> fail +* * prefer direct - -> fail }; note("Running tests with both GSS and SSL enabled in server"); -- 2.39.2 ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 13:55 Jelte Fennema-Nio <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 1 reply; 30+ messages in thread From: Jelte Fennema-Nio @ 2024-05-13 13:55 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, 13 May 2024 at 15:38, Heikki Linnakangas <[email protected]> wrote: > Here's a patch to implement that. + if (conn->sslnegotiation[0] == 'd' && + conn->sslmode[0] != 'r' && conn->sslmode[0] != 'v') I think these checks should use strcmp instead of checking magic first characters. I see this same clever trick is used in the recently added init_allowed_encryption_methods, and I think that should be changed to use strcmp too for readability. ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 14:19 Jelte Fennema-Nio <[email protected]> parent: Heikki Linnakangas <[email protected]> 2 siblings, 0 replies; 30+ messages in thread From: Jelte Fennema-Nio @ 2024-05-13 14:19 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, 13 May 2024 at 13:07, Heikki Linnakangas <[email protected]> wrote: > "channel_binding=require sslmode=require" also protects from MITM attacks. Cool, I didn't realize we had this connection option and it could be used like this. But I think there's a few security downsides of channel_binding=require over sslmode=verify-full: If the client relies on channel_binding to validate server authenticity, a leaked server-side SCRAM hash is enough for an attacker to impersonate a server. While normally a leaked scram hash isn't really much of a security concern (assuming long enough passwords). I also don't know of many people rotating their scram hashes, even though many rotate TLS certs. > I think these options should be designed from the user's point of view, > so that the user can specify the risks they're willing to accept, and > the details of how that's accomplished are handled by libpq. For > example, I'm OK with (tick all that apply): > > [ ] passive eavesdroppers seeing all the traffic > [ ] MITM being able to hijack the session > [ ] connecting without verifying the server's identity > [ ] divulging the plaintext password to the server > [ ] ... I think that sounds like a great idea, looking forward to the proposal. ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 14:54 Heikki Linnakangas <[email protected]> parent: Jelte Fennema-Nio <[email protected]> 0 siblings, 0 replies; 30+ messages in thread From: Heikki Linnakangas @ 2024-05-13 14:54 UTC (permalink / raw) To: Jelte Fennema-Nio <[email protected]>; +Cc: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On 13/05/2024 16:55, Jelte Fennema-Nio wrote: > On Mon, 13 May 2024 at 15:38, Heikki Linnakangas <[email protected]> wrote: >> Here's a patch to implement that. > > + if (conn->sslnegotiation[0] == 'd' && > + conn->sslmode[0] != 'r' && conn->sslmode[0] != 'v') > > I think these checks should use strcmp instead of checking magic first > characters. I see this same clever trick is used in the recently added > init_allowed_encryption_methods, and I think that should be changed to > use strcmp too for readability. Oh yeah, I hate that too. These should be refactored into enums, with a clear separate stage of parsing the options from strings. But we use that pattern all over the place, so I didn't want to start reforming it with this patch. -- Heikki Linnakangas Neon (https://neon.tech) ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 16:13 Robert Haas <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 3 replies; 30+ messages in thread From: Robert Haas @ 2024-05-13 16:13 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, May 13, 2024 at 9:37 AM Heikki Linnakangas <[email protected]> wrote: > On 10/05/2024 16:50, Heikki Linnakangas wrote: > > New proposal: > > > > - Remove the "try both" mode completely, and rename "requiredirect" to > > just "direct". So there would be just two modes: "postgres" and > > "direct". On reflection, the automatic fallback mode doesn't seem very > > useful. It would make sense as the default, because then you would get > > the benefits automatically in most cases but still be compatible with > > old servers. But if it's not the default, you have to fiddle with libpq > > settings anyway to enable it, and then you might as well use the > > "requiredirect" mode when you know the server supports it. There isn't > > anything wrong with it as such, but given how much confusion there's > > been on how this all works, I'd prefer to cut this back to the bare > > minimum now. We can add it back in the future, and perhaps make it the > > default at the same time. This addresses points 2. and 3. above. > > > > and: > > > > - Only allow sslnegotiation=direct with sslmode=require or higher. This > > is what you, Jacob, wanted to do all along, and addresses point 1. > > Here's a patch to implement that. I find this idea to be a massive improvement over the status quo, and I didn't spot any major problems when I read through the patch, either. I'm not quite sure if the patch takes the right approach in emphasizing that weaker sslmode settings are not allowed because of unintended fallbacks. It seems to me that we could equally well say that those combinations are nonsensical. If we're making a direct SSL connection, SSL is eo ipso required. I don't have a strong opinion about whether sslnegotiation=direct should error out (as you propose here) or silently promote sslmode to require. I think either is defensible. Had I been implementing it, I think I would have done as Jacob proposes, just because once we've forced a direct SSL negotiation surely the only sensible behavior is to be using SSL, unless you think there should be a silently-reconnect-without-SSL behavior, which I sure don't. However, I disagree with Jacob's assertion that sslmode=require has no security benefits over sslmode=prefer. That seems like the kind of pessimism that makes people hate security professionals. There have got to be some attacks that are foreclosed by encrypting the connection, even if you don't stop MITM attacks or other things that are more sophisticated than running wireshark and seeing what goes by on the wire. I'm pleased to hear that you will propose to make sslmode=require the default in v18. I think we'll need to do some work to figure out how much collateral damage that will cause, and maybe it will be more than we can stomach, but Magnus has been saying for years that the current default is terrible. I'm not sure I was entirely convinced of that the first time I heard him say it, but I'm smarter now than I was then. It's really hard to believe in 2024 that anyone should ever be using a setting that may or may not encrypt the connection. There's http and https but there's no httpmaybes. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 16:45 Jacob Champion <[email protected]> parent: Robert Haas <[email protected]> 2 siblings, 1 reply; 30+ messages in thread From: Jacob Champion @ 2024-05-13 16:45 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Daniel Gustafsson <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> (There's, uh, a lot to respond to above and I'm trying to figure out how best to type up all of it.) On Mon, May 13, 2024 at 9:13 AM Robert Haas <[email protected]> wrote: > However, > I disagree with Jacob's assertion that sslmode=require has no security > benefits over sslmode=prefer. For the record, I didn't say that... You mean Jelte's quote up above?: > sslmode=prefer and sslmode=require > are the same amount of insecure imho (i.e. extremely insecure). I agree that requiring passive security is tangibly better than allowing fallback to plaintext. I think Jelte's point might be better stated as, =prefer and =require give the same amount of protection against active attack (none). --Jacob ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 17:41 Jelte Fennema-Nio <[email protected]> parent: Robert Haas <[email protected]> 2 siblings, 1 reply; 30+ messages in thread From: Jelte Fennema-Nio @ 2024-05-13 17:41 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, 13 May 2024 at 18:14, Robert Haas <[email protected]> wrote: > I disagree with Jacob's assertion that sslmode=require has no security > benefits over sslmode=prefer. That seems like the kind of pessimism > that makes people hate security professionals. There have got to be > some attacks that are foreclosed by encrypting the connection, even if > you don't stop MITM attacks or other things that are more > sophisticated than running wireshark and seeing what goes by on the > wire. Like Jacob already said, I guess you meant me here. The main point I was trying to make is that sslmode=require is extremely insecure too, so if we're changing the default then I'd rather bite the bullet and actually make the default a secure one this time. No-ones browser trusts self-signed certs by default, but currently lots of people trust self-signed certs when connecting to their production database without realizing. IMHO the only benefit that sslmode=require brings over sslmode=prefer is detecting incorrectly configured servers i.e. servers that are supposed to support ssl but somehow don't due to a misconfigured GUC/pg_hba. Such "incorrectly configured server" detection avoids sending data to such a server, which an eavesdropper on the network could see. Which is definitely a security benefit, but it's an extremely small one. In all other cases sslmode=prefer brings exactly the same protection as sslmode=require, because sslmode=prefer encrypts the connection unless postgres actively tells the client to downgrade to plaintext (which never happens when the server is configured correctly). ^ permalink raw reply [nested|flat] 30+ messages in thread
* On the use of channel binding without server certificates (was: Direct SSL connection with ALPN and HBA rules) @ 2024-05-13 18:09 Jacob Champion <[email protected]> parent: Heikki Linnakangas <[email protected]> 2 siblings, 0 replies; 30+ messages in thread From: Jacob Champion @ 2024-05-13 18:09 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> [soapbox thread, so I've changed the Subject] On Mon, May 13, 2024 at 4:08 AM Heikki Linnakangas <[email protected]> wrote: > "channel_binding=require sslmode=require" also protects from MITM attacks. This isn't true in the same way that "standard" TLS protects against MITM. I know you know that, but for the benefit of bystanders reading the threads, I think we should stop phrasing it like this. Most people who want MITM protection need to be using verify-full. Details for those bystanders: Channel binding alone will only disconnect you after the MITM is discovered, after your startup packet is leaked but before you send any queries to the server. A hash of your password will also be leaked in that situation, which starts the timer on an offline attack. And IIRC, you won't get an alert that says "someone's in the middle"; it'll just look like you mistyped your password. (Stronger passwords provide stronger protection in this situation, which is not a property that most people are used to. If I choose to sign into Google with the password "hunter2", it doesn't somehow make the TLS protection weaker. But if you rely on SCRAM by itself for server authentication, it does more or less work like that.) Use channel_binding *in addition to* sslmode=verify-full if you want enhanced authentication of the peer, as suggested in the docs [1]. Don't rely on channel binding alone for the vast majority of use cases, and if you know better for your particular use case, then you already know enough to be able to ignore my advice. [/soapbox] Thanks, --Jacob [1] https://www.postgresql.org/docs/current/preventing-server-spoofing.html ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 19:12 Robert Haas <[email protected]> parent: Jacob Champion <[email protected]> 0 siblings, 0 replies; 30+ messages in thread From: Robert Haas @ 2024-05-13 19:12 UTC (permalink / raw) To: Jacob Champion <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Daniel Gustafsson <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, May 13, 2024 at 12:45 PM Jacob Champion <[email protected]> wrote: > For the record, I didn't say that... You mean Jelte's quote up above? Yeah, sorry, I got my J-named hackers confused. Apologies. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 19:24 Robert Haas <[email protected]> parent: Jelte Fennema-Nio <[email protected]> 0 siblings, 0 replies; 30+ messages in thread From: Robert Haas @ 2024-05-13 19:24 UTC (permalink / raw) To: Jelte Fennema-Nio <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, May 13, 2024 at 1:42 PM Jelte Fennema-Nio <[email protected]> wrote: > Like Jacob already said, I guess you meant me here. The main point I > was trying to make is that sslmode=require is extremely insecure too, > so if we're changing the default then I'd rather bite the bullet and > actually make the default a secure one this time. No-ones browser > trusts self-signed certs by default, but currently lots of people > trust self-signed certs when connecting to their production database > without realizing. > > IMHO the only benefit that sslmode=require brings over sslmode=prefer > is detecting incorrectly configured servers i.e. servers that are > supposed to support ssl but somehow don't due to a misconfigured > GUC/pg_hba. Such "incorrectly configured server" detection avoids > sending data to such a server, which an eavesdropper on the network > could see. Which is definitely a security benefit, but it's an > extremely small one. In all other cases sslmode=prefer brings exactly > the same protection as sslmode=require, because sslmode=prefer > encrypts the connection unless postgres actively tells the client to > downgrade to plaintext (which never happens when the server is > configured correctly). I think I agree with *nearly* every word of this. However: (1) I don't want to hijack this thread about a v17 open item to talk too much about a hypothetical v18 proposal. (2) While in general you need more than just SSL to ensure security, I'm not sure that there's only one way to do it, and I doubt that we should try to pick a winner. (3) I suspect that even going as far as sslmode=require by default is going to be extremely painful for hackers, the project, and users. Moving the goalposts further increases the likelihood of nothing happening at all. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-13 22:29 Jacob Champion <[email protected]> parent: Robert Haas <[email protected]> 2 siblings, 1 reply; 30+ messages in thread From: Jacob Champion @ 2024-05-13 22:29 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Daniel Gustafsson <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Mon, May 13, 2024 at 9:13 AM Robert Haas <[email protected]> wrote: > I find this idea to be a massive improvement over the status quo, +1 > and > I didn't spot any major problems when I read through the patch, > either. Definitely not a major problem, but I think select_next_encryption_method() has gone stale, since it originally provided generality and lines of fallback that no longer exist. In other words, I think the following code is now misleading: > if (conn->sslmode[0] == 'a') > SELECT_NEXT_METHOD(ENC_PLAINTEXT); > > SELECT_NEXT_METHOD(ENC_NEGOTIATED_SSL); > SELECT_NEXT_METHOD(ENC_DIRECT_SSL); > > if (conn->sslmode[0] != 'a') > SELECT_NEXT_METHOD(ENC_PLAINTEXT); To me, that implies that negotiated mode takes precedence over direct, but the point of the patch is that it's not possible to have both. And if direct SSL is in use, then sslmode can't be "allow" anyway, and we definitely don't want ENC_PLAINTEXT. So if someone proposes a change to select_next_encryption_method(), you'll have to remember to stare at init_allowed_encryption_methods() as well, and think really hard about what's going on. And vice-versa. That worries me. > I don't have a strong opinion about whether sslnegotiation=direct > should error out (as you propose here) or silently promote sslmode to > require. I think either is defensible. I'm comforted that, since sslrootcert=system already does it, plenty of use cases will get that for free. And if you decide in the future that you really really want it to promote, it won't be a compatibility break to make that change. (That gives us more time for wider v16-17 adoption, to see how the sslrootcert=system magic promotion behavior is going in practice.) > Had I been implementing it, I > think I would have done as Jacob proposes, just because once we've > forced a direct SSL negotiation surely the only sensible behavior is > to be using SSL, unless you think there should be a > silently-reconnect-without-SSL behavior, which I sure don't. We still allow GSS to preempt SSL, though, so "forced" is probably overstating things. > It's really hard to believe in 2024 that anyone should ever be using a > setting that may or may not encrypt the connection. There's http and > https but there's no httpmaybes. +1. I think (someone hop in and correct me please) that Opportunistic Encryption for HTTP mostly fizzled, and they gave it a *lot* of thought. --Jacob ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-14 00:05 Jacob Champion <[email protected]> parent: Heikki Linnakangas <[email protected]> 2 siblings, 0 replies; 30+ messages in thread From: Jacob Champion @ 2024-05-14 00:05 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; Daniel Gustafsson <[email protected]>; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> [this should probably belong to a different thread, but I'm not sure what to title it] On Mon, May 13, 2024 at 4:08 AM Heikki Linnakangas <[email protected]> wrote: > I think these options should be designed from the user's point of view, > so that the user can specify the risks they're willing to accept, and > the details of how that's accomplished are handled by libpq. For > example, I'm OK with (tick all that apply): > > [ ] passive eavesdroppers seeing all the traffic > [ ] MITM being able to hijack the session > [ ] connecting without verifying the server's identity > [ ] divulging the plaintext password to the server > [ ] ... I'm pessimistic about a quality-of-protection scheme for this use case (*). I don't think users need more knobs in their connection strings, and many of the goals of transport encryption are really not independent from each other in practice. As evidence I point to the absolute mess of GSSAPI wrapping, which lets you check separate boxes for things like "require the server to authenticate itself" and "require integrity" and "allow MITMs to reorder messages" and so on, as if the whole idea of "integrity" is useful if you don't know who you're talking to in the first place. I think I recall slapd having something similarly arcane (but at least it doesn't make the clients do it!). Those kinds of APIs don't evolve well, in my opinion. I think most people probably just want browser-grade security as quickly and cheaply as possible, and we don't make that very easy today. I'll try to review a QoP scheme if someone works on it, don't get me wrong, but I'd much rather spend time on a "very secure by default" mode that gets rid of most of the options (i.e. a postgresqls:// scheme). (*) I've proposed quality-of-protection in the past, for Postgres proxy authentication [1]. But I'm comfortable in my hypocrisy, because in that case, the end user doing the configuration is a DBA with a config file who is expected to understand the whole system, and it's a niche use case (IMO) without an obvious "common setup". And frankly I think my proposal is unlikely to go anywhere; the cost/benefit probably isn't good enough. > If you need to verify > the server's identity, it implies sslmode=verify-CA or > channel_binding=true. Neither of those two options provides strong authentication of the peer, and personally I wouldn't want them to be considered worthy of "verify the server's identity" mode. And -- taking a wild leap here -- if we disagree, then granularity becomes a problem: either the QoP scheme now has to have sub-checkboxes for "if the server knows my password, that's good enough" and "it's fine if the server's hostname doesn't match the cert, for some reason", or it smashes all of those different ideas into one setting and then I have to settle for the weakest common denominator during an active attack. Assuming I haven't missed a third option, will that be easier/better than the status quo of require_auth+sslmode? > A different line of thought is that to keep the attack surface as smal > as possible, you should specify very explicitly what exactly you expect > to happen in the authentication, and disallow any variance. For example, > you expect SCRAM to be used, with a certificate signed by a particular > CA, and if the server requests anything else, that's suspicious and the > connection is aborted. We should make that possible too That's 'require_auth=scram-sha-256 sslmode=verify-ca', no? --Jacob [1] https://www.postgresql.org/message-id/0768cedb-695a-8841-5f8b-da2aa64c8f3a%40timescale.com ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-15 13:33 Heikki Linnakangas <[email protected]> parent: Jacob Champion <[email protected]> 0 siblings, 2 replies; 30+ messages in thread From: Heikki Linnakangas @ 2024-05-15 13:33 UTC (permalink / raw) To: Jacob Champion <[email protected]>; Robert Haas <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On 14/05/2024 01:29, Jacob Champion wrote: > Definitely not a major problem, but I think > select_next_encryption_method() has gone stale, since it originally > provided generality and lines of fallback that no longer exist. In > other words, I think the following code is now misleading: > >> if (conn->sslmode[0] == 'a') >> SELECT_NEXT_METHOD(ENC_PLAINTEXT); >> >> SELECT_NEXT_METHOD(ENC_NEGOTIATED_SSL); >> SELECT_NEXT_METHOD(ENC_DIRECT_SSL); >> >> if (conn->sslmode[0] != 'a') >> SELECT_NEXT_METHOD(ENC_PLAINTEXT); > > To me, that implies that negotiated mode takes precedence over direct, > but the point of the patch is that it's not possible to have both. And > if direct SSL is in use, then sslmode can't be "allow" anyway, and we > definitely don't want ENC_PLAINTEXT. > > So if someone proposes a change to select_next_encryption_method(), > you'll have to remember to stare at init_allowed_encryption_methods() > as well, and think really hard about what's going on. And vice-versa. > That worries me. Ok, yeah, I can see that now. Here's a new version to address that. I merged ENC_SSL_NEGOTIATED_SSL and ENC_SSL_DIRECT_SSL to a single method, ENC_SSL. The places that need to distinguish between them now check conn-sslnegotiation. That seems more clear now that there is no fallback. -- Heikki Linnakangas Neon (https://neon.tech) Attachments: [text/x-patch] v2-0001-Remove-option-to-fall-back-from-direct-to-postgre.patch (46.9K, ../../[email protected]/2-v2-0001-Remove-option-to-fall-back-from-direct-to-postgre.patch) download | inline diff: From 7a2bc2ede5ba7bef147e509ce3c4d5472c8e0247 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas <[email protected]> Date: Wed, 15 May 2024 16:27:51 +0300 Subject: [PATCH v2 1/1] Remove option to fall back from direct to postgres SSL negotiation There were three problems with the sslnegotiation options: 1. The sslmode=prefer and sslnegotiation=requiredirect combination was somewhat dangerous, as you might unintentionally fall back to plaintext authentication when connecting to a pre-v17 server. 2. There was an asymmetry between 'postgres' and 'direct' options. 'postgres' meant "try only traditional negotiation", while 'direct' meant "try direct first, and fall back to traditional negotiation if it fails". That was apparent only if you knew that the 'requiredirect' mode also exists. 3. The "require" word in 'requiredirect' suggests that it's somehow more strict or more secure, similar to sslmode. However, I don't consider direct SSL connections to be a security feature. To address these problems: - Only allow sslnegotiation='direct' if sslmode='require' or stronger. And for the record, Jacob and Robert felt that we should do that (or have sslnegotiation='direct' imply sslmode='require') anyway, regardless of the first issue. - Remove the 'direct' mode that falls back to traditional negotiation, and rename what was called 'requiredirect' to 'direct' instead. In other words, there is no "try both methods" option anymore, 'postgres' now means the traditional negotiation and 'direct' means a direct SSL connection. Reviewed-by: Jelte Fennema-Nio, Robert Haas, Jacob Champion Discussion: https://www.postgresql.org/message-id/d3b1608a-a1b6-4eda-9ec5-ddb3e4375808%40iki.fi --- doc/src/sgml/libpq.sgml | 49 ++-- src/interfaces/libpq/fe-connect.c | 144 +++++----- src/interfaces/libpq/fe-secure-openssl.c | 2 +- src/interfaces/libpq/libpq-int.h | 6 +- .../libpq/t/005_negotiate_encryption.pl | 254 ++++++++---------- 5 files changed, 202 insertions(+), 253 deletions(-) diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 1d32c226d8..b32e497b1b 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -1772,15 +1772,18 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname <term><literal>sslnegotiation</literal></term> <listitem> <para> - This option controls whether <productname>PostgreSQL</productname> - will perform its protocol negotiation to request encryption from the - server or will just directly make a standard <acronym>SSL</acronym> - connection. Traditional <productname>PostgreSQL</productname> - protocol negotiation is the default and the most flexible with - different server configurations. If the server is known to support - direct <acronym>SSL</acronym> connections then the latter requires one - fewer round trip reducing connection latency and also allows the use - of protocol agnostic SSL network tools. + This option controls how SSL encryption is negotiated with the server, + if SSL is used. In the default <literal>postgres</literal> mode, the + client first asks the server if SSL is supported. In + <literal>direct</literal> mode, the client starts the standard SSL + handshake directly after establishing the TCP/IP connection. Traditional + <productname>PostgreSQL</productname> protocol negotiation is the most + flexible with different server configurations. If the server is known + to support direct <acronym>SSL</acronym> connections then the latter + requires one fewer round trip reducing connection latency and also + allows the use of protocol agnostic SSL network tools. The direct SSL + option was introduced in <productname>PostgreSQL</productname> version + 17. </para> <variablelist> @@ -1798,32 +1801,14 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname <term><literal>direct</literal></term> <listitem> <para> - first attempt to establish a standard SSL connection and if that - fails reconnect and perform the negotiation. This fallback - process adds significant latency if the initial SSL connection - fails. - </para> - <para> - An exception is if <literal>gssencmode</literal> is set - to <literal>prefer</literal>, but the server rejects GSS encryption. - In that case, SSL is negotiated over the same TCP connection using - <productname>PostgreSQL</productname> protocol negotiation. In - other words, the direct SSL handshake is not used, if a TCP - connection has already been established and can be used for the + start SSL handshake directly after establishing the TCP/IP + connection. This is only allowed with sslmode=require or higher, + because the weaker settings could lead to unintended fallback to + plaintext authentication when the server does not support direct SSL handshake. </para> </listitem> </varlistentry> - - <varlistentry> - <term><literal>requiredirect</literal></term> - <listitem> - <para> - attempt to establish a standard SSL connection and if that fails - return a connection failure immediately. - </para> - </listitem> - </varlistentry> </variablelist> </listitem> </varlistentry> @@ -2064,7 +2049,7 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname connections without having to decrypt the SSL stream. (Note that unless the proxy is aware of the PostgreSQL protocol handshake this would require setting <literal>sslnegotiation</literal> - to <literal>direct</literal> or <literal>requiredirect</literal>.) + to <literal>direct</literal>.) However, <acronym>SNI</acronym> makes the destination host name appear in cleartext in the network traffic, so it might be undesirable in some cases. diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index 81d278c395..85f55b14a9 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -274,7 +274,7 @@ static const internalPQconninfoOption PQconninfoOptions[] = { offsetof(struct pg_conn, sslmode)}, {"sslnegotiation", "PGSSLNEGOTIATION", DefaultSSLNegotiation, NULL, - "SSL-Negotiation", "", 14, /* sizeof("requiredirect") == 14 */ + "SSL-Negotiation", "", 9, /* sizeof("postgres") == 9 */ offsetof(struct pg_conn, sslnegotiation)}, {"sslcompression", "PGSSLCOMPRESSION", "0", NULL, @@ -1590,8 +1590,7 @@ pqConnectOptions2(PGconn *conn) if (conn->sslnegotiation) { if (strcmp(conn->sslnegotiation, "postgres") != 0 - && strcmp(conn->sslnegotiation, "direct") != 0 - && strcmp(conn->sslnegotiation, "requiredirect") != 0) + && strcmp(conn->sslnegotiation, "direct") != 0) { conn->status = CONNECTION_BAD; libpq_append_conn_error(conn, "invalid %s value: \"%s\"", @@ -1608,6 +1607,25 @@ pqConnectOptions2(PGconn *conn) return false; } #endif + + /* + * Don't allow direct SSL negotiation with sslmode='prefer', because + * that poses a risk of unintentional fallback to plaintext connection + * when connecting to a pre-v17 server that does not support direct + * SSL connections. To keep things simple, don't allow it with + * sslmode='allow' or sslmode='disable' either. If a user goes through + * the trouble of setting sslnegotiation='direct', they probably + * intend to use SSL, and sslmode=disable or allow is probably a user + * user mistake anyway. + */ + if (conn->sslnegotiation[0] == 'd' && + conn->sslmode[0] != 'r' && conn->sslmode[0] != 'v') + { + conn->status = CONNECTION_BAD; + libpq_append_conn_error(conn, "weak sslmode \"%s\" may not be used with sslnegotiation=direct (use \"require\", \"verify-ca\", or \"verify-full\")", + conn->sslmode); + return false; + } } else { @@ -3347,42 +3365,47 @@ keep_going: /* We will come back to here until there is goto error_return; /* - * If direct SSL is enabled, jump right into SSL handshake. We - * will come back here after SSL encryption has been - * established, with ssl_in_use set. - */ - if (conn->current_enc_method == ENC_DIRECT_SSL && !conn->ssl_in_use) - { - conn->status = CONNECTION_SSL_STARTUP; - return PGRES_POLLING_WRITING; - } - - /* - * If negotiated SSL is enabled, request SSL and proceed with - * SSL handshake. We will come back here after SSL encryption - * has been established, with ssl_in_use set. + * If SSL is enabled, start the SSL negotiation. We will come + * back here after SSL encryption has been established, with + * ssl_in_use set. */ - if (conn->current_enc_method == ENC_NEGOTIATED_SSL && !conn->ssl_in_use) + if (conn->current_enc_method == ENC_SSL && !conn->ssl_in_use) { - ProtocolVersion pv; - /* - * Send the SSL request packet. - * - * Theoretically, this could block, but it really - * shouldn't since we only got here if the socket is - * write-ready. + * If traditional postgres SSL negotiation is used, send + * the SSL request. In direct negotiation, jump straight + * into the SSL handshake. */ - pv = pg_hton32(NEGOTIATE_SSL_CODE); - if (pqPacketSend(conn, 0, &pv, sizeof(pv)) != STATUS_OK) + if (conn->sslnegotiation[0] == 'p') { - libpq_append_conn_error(conn, "could not send SSL negotiation packet: %s", - SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf))); - goto error_return; + ProtocolVersion pv; + + Assert(conn->sslnegotiation[0] == 'p'); + + /* + * Send the SSL request packet. + * + * Theoretically, this could block, but it really + * shouldn't since we only got here if the socket is + * write-ready. + */ + pv = pg_hton32(NEGOTIATE_SSL_CODE); + if (pqPacketSend(conn, 0, &pv, sizeof(pv)) != STATUS_OK) + { + libpq_append_conn_error(conn, "could not send SSL negotiation packet: %s", + SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf))); + goto error_return; + } + /* Ok, wait for response */ + conn->status = CONNECTION_SSL_STARTUP; + return PGRES_POLLING_READING; + } + else + { + Assert(conn->sslnegotiation[0] == 'd'); + conn->status = CONNECTION_SSL_STARTUP; + return PGRES_POLLING_WRITING; } - /* Ok, wait for response */ - conn->status = CONNECTION_SSL_STARTUP; - return PGRES_POLLING_READING; } #endif /* USE_SSL */ @@ -3453,11 +3476,11 @@ keep_going: /* We will come back to here until there is PostgresPollingStatusType pollres; /* - * On first time through, get the postmaster's response to our - * SSL negotiation packet. If we are trying a direct ssl - * connection, go straight to initiating ssl. + * On first time through with traditional SSL negotiation, get + * the postmaster's response to our SSLRequest packet. With + * sslnegotiation='direct', go straight to initiating SSL. */ - if (!conn->ssl_in_use && conn->current_enc_method == ENC_NEGOTIATED_SSL) + if (!conn->ssl_in_use && conn->sslnegotiation[0] == 'p') { /* * We use pqReadData here since it has the logic to @@ -4282,7 +4305,7 @@ init_allowed_encryption_methods(PGconn *conn) if (conn->raddr.addr.ss_family == AF_UNIX) { /* Don't request SSL or GSSAPI over Unix sockets */ - conn->allowed_enc_methods &= ~(ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL | ENC_GSSAPI); + conn->allowed_enc_methods &= ~(ENC_SSL | ENC_GSSAPI); /* * XXX: we probably should not do this. sslmode=require works @@ -4309,12 +4332,7 @@ init_allowed_encryption_methods(PGconn *conn) /* sslmode anything but 'disable', and GSSAPI not required */ if (conn->sslmode[0] != 'd' && conn->gssencmode[0] != 'r') { - if (conn->sslnegotiation[0] == 'p') - conn->allowed_enc_methods |= ENC_NEGOTIATED_SSL; - else if (conn->sslnegotiation[0] == 'd') - conn->allowed_enc_methods |= ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL; - else if (conn->sslnegotiation[0] == 'r') - conn->allowed_enc_methods |= ENC_DIRECT_SSL; + conn->allowed_enc_methods |= ENC_SSL; } #endif @@ -4354,7 +4372,8 @@ encryption_negotiation_failed(PGconn *conn) if (select_next_encryption_method(conn, true)) { - if (conn->current_enc_method == ENC_DIRECT_SSL) + /* An existing connection cannot be reused for direct SSL */ + if (conn->current_enc_method == ENC_SSL && conn->sslnegotiation[0] == 'd') return 2; else return 1; @@ -4376,18 +4395,6 @@ connection_failed(PGconn *conn) Assert((conn->failed_enc_methods & conn->current_enc_method) == 0); conn->failed_enc_methods |= conn->current_enc_method; - /* - * If the server reported an error after the SSL handshake, no point in - * retrying with negotiated vs direct SSL. - */ - if ((conn->current_enc_method & (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL)) != 0 && - conn->ssl_handshake_started) - { - conn->failed_enc_methods |= (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL) & conn->allowed_enc_methods; - } - else - conn->failed_enc_methods |= conn->current_enc_method; - return select_next_encryption_method(conn, false); } @@ -4445,24 +4452,17 @@ select_next_encryption_method(PGconn *conn, bool have_valid_connection) SELECT_NEXT_METHOD(ENC_GSSAPI); #endif - /* With sslmode=allow, try plaintext connection before SSL. */ - if (conn->sslmode[0] == 'a') - SELECT_NEXT_METHOD(ENC_PLAINTEXT); - /* - * If enabled, try direct SSL. Unless we have a valid TCP connection that - * failed negotiating GSSAPI encryption; in that case we prefer to reuse - * the connection with negotiated SSL, instead of reconnecting to do - * direct SSL. The point of sslnegotiation=direct is to avoid the - * roundtrip from the negotiation, but reconnecting would also incur a - * roundtrip. (In sslnegotiation=requiredirect mode, negotiated SSL is not - * in the list of allowed methods and we will reconnect.) + * The order between SSL encryption and plaintext depends on sslmode. With + * sslmode=allow, try plaintext connection before SSL. With + * sslmode=prefer, it's the other way round. With other modes, we only try + * plaintext or SSL connections so the order they're listed here doesn't + * matter. */ - if (have_valid_connection) - SELECT_NEXT_METHOD(ENC_NEGOTIATED_SSL); + if (conn->sslmode[0] == 'a') + SELECT_NEXT_METHOD(ENC_PLAINTEXT); - SELECT_NEXT_METHOD(ENC_DIRECT_SSL); - SELECT_NEXT_METHOD(ENC_NEGOTIATED_SSL); + SELECT_NEXT_METHOD(ENC_SSL); if (conn->sslmode[0] != 'a') SELECT_NEXT_METHOD(ENC_PLAINTEXT); diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c index fb6bb911f5..bf9dfbf918 100644 --- a/src/interfaces/libpq/fe-secure-openssl.c +++ b/src/interfaces/libpq/fe-secure-openssl.c @@ -1586,7 +1586,7 @@ open_client_SSL(PGconn *conn) } /* ALPN is mandatory with direct SSL connections */ - if (conn->current_enc_method == ENC_DIRECT_SSL) + if (conn->current_enc_method == ENC_SSL && conn->sslnegotiation[0] == 'd') { const unsigned char *selected; unsigned int len; diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h index 3691e5ee96..3886204c70 100644 --- a/src/interfaces/libpq/libpq-int.h +++ b/src/interfaces/libpq/libpq-int.h @@ -235,8 +235,7 @@ typedef enum #define ENC_ERROR 0 #define ENC_PLAINTEXT 0x01 #define ENC_GSSAPI 0x02 -#define ENC_DIRECT_SSL 0x04 -#define ENC_NEGOTIATED_SSL 0x08 +#define ENC_SSL 0x04 /* Target server type (decoded value of target_session_attrs) */ typedef enum @@ -395,8 +394,7 @@ struct pg_conn char *keepalives_count; /* maximum number of TCP keepalive * retransmits */ char *sslmode; /* SSL mode (require,prefer,allow,disable) */ - char *sslnegotiation; /* SSL initiation style - * (postgres,direct,requiredirect) */ + char *sslnegotiation; /* SSL initiation style (postgres,direct) */ char *sslcompression; /* SSL compression (0 or 1) */ char *sslkey; /* client key filename */ char *sslcert; /* client certificate filename */ diff --git a/src/interfaces/libpq/t/005_negotiate_encryption.pl b/src/interfaces/libpq/t/005_negotiate_encryption.pl index bc19cd244b..c3f70d31bc 100644 --- a/src/interfaces/libpq/t/005_negotiate_encryption.pl +++ b/src/interfaces/libpq/t/005_negotiate_encryption.pl @@ -213,7 +213,7 @@ my @all_test_users = ('testuser', 'ssluser', 'nossluser', 'gssuser', 'nogssuser'); my @all_gssencmodes = ('disable', 'prefer', 'require'); my @all_sslmodes = ('disable', 'allow', 'prefer', 'require'); -my @all_sslnegotiations = ('postgres', 'direct', 'requiredirect'); +my @all_sslnegotiations = ('postgres', 'direct'); my $server_config = { server_ssl => 0, @@ -228,23 +228,22 @@ if ($ssl_supported) { $test_table = q{ # USER GSSENCMODE SSLMODE SSLNEGOTIATION EVENTS -> OUTCOME -testuser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain +testuser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslreject, authok -> plain -. . . direct connect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, directsslreject, reconnect, authok -> plain . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail -. prefer disable * connect, authok -> plain -. . allow * connect, authok -> plain +. . . direct connect, directsslreject -> fail +. prefer disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslreject, authok -> plain -. . . direct connect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, directsslreject, reconnect, authok -> plain . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail - }; +. . . direct connect, directsslreject -> fail + +# sslnegotiation=direct is not acccepted unless sslmode=require or stronger +* * disable direct - -> fail +* * allow direct - -> fail +* * prefer direct - -> fail +}; } else { @@ -258,11 +257,10 @@ testuser disable disable postgres connect, authok . . allow postgres connect, authok -> plain . . prefer postgres connect, authok -> plain -# Without SSL support, sslmode=require and sslnegotiation=direct/requiredirect -# are not accepted at all. -. * require * - -> fail -. * * direct - -> fail -. * * requiredirect - -> fail +# Without SSL support, sslmode=require and sslnegotiation=direct are +# not accepted at all +* * require * - -> fail +* * * direct - -> fail }; } @@ -288,34 +286,26 @@ SKIP: $test_table = q{ # USER GSSENCMODE SSLMODE SSLNEGOTIATION EVENTS -> OUTCOME -testuser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain +testuser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl . . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -ssluser . disable * connect, authfail -> fail +ssluser . disable postgres connect, authfail -> fail . . allow postgres connect, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, authfail, reconnect, directsslaccept, authok -> ssl . . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl . . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -nossluser . disable * connect, authok -> plain +nossluser . disable postgres connect, authok -> plain . . allow postgres connect, authok -> plain -. . . direct connect, authok -> plain -. . . requiredirect connect, authok -> plain . . prefer postgres connect, sslaccept, authfail, reconnect, authok -> plain -. . . direct connect, directsslaccept, authfail, reconnect, authok -> plain -. . . requiredirect connect, directsslaccept, authfail, reconnect, authok -> plain . . require postgres connect, sslaccept, authfail -> fail . . require direct connect, directsslaccept, authfail -> fail -. . require requiredirect connect, directsslaccept, authfail -> fail + +# sslnegotiation=direct is not acccepted unless sslmode=require or stronger +* * disable direct - -> fail +* * allow direct - -> fail +* * prefer direct - -> fail }; # Enable SSL in the server @@ -350,62 +340,54 @@ SKIP: $test_table = q{ # USER GSSENCMODE SSLMODE SSLNEGOTIATION EVENTS -> OUTCOME -testuser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain +testuser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslreject, authok -> plain -. . . direct connect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, directsslreject, reconnect, authok -> plain . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail -. prefer * * connect, gssaccept, authok -> gss -. require * * connect, gssaccept, authok -> gss +. . . direct connect, directsslreject -> fail +. prefer * postgres connect, gssaccept, authok -> gss +. prefer require direct connect, gssaccept, authok -> gss +. require * postgres connect, gssaccept, authok -> gss +. . require direct connect, gssaccept, authok -> gss -gssuser disable disable * connect, authfail -> fail +gssuser disable disable postgres connect, authfail -> fail . . allow postgres connect, authfail, reconnect, sslreject -> fail -. . . direct connect, authfail, reconnect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, authfail, reconnect, directsslreject -> fail . . prefer postgres connect, sslreject, authfail -> fail -. . . direct connect, directsslreject, reconnect, sslreject, authfail -> fail -. . . requiredirect connect, directsslreject, reconnect, authfail -> fail . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail -. prefer * * connect, gssaccept, authok -> gss -. require * * connect, gssaccept, authok -> gss +. . . direct connect, directsslreject -> fail +. prefer * postgres connect, gssaccept, authok -> gss +. prefer require direct connect, gssaccept, authok -> gss +. require * postgres connect, gssaccept, authok -> gss +. . require direct connect, gssaccept, authok -> gss -nogssuser disable disable * connect, authok -> plain +nogssuser disable disable postgres connect, authok -> plain . . allow postgres connect, authok -> plain -. . . direct connect, authok -> plain -. . . requiredirect connect, authok -> plain . . prefer postgres connect, sslreject, authok -> plain -. . . direct connect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, directsslreject, reconnect, authok -> plain . . require postgres connect, sslreject -> fail -. . . direct connect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, directsslreject -> fail -. prefer disable * connect, gssaccept, authfail, reconnect, authok -> plain +. . . direct connect, directsslreject -> fail +. prefer disable postgres connect, gssaccept, authfail, reconnect, authok -> plain . . allow postgres connect, gssaccept, authfail, reconnect, authok -> plain -. . . direct connect, gssaccept, authfail, reconnect, authok -> plain -. . . requiredirect connect, gssaccept, authfail, reconnect, authok -> plain . . prefer postgres connect, gssaccept, authfail, reconnect, sslreject, authok -> plain -. . . direct connect, gssaccept, authfail, reconnect, directsslreject, reconnect, sslreject, authok -> plain -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslreject, reconnect, authok -> plain . . require postgres connect, gssaccept, authfail, reconnect, sslreject -> fail -. . . direct connect, gssaccept, authfail, reconnect, directsslreject, reconnect, sslreject -> fail -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslreject -> fail -. require disable * connect, gssaccept, authfail -> fail -. . allow * connect, gssaccept, authfail -> fail -. . prefer * connect, gssaccept, authfail -> fail -. . require * connect, gssaccept, authfail -> fail # If both GSSAPI and sslmode are required, and GSS is not available -> fail +. . . direct connect, gssaccept, authfail, reconnect, directsslreject -> fail +. require disable postgres connect, gssaccept, authfail -> fail +. . allow postgres connect, gssaccept, authfail -> fail +. . prefer postgres connect, gssaccept, authfail -> fail +. . require postgres connect, gssaccept, authfail -> fail # If both GSSAPI and sslmode are required, and GSS is not available -> fail +. . . direct connect, gssaccept, authfail -> fail # If both GSSAPI and sslmode are required, and GSS is not available -> fail + +# sslnegotiation=direct is not acccepted unless sslmode=require or stronger +* * disable direct - -> fail +* * allow direct - -> fail +* * prefer direct - -> fail }; # The expected events and outcomes above assume that SSL support # is enabled. When libpq is compiled without SSL support, all # attempts to connect with sslmode=require or - # sslnegotiation=direct/requiredirect would fail immediately without - # even connecting to the server. Skip those, because we tested - # them earlier already. + # sslnegotiation=direct would fail immediately without even + # connecting to the server. Skip those, because we tested them + # earlier already. my ($sslmodes, $sslnegotiations); if ($ssl_supported != 0) { @@ -445,100 +427,84 @@ SKIP: $test_table = q{ # USER GSSENCMODE SSLMODE SSLNEGOTIATION EVENTS -> OUTCOME -testuser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain -. . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -. . require postgres connect, sslaccept, authok -> ssl +testuser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain +. . prefer postgres connect, sslaccept, authok -> ssl +. . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -. prefer disable * connect, gssaccept, authok -> gss -. . allow * connect, gssaccept, authok -> gss -. . prefer * connect, gssaccept, authok -> gss -. . require * connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require -. require disable * connect, gssaccept, authok -> gss -. . allow * connect, gssaccept, authok -> gss -. . prefer * connect, gssaccept, authok -> gss -. . require * connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require - -gssuser disable disable * connect, authfail -> fail +. prefer disable postgres connect, gssaccept, authok -> gss +. . allow postgres connect, gssaccept, authok -> gss +. . prefer postgres connect, gssaccept, authok -> gss +. . require postgres connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require +. . . direct connect, gssaccept, authok -> gss +. require disable postgres connect, gssaccept, authok -> gss +. . allow postgres connect, gssaccept, authok -> gss +. . prefer postgres connect, gssaccept, authok -> gss +. . require postgres connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require +. . . direct connect, gssaccept, authok -> gss + +gssuser disable disable postgres connect, authfail -> fail . . allow postgres connect, authfail, reconnect, sslaccept, authfail -> fail -. . . direct connect, authfail, reconnect, directsslaccept, authfail -> fail -. . . requiredirect connect, authfail, reconnect, directsslaccept, authfail -> fail . . prefer postgres connect, sslaccept, authfail, reconnect, authfail -> fail -. . . direct connect, directsslaccept, authfail, reconnect, authfail -> fail -. . . requiredirect connect, directsslaccept, authfail, reconnect, authfail -> fail . . require postgres connect, sslaccept, authfail -> fail . . . direct connect, directsslaccept, authfail -> fail -. . . requiredirect connect, directsslaccept, authfail -> fail -. prefer disable * connect, gssaccept, authok -> gss -. . allow * connect, gssaccept, authok -> gss -. . prefer * connect, gssaccept, authok -> gss -. . require * connect, gssaccept, authok -> gss # GSS is chosen over SSL, even though sslmode=require -. require disable * connect, gssaccept, authok -> gss -. . allow * connect, gssaccept, authok -> gss -. . prefer * connect, gssaccept, authok -> gss -. . require * connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require - -ssluser disable disable * connect, authfail -> fail +. prefer disable postgres connect, gssaccept, authok -> gss +. . allow postgres connect, gssaccept, authok -> gss +. . prefer postgres connect, gssaccept, authok -> gss +. . require postgres connect, gssaccept, authok -> gss # GSS is chosen over SSL, even though sslmode=require +. . . direct connect, gssaccept, authok -> gss +. require disable postgres connect, gssaccept, authok -> gss +. . allow postgres connect, gssaccept, authok -> gss +. . prefer postgres connect, gssaccept, authok -> gss +. . require postgres connect, gssaccept, authok -> gss # If both GSS and SSL is possible, GSS is chosen over SSL, even if sslmode=require +. . . direct connect, gssaccept, authok -> gss + +ssluser disable disable postgres connect, authfail -> fail . . allow postgres connect, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, authfail, reconnect, directsslaccept, authok -> ssl . . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl . . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -. prefer disable * connect, gssaccept, authfail, reconnect, authfail -> fail +. prefer disable postgres connect, gssaccept, authfail, reconnect, authfail -> fail . . allow postgres connect, gssaccept, authfail, reconnect, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, gssaccept, authfail, reconnect, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, authfail, reconnect, directsslaccept, authok -> ssl . . prefer postgres connect, gssaccept, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl . . require postgres connect, gssaccept, authfail, reconnect, sslaccept, authok -> ssl . . . direct connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. require disable * connect, gssaccept, authfail -> fail -. . allow * connect, gssaccept, authfail -> fail -. . prefer * connect, gssaccept, authfail -> fail -. . require * connect, gssaccept, authfail -> fail # If both GSS and SSL are required, the sslmode=require is effectively ignored and GSS is required +. require disable postgres connect, gssaccept, authfail -> fail +. . allow postgres connect, gssaccept, authfail -> fail +. . prefer postgres connect, gssaccept, authfail -> fail +. . require postgres connect, gssaccept, authfail -> fail # If both GSS and SSL are required, the sslmode=require is effectively ignored and GSS is required +. . . direct connect, gssaccept, authfail -> fail -nogssuser disable disable * connect, authok -> plain +nogssuser disable disable postgres connect, authok -> plain . . allow postgres connect, authok -> plain -. . . direct connect, authok -> plain -. . . requiredirect connect, authok -> plain . . prefer postgres connect, sslaccept, authok -> ssl -. . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl . . require postgres connect, sslaccept, authok -> ssl . . . direct connect, directsslaccept, authok -> ssl -. . . requiredirect connect, directsslaccept, authok -> ssl -. prefer disable * connect, gssaccept, authfail, reconnect, authok -> plain -. . allow * connect, gssaccept, authfail, reconnect, authok -> plain +. prefer disable postgres connect, gssaccept, authfail, reconnect, authok -> plain +. . allow postgres connect, gssaccept, authfail, reconnect, authok -> plain . . prefer postgres connect, gssaccept, authfail, reconnect, sslaccept, authok -> ssl -. . . direct connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl . . require postgres connect, gssaccept, authfail, reconnect, sslaccept, authok -> ssl . . . direct connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. . . requiredirect connect, gssaccept, authfail, reconnect, directsslaccept, authok -> ssl -. require disable * connect, gssaccept, authfail -> fail -. . allow * connect, gssaccept, authfail -> fail -. . prefer * connect, gssaccept, authfail -> fail -. . require * connect, gssaccept, authfail -> fail # If both GSS and SSL are required, the sslmode=require is effectively ignored and GSS is required - -nossluser disable disable * connect, authok -> plain -. . allow * connect, authok -> plain +. require disable postgres connect, gssaccept, authfail -> fail +. . allow postgres connect, gssaccept, authfail -> fail +. . prefer postgres connect, gssaccept, authfail -> fail +. . require postgres connect, gssaccept, authfail -> fail # If both GSS and SSL are required, the sslmode=require is effectively ignored and GSS is required +. . . direct connect, gssaccept, authfail -> fail + +nossluser disable disable postgres connect, authok -> plain +. . allow postgres connect, authok -> plain . . prefer postgres connect, sslaccept, authfail, reconnect, authok -> plain -. . . direct connect, directsslaccept, authfail, reconnect, authok -> plain -. . . requiredirect connect, directsslaccept, authfail, reconnect, authok -> plain . . require postgres connect, sslaccept, authfail -> fail . . . direct connect, directsslaccept, authfail -> fail -. . . requiredirect connect, directsslaccept, authfail -> fail -. prefer * * connect, gssaccept, authok -> gss -. require * * connect, gssaccept, authok -> gss +. prefer * postgres connect, gssaccept, authok -> gss +. . require direct connect, gssaccept, authok -> gss +. require * postgres connect, gssaccept, authok -> gss +. . require direct connect, gssaccept, authok -> gss + +# sslnegotiation=direct is not acccepted unless sslmode=require or stronger +* * disable direct - -> fail +* * allow direct - -> fail +* * prefer direct - -> fail }; note("Running tests with both GSS and SSL enabled in server"); -- 2.39.2 ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-15 18:24 Jacob Champion <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 0 replies; 30+ messages in thread From: Jacob Champion @ 2024-05-15 18:24 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Wed, May 15, 2024 at 6:33 AM Heikki Linnakangas <[email protected]> wrote: > Ok, yeah, I can see that now. Here's a new version to address that. I > merged ENC_SSL_NEGOTIATED_SSL and ENC_SSL_DIRECT_SSL to a single method, > ENC_SSL. The places that need to distinguish between them now check > conn-sslnegotiation. That seems more clear now that there is no fallback. That change and the new comment that were added seem a lot clearer to me, too; +1. And I like that this potentially preps for encryption=gss/ssl/none or similar. This assertion seems a little strange to me: > if (conn->sslnegotiation[0] == 'p') > { > ProtocolVersion pv; > > Assert(conn->sslnegotiation[0] == 'p'); But other than that nitpick, nothing else jumps out at me at the moment. Thanks, --Jacob ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: Direct SSL connection with ALPN and HBA rules @ 2024-05-16 13:53 Robert Haas <[email protected]> parent: Heikki Linnakangas <[email protected]> 1 sibling, 0 replies; 30+ messages in thread From: Robert Haas @ 2024-05-16 13:53 UTC (permalink / raw) To: Heikki Linnakangas <[email protected]>; +Cc: Jacob Champion <[email protected]>; Daniel Gustafsson <[email protected]>; Michael Paquier <[email protected]>; Postgres hackers <[email protected]> On Wed, May 15, 2024 at 9:33 AM Heikki Linnakangas <[email protected]> wrote: > Ok, yeah, I can see that now. Here's a new version to address that. I > merged ENC_SSL_NEGOTIATED_SSL and ENC_SSL_DIRECT_SSL to a single method, > ENC_SSL. The places that need to distinguish between them now check > conn-sslnegotiation. That seems more clear now that there is no fallback. Unless there is a compelling reason to do otherwise, we should expedite getting this committed so that it is included in beta1. Release freeze begins Saturday. Thanks, -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: relfilenode statistics @ 2026-07-10 06:05 Michael Paquier <[email protected]> 0 siblings, 1 reply; 30+ messages in thread From: Michael Paquier @ 2026-07-10 06:05 UTC (permalink / raw) To: Bertrand Drouvot <[email protected]>; +Cc: Andres Freund <[email protected]>; Kirill Reshke <[email protected]>; Robert Haas <[email protected]>; [email protected] On Mon, Mar 09, 2026 at 07:43:43AM +0000, Bertrand Drouvot wrote: > PFA, a new mandatory rebase. So, we have discussed that a couple of days ago offline, and I still felt uncomfortable to not do a cleanup of the relation stats data for indexes. A lot of the fields are stored in memory and written in the file, and remain unused for indexes. So, please find attached a patch set demonstrating the ideas I had on the matter: - 0001 refactors PgStat_StatTabEntry so as the data for indexes and tables is split. The advantage of this one is a reduction of the shared memory usage, as most of the fields of PgStat_StatTabEntry are not used for indexes. These are hidden behind a new stats kind. That's only refactoring, no changes. If you look closely, you would notice this one: + .pending_size = sizeof(PgStat_TableStatus), While weird in itself, that links to a problem addressed by 0002 related to the handling of pending data. - 0002 reworks the way we store pending data, for the transactional and non-transactional bits, as indexes don't care about the transactional parts. PgStat_TableStatus is renamed, refactored so as the table and index parts are split into two. That finishes the split of both stats kinds, preparing for the next patch with the introduction of relfilenode stats. - 0003, super WIP, that I am happy to share based on the fact that it is able to populate the stats entries at replay (tested quickly a few replay scenarios, seems to basically work). Now, while it passes check-world, this patch has various issues that I did not look much into yet (lack of time), but it shows the idea. Note that "get" functions do a direct lookup at the relfilenode, implying modifications in the views to rely on pg_relation_filenode(). Perhaps we should do things the other way around, where we still pass an OID but look at the relation mapper for shared catalogs each time one of these functions is called. This has the effect to replace the various "get" functions to some "relfilenode/rfn" flavors, that can get the fields directly by getting a relfilenode in input. One property that I do not wish to keep around is the aggregation of counters across rewrites, as the new counters don't make sense once we switch to a new relfilenode. I'm feeling the right amount of interface in 0001. 0002 is also quite nice after some time looking at it. 0003 needs a lot more work to be presentable; the basics are surely interesting to see done this way. Comments and thoughts are welcome. -- Michael From 0fd23ab6f58fc43a9086975d9d2d5a3ad67c466b Mon Sep 17 00:00:00 2001 From: Michael Paquier <[email protected]> Date: Fri, 10 Jul 2026 12:52:03 +0900 Subject: [PATCH v13 1/3] Split PgStat_StatTabEntry into separate table and index structures Table and index stats structures are now split, with PgStat_StatIdxEntry containing only the fields relevant to index statistics. Index entries do not care about most of the fields saved for tables, so this saves up to 200 bytes for each stats entry in shared memory. A new stats kind called PGSTAT_KIND_INDEX is created, with the fetch functions for the fields, that are plugged into the system views. XXX: Bump of PGSTAT_FILE_FORMAT_ID required. --- src/include/catalog/pg_proc.dat | 59 ++++++++- src/include/pgstat.h | 20 +++ src/include/utils/pgstat_internal.h | 15 +++ src/include/utils/pgstat_kind.h | 23 ++-- src/backend/catalog/system_views.sql | 40 +++--- src/backend/utils/activity/Makefile | 1 + src/backend/utils/activity/meson.build | 1 + src/backend/utils/activity/pgstat.c | 16 +++ src/backend/utils/activity/pgstat_index.c | 129 +++++++++++++++++++ src/backend/utils/activity/pgstat_relation.c | 106 +++++++++++---- src/backend/utils/adt/pgstatfuncs.c | 108 ++++++++++++++++ src/test/regress/expected/rules.out | 34 ++--- src/test/regress/expected/stats.out | 49 +++---- src/test/regress/sql/stats.sql | 22 ++-- doc/src/sgml/monitoring.sgml | 20 ++- 15 files changed, 531 insertions(+), 112 deletions(-) create mode 100644 src/backend/utils/activity/pgstat_index.c diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 3cb84359adf0..dc6c61e84e54 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5558,11 +5558,11 @@ proargnames => '{mcv_list,index,values,nulls,frequency,base_frequency}', prosrc => 'pg_stats_ext_mcvlist_items' }, -{ oid => '1928', descr => 'statistics: number of scans done for table/index', +{ oid => '1928', descr => 'statistics: number of scans done for table', proname => 'pg_stat_get_numscans', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_numscans' }, -{ oid => '6310', descr => 'statistics: time of the last scan for table/index', +{ oid => '6310', descr => 'statistics: time of the last scan for table', proname => 'pg_stat_get_lastscan', provolatile => 's', proparallel => 'r', prorettype => 'timestamptz', proargtypes => 'oid', prosrc => 'pg_stat_get_lastscan' }, @@ -6237,10 +6237,63 @@ proargdefaults => '{NULL}', prosrc => 'pg_stat_reset_shared', proacl => '{POSTGRES=X}' }, { oid => '3776', - descr => 'statistics: reset collected statistics for a single table or index in the current database or shared across all databases in the cluster', + descr => 'statistics: reset collected statistics for a single table in the current database or shared across all databases in the cluster', proname => 'pg_stat_reset_single_table_counters', provolatile => 'v', prorettype => 'void', proargtypes => 'oid', prosrc => 'pg_stat_reset_single_table_counters', proacl => '{POSTGRES=X}' }, +{ oid => '8450', + descr => 'statistics: reset collected statistics for a single index in the current database or shared across all databases in the cluster', + proname => 'pg_stat_reset_single_index_counters', provolatile => 'v', + prorettype => 'void', proargtypes => 'oid', + prosrc => 'pg_stat_reset_single_index_counters', proacl => '{POSTGRES=X}' }, +{ oid => '8451', descr => 'statistics: number of scans done for index', + proname => 'pg_stat_get_idx_numscans', provolatile => 's', proparallel => 'r', + prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_numscans' }, +{ oid => '8452', descr => 'statistics: number of index entries returned by index', + proname => 'pg_stat_get_idx_tuples_returned', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_tuples_returned' }, +{ oid => '8453', descr => 'statistics: number of tuples fetched from table by index', + proname => 'pg_stat_get_idx_tuples_fetched', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_tuples_fetched' }, +{ oid => '8454', descr => 'statistics: number of blocks fetched for index', + proname => 'pg_stat_get_idx_blocks_fetched', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_blocks_fetched' }, +{ oid => '8455', descr => 'statistics: number of blocks found in cache for index', + proname => 'pg_stat_get_idx_blocks_hit', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_blocks_hit' }, +{ oid => '8456', descr => 'statistics: time of the last scan for index', + proname => 'pg_stat_get_idx_lastscan', provolatile => 's', proparallel => 'r', + prorettype => 'timestamptz', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_lastscan' }, +{ oid => '8457', descr => 'statistics: last reset for an index', + proname => 'pg_stat_get_idx_stat_reset_time', provolatile => 's', + proparallel => 'r', prorettype => 'timestamptz', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_stat_reset_time' }, +{ oid => '8458', descr => 'statistics: number of scans done for index in current transaction', + proname => 'pg_stat_get_xact_idx_numscans', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_numscans' }, +{ oid => '8459', descr => 'statistics: number of index entries returned in current transaction', + proname => 'pg_stat_get_xact_idx_tuples_returned', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_tuples_returned' }, +{ oid => '8460', descr => 'statistics: number of tuples fetched by index in current transaction', + proname => 'pg_stat_get_xact_idx_tuples_fetched', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_tuples_fetched' }, +{ oid => '8461', descr => 'statistics: number of blocks fetched for index in current transaction', + proname => 'pg_stat_get_xact_idx_blocks_fetched', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_blocks_fetched' }, +{ oid => '8462', descr => 'statistics: number of blocks found in cache for index in current transaction', + proname => 'pg_stat_get_xact_idx_blocks_hit', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_blocks_hit' }, { oid => '3777', descr => 'statistics: reset collected statistics for a single function in the current database', proname => 'pg_stat_reset_single_function_counters', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 58a44857f131..23b85fc0c223 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -487,6 +487,20 @@ typedef struct PgStat_StatTabEntry TimestampTz stat_reset_time; } PgStat_StatTabEntry; +typedef struct PgStat_StatIdxEntry +{ + PgStat_Counter numscans; + TimestampTz lastscan; + + PgStat_Counter tuples_returned; + PgStat_Counter tuples_fetched; + + PgStat_Counter blocks_fetched; + PgStat_Counter blocks_hit; + + TimestampTz stat_reset_time; +} PgStat_StatIdxEntry; + /* ------ * PgStat_WalCounters WAL activity data gathered from WalUsage * @@ -779,6 +793,12 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free); extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id); +extern PgStat_TableStatus *find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id); + +extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry(Oid relid); +extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry_ext(bool shared, + Oid reloid, + bool *may_free); /* diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h index b3dc3ff7d8bb..94c94c103feb 100644 --- a/src/include/utils/pgstat_internal.h +++ b/src/include/utils/pgstat_internal.h @@ -507,6 +507,12 @@ typedef struct PgStatShared_Relation PgStat_StatTabEntry stats; } PgStatShared_Relation; +typedef struct PgStatShared_Index +{ + PgStatShared_Common header; + PgStat_StatIdxEntry stats; +} PgStatShared_Index; + typedef struct PgStatShared_Function { PgStatShared_Common header; @@ -787,6 +793,15 @@ extern void pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref); extern void pgstat_relation_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); +/* + * Functions in pgstat_index.c + */ + +extern bool pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait); +extern void pgstat_index_delete_pending_cb(PgStat_EntryRef *entry_ref); +extern void pgstat_index_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); + + /* * Functions in pgstat_replslot.c */ diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h index 2d78a0296834..45ca599d0dd6 100644 --- a/src/include/utils/pgstat_kind.h +++ b/src/include/utils/pgstat_kind.h @@ -26,19 +26,20 @@ /* stats for variable-numbered objects */ #define PGSTAT_KIND_DATABASE 1 /* database-wide statistics */ #define PGSTAT_KIND_RELATION 2 /* per-table statistics */ -#define PGSTAT_KIND_FUNCTION 3 /* per-function statistics */ -#define PGSTAT_KIND_REPLSLOT 4 /* per-slot statistics */ -#define PGSTAT_KIND_SUBSCRIPTION 5 /* per-subscription statistics */ -#define PGSTAT_KIND_BACKEND 6 /* per-backend statistics */ +#define PGSTAT_KIND_INDEX 3 /* per-index statistics */ +#define PGSTAT_KIND_FUNCTION 4 /* per-function statistics */ +#define PGSTAT_KIND_REPLSLOT 5 /* per-slot statistics */ +#define PGSTAT_KIND_SUBSCRIPTION 6 /* per-subscription statistics */ +#define PGSTAT_KIND_BACKEND 7 /* per-backend statistics */ /* stats for fixed-numbered objects */ -#define PGSTAT_KIND_ARCHIVER 7 -#define PGSTAT_KIND_BGWRITER 8 -#define PGSTAT_KIND_CHECKPOINTER 9 -#define PGSTAT_KIND_IO 10 -#define PGSTAT_KIND_LOCK 11 -#define PGSTAT_KIND_SLRU 12 -#define PGSTAT_KIND_WAL 13 +#define PGSTAT_KIND_ARCHIVER 8 +#define PGSTAT_KIND_BGWRITER 9 +#define PGSTAT_KIND_CHECKPOINTER 10 +#define PGSTAT_KIND_IO 11 +#define PGSTAT_KIND_LOCK 12 +#define PGSTAT_KIND_SLRU 13 +#define PGSTAT_KIND_WAL 14 #define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE #define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 6c1c5545cb56..ce991e4269e7 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -722,9 +722,9 @@ CREATE VIEW pg_stat_all_tables AS pg_stat_get_numscans(C.oid) AS seq_scan, pg_stat_get_lastscan(C.oid) AS last_seq_scan, pg_stat_get_tuples_returned(C.oid) AS seq_tup_read, - sum(pg_stat_get_numscans(I.indexrelid))::bigint AS idx_scan, - max(pg_stat_get_lastscan(I.indexrelid)) AS last_idx_scan, - sum(pg_stat_get_tuples_fetched(I.indexrelid))::bigint + + sum(pg_stat_get_idx_numscans(I.indexrelid))::bigint AS idx_scan, + max(pg_stat_get_idx_lastscan(I.indexrelid)) AS last_idx_scan, + sum(pg_stat_get_idx_tuples_fetched(I.indexrelid))::bigint + pg_stat_get_tuples_fetched(C.oid) AS idx_tup_fetch, pg_stat_get_tuples_inserted(C.oid) AS n_tup_ins, pg_stat_get_tuples_updated(C.oid) AS n_tup_upd, @@ -761,8 +761,8 @@ CREATE VIEW pg_stat_xact_all_tables AS C.relname AS relname, pg_stat_get_xact_numscans(C.oid) AS seq_scan, pg_stat_get_xact_tuples_returned(C.oid) AS seq_tup_read, - sum(pg_stat_get_xact_numscans(I.indexrelid))::bigint AS idx_scan, - sum(pg_stat_get_xact_tuples_fetched(I.indexrelid))::bigint + + sum(pg_stat_get_xact_idx_numscans(I.indexrelid))::bigint AS idx_scan, + sum(pg_stat_get_xact_idx_tuples_fetched(I.indexrelid))::bigint + pg_stat_get_xact_tuples_fetched(C.oid) AS idx_tup_fetch, pg_stat_get_xact_tuples_inserted(C.oid) AS n_tup_ins, pg_stat_get_xact_tuples_updated(C.oid) AS n_tup_upd, @@ -833,17 +833,17 @@ CREATE VIEW pg_statio_all_tables AS pg_class T ON C.reltoastrelid = T.oid LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) LEFT JOIN LATERAL ( - SELECT sum(pg_stat_get_blocks_fetched(indexrelid) - - pg_stat_get_blocks_hit(indexrelid))::bigint + SELECT sum(pg_stat_get_idx_blocks_fetched(indexrelid) - + pg_stat_get_idx_blocks_hit(indexrelid))::bigint AS idx_blks_read, - sum(pg_stat_get_blocks_hit(indexrelid))::bigint + sum(pg_stat_get_idx_blocks_hit(indexrelid))::bigint AS idx_blks_hit FROM pg_index WHERE indrelid = C.oid ) I ON true LEFT JOIN LATERAL ( - SELECT sum(pg_stat_get_blocks_fetched(indexrelid) - - pg_stat_get_blocks_hit(indexrelid))::bigint + SELECT sum(pg_stat_get_idx_blocks_fetched(indexrelid) - + pg_stat_get_idx_blocks_hit(indexrelid))::bigint AS idx_blks_read, - sum(pg_stat_get_blocks_hit(indexrelid))::bigint + sum(pg_stat_get_idx_blocks_hit(indexrelid))::bigint AS idx_blks_hit FROM pg_index WHERE indrelid = T.oid ) X ON true WHERE C.relkind IN ('r', 't', 'm'); @@ -865,11 +865,11 @@ CREATE VIEW pg_stat_all_indexes AS N.nspname AS schemaname, C.relname AS relname, I.relname AS indexrelname, - pg_stat_get_numscans(I.oid) AS idx_scan, - pg_stat_get_lastscan(I.oid) AS last_idx_scan, - pg_stat_get_tuples_returned(I.oid) AS idx_tup_read, - pg_stat_get_tuples_fetched(I.oid) AS idx_tup_fetch, - pg_stat_get_stat_reset_time(I.oid) AS stats_reset + pg_stat_get_idx_numscans(I.oid) AS idx_scan, + pg_stat_get_idx_lastscan(I.oid) AS last_idx_scan, + pg_stat_get_idx_tuples_returned(I.oid) AS idx_tup_read, + pg_stat_get_idx_tuples_fetched(I.oid) AS idx_tup_fetch, + pg_stat_get_idx_stat_reset_time(I.oid) AS stats_reset FROM pg_class C JOIN pg_index X ON C.oid = X.indrelid JOIN pg_class I ON I.oid = X.indexrelid @@ -893,10 +893,10 @@ CREATE VIEW pg_statio_all_indexes AS N.nspname AS schemaname, C.relname AS relname, I.relname AS indexrelname, - pg_stat_get_blocks_fetched(I.oid) - - pg_stat_get_blocks_hit(I.oid) AS idx_blks_read, - pg_stat_get_blocks_hit(I.oid) AS idx_blks_hit, - pg_stat_get_stat_reset_time(I.oid) AS stats_reset + pg_stat_get_idx_blocks_fetched(I.oid) - + pg_stat_get_idx_blocks_hit(I.oid) AS idx_blks_read, + pg_stat_get_idx_blocks_hit(I.oid) AS idx_blks_hit, + pg_stat_get_idx_stat_reset_time(I.oid) AS stats_reset FROM pg_class C JOIN pg_index X ON C.oid = X.indrelid JOIN pg_class I ON I.oid = X.indexrelid diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile index 5fed953c28a7..2e32d1485d64 100644 --- a/src/backend/utils/activity/Makefile +++ b/src/backend/utils/activity/Makefile @@ -25,6 +25,7 @@ OBJS = \ pgstat_checkpointer.o \ pgstat_database.o \ pgstat_function.o \ + pgstat_index.o \ pgstat_io.o \ pgstat_kind.o \ pgstat_lock.o \ diff --git a/src/backend/utils/activity/meson.build b/src/backend/utils/activity/meson.build index 470b5dac402b..e6dcb2e26fc5 100644 --- a/src/backend/utils/activity/meson.build +++ b/src/backend/utils/activity/meson.build @@ -10,6 +10,7 @@ backend_sources += files( 'pgstat_checkpointer.c', 'pgstat_database.c', 'pgstat_function.c', + 'pgstat_index.c', 'pgstat_io.c', 'pgstat_kind.c', 'pgstat_lock.c', diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 8b2a5ec36750..3aa10162f8ad 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -317,6 +317,22 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .reset_timestamp_cb = pgstat_relation_reset_timestamp_cb, }, + [PGSTAT_KIND_INDEX] = { + .name = "index", + + .fixed_amount = false, + .write_to_file = true, + + .shared_size = sizeof(PgStatShared_Index), + .shared_data_off = offsetof(PgStatShared_Index, stats), + .shared_data_len = sizeof(((PgStatShared_Index *) 0)->stats), + .pending_size = sizeof(PgStat_TableStatus), + + .flush_pending_cb = pgstat_index_flush_cb, + .delete_pending_cb = pgstat_index_delete_pending_cb, + .reset_timestamp_cb = pgstat_index_reset_timestamp_cb, + }, + [PGSTAT_KIND_FUNCTION] = { .name = "function", diff --git a/src/backend/utils/activity/pgstat_index.c b/src/backend/utils/activity/pgstat_index.c new file mode 100644 index 000000000000..e9fc27f2e4b4 --- /dev/null +++ b/src/backend/utils/activity/pgstat_index.c @@ -0,0 +1,129 @@ +/* ------------------------------------------------------------------------- + * + * pgstat_index.c + * Implementation of index statistics. + * + * This file contains the implementation of index statistics. + * + * Copyright (c) 2001-2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/backend/utils/activity/pgstat_index.c + * ------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "access/xact.h" +#include "catalog/catalog.h" +#include "utils/memutils.h" +#include "utils/pgstat_internal.h" +#include "utils/rel.h" +#include "utils/timestamp.h" + + +/* + * Flush out pending stats for an index entry. + * + * If nowait is true and the lock could not be immediately acquired, returns + * false without flushing the entry. Otherwise returns true. + * + * Some of the stats are copied to the corresponding pending database stats + * entry when successfully flushing. + */ +bool +pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) +{ + Oid dboid; + PgStat_TableStatus *lstats; /* pending stats entry */ + PgStatShared_Index *shidxstats; + PgStat_StatIdxEntry *idxentry; /* index entry of shared stats */ + PgStat_StatDBEntry *dbentry; /* pending database entry */ + + dboid = entry_ref->shared_entry->key.dboid; + lstats = (PgStat_TableStatus *) entry_ref->pending; + shidxstats = (PgStatShared_Index *) entry_ref->shared_stats; + + /* + * Ignore entries that didn't accumulate any actual counts, such as + * indexes that were opened by the planner but not used. + */ + if (pg_memory_is_all_zeros(&lstats->counts, + sizeof(struct PgStat_TableCounts))) + return true; + + if (!pgstat_lock_entry(entry_ref, nowait)) + return false; + + /* Add the values to the shared entry. */ + idxentry = &shidxstats->stats; + + idxentry->numscans += lstats->counts.numscans; + if (lstats->counts.numscans) + { + TimestampTz t = GetCurrentTransactionStopTimestamp(); + + if (t > idxentry->lastscan) + idxentry->lastscan = t; + } + idxentry->tuples_returned += lstats->counts.tuples_returned; + idxentry->tuples_fetched += lstats->counts.tuples_fetched; + idxentry->blocks_fetched += lstats->counts.blocks_fetched; + idxentry->blocks_hit += lstats->counts.blocks_hit; + + pgstat_unlock_entry(entry_ref); + + /* The entry was successfully flushed, add the same to database stats */ + dbentry = pgstat_prep_database_pending(dboid); + dbentry->blocks_fetched += lstats->counts.blocks_fetched; + dbentry->blocks_hit += lstats->counts.blocks_hit; + + return true; +} + +/* + * Callback to delete pending index stats. + */ +void +pgstat_index_delete_pending_cb(PgStat_EntryRef *entry_ref) +{ + PgStat_TableStatus *pending = (PgStat_TableStatus *) entry_ref->pending; + + if (pending->relation) + pgstat_unlink_relation(pending->relation); +} + +/* + * Callback to reset the timestamp on an index stats entry. + */ +void +pgstat_index_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts) +{ + ((PgStatShared_Index *) header)->stats.stat_reset_time = ts; +} + +/* + * Support function for the SQL-callable pgstat* functions. Returns + * the collected statistics for one index or NULL. NULL doesn't mean + * that the index doesn't exist, just that there are no statistics, so the + * caller is better off to report ZERO instead. + */ +PgStat_StatIdxEntry * +pgstat_fetch_stat_idxentry(Oid relid) +{ + return pgstat_fetch_stat_idxentry_ext(IsSharedRelation(relid), relid, NULL); +} + +/* + * More efficient version of pgstat_fetch_stat_idxentry(), allowing to specify + * whether the to-be-accessed index is a shared relation or not. This version + * also returns whether the caller can pfree() the result if desired. + */ +PgStat_StatIdxEntry * +pgstat_fetch_stat_idxentry_ext(bool shared, Oid reloid, bool *may_free) +{ + Oid dboid = (shared ? InvalidOid : MyDatabaseId); + + return (PgStat_StatIdxEntry *) + pgstat_fetch_entry(PGSTAT_KIND_INDEX, dboid, reloid, may_free); +} diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 04f2eb21d0bb..379a65aa28fb 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -42,12 +42,24 @@ typedef struct TwoPhasePgStatRecord } TwoPhasePgStatRecord; -static PgStat_TableStatus *pgstat_prep_relation_pending(Oid rel_id, bool isshared); +static PgStat_TableStatus *pgstat_prep_relation_pending(PgStat_Kind kind, + Oid rel_id, bool isshared); static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level); static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info); static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop); static void restore_truncdrop_counters(PgStat_TableXactStatus *trans); +/* + * Determine the stats kind for a relation based on its relkind. + */ +static inline PgStat_Kind +pgstat_get_relation_kind(char relkind) +{ + if (relkind == RELKIND_INDEX) + return PGSTAT_KIND_INDEX; + return PGSTAT_KIND_RELATION; +} + /* * Copy stats between relations. This is used for things like REINDEX @@ -56,25 +68,52 @@ static void restore_truncdrop_counters(PgStat_TableXactStatus *trans); void pgstat_copy_relation_stats(Relation dst, Relation src) { - PgStat_StatTabEntry *srcstats; - PgStatShared_Relation *dstshstats; - PgStat_EntryRef *dst_ref; + PgStat_Kind kind = pgstat_get_relation_kind(src->rd_rel->relkind); - srcstats = pgstat_fetch_stat_tabentry_ext(src->rd_rel->relisshared, - RelationGetRelid(src), - NULL); - if (!srcstats) - return; + if (kind == PGSTAT_KIND_INDEX) + { + PgStat_StatIdxEntry *srcstats; + PgStatShared_Index *dstshstats; + PgStat_EntryRef *dst_ref; - dst_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION, - dst->rd_rel->relisshared ? InvalidOid : MyDatabaseId, - RelationGetRelid(dst), - false); + srcstats = pgstat_fetch_stat_idxentry_ext(src->rd_rel->relisshared, + RelationGetRelid(src), + NULL); + if (!srcstats) + return; - dstshstats = (PgStatShared_Relation *) dst_ref->shared_stats; - dstshstats->stats = *srcstats; + dst_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_INDEX, + dst->rd_rel->relisshared ? InvalidOid : MyDatabaseId, + RelationGetRelid(dst), + false); - pgstat_unlock_entry(dst_ref); + dstshstats = (PgStatShared_Index *) dst_ref->shared_stats; + dstshstats->stats = *srcstats; + + pgstat_unlock_entry(dst_ref); + } + else + { + PgStat_StatTabEntry *srcstats; + PgStatShared_Relation *dstshstats; + PgStat_EntryRef *dst_ref; + + srcstats = pgstat_fetch_stat_tabentry_ext(src->rd_rel->relisshared, + RelationGetRelid(src), + NULL); + if (!srcstats) + return; + + dst_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION, + dst->rd_rel->relisshared ? InvalidOid : MyDatabaseId, + RelationGetRelid(dst), + false); + + dstshstats = (PgStatShared_Relation *) dst_ref->shared_stats; + dstshstats->stats = *srcstats; + + pgstat_unlock_entry(dst_ref); + } } /* @@ -131,11 +170,16 @@ pgstat_init_relation(Relation rel) void pgstat_assoc_relation(Relation rel) { + PgStat_Kind kind; + Assert(rel->pgstat_enabled); Assert(rel->pgstat_info == NULL); + kind = pgstat_get_relation_kind(rel->rd_rel->relkind); + /* find or make the PgStat_TableStatus entry, and update link */ - rel->pgstat_info = pgstat_prep_relation_pending(RelationGetRelid(rel), + rel->pgstat_info = pgstat_prep_relation_pending(kind, + RelationGetRelid(rel), rel->rd_rel->relisshared); /* don't allow link a stats to multiple relcache entries */ @@ -168,7 +212,9 @@ pgstat_unlink_relation(Relation rel) void pgstat_create_relation(Relation rel) { - pgstat_create_transactional(PGSTAT_KIND_RELATION, + PgStat_Kind kind = pgstat_get_relation_kind(rel->rd_rel->relkind); + + pgstat_create_transactional(kind, rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId, RelationGetRelid(rel)); } @@ -181,8 +227,9 @@ pgstat_drop_relation(Relation rel) { int nest_level = GetCurrentTransactionNestLevel(); PgStat_TableStatus *pgstat_info; + PgStat_Kind kind = pgstat_get_relation_kind(rel->rd_rel->relkind); - pgstat_drop_transactional(PGSTAT_KIND_RELATION, + pgstat_drop_transactional(kind, rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId, RelationGetRelid(rel)); @@ -500,16 +547,25 @@ pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free) */ PgStat_TableStatus * find_tabstat_entry(Oid rel_id) +{ + return find_tabstat_entry_kind(PGSTAT_KIND_RELATION, rel_id); +} + +/* + * Same as find_tabstat_entry but for a specific stats kind. + */ +PgStat_TableStatus * +find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id) { PgStat_EntryRef *entry_ref; PgStat_TableXactStatus *trans; PgStat_TableStatus *tabentry = NULL; PgStat_TableStatus *tablestatus = NULL; - entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, MyDatabaseId, rel_id); + entry_ref = pgstat_fetch_pending_entry(kind, MyDatabaseId, rel_id); if (!entry_ref) { - entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, InvalidOid, rel_id); + entry_ref = pgstat_fetch_pending_entry(kind, InvalidOid, rel_id); if (!entry_ref) return tablestatus; } @@ -752,7 +808,7 @@ pgstat_twophase_postcommit(FullTransactionId fxid, uint16 info, PgStat_TableStatus *pgstat_info; /* Find or create a tabstat entry for the rel */ - pgstat_info = pgstat_prep_relation_pending(rec->id, rec->shared); + pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); /* Same math as in AtEOXact_PgStat, commit case */ pgstat_info->counts.tuples_inserted += rec->tuples_inserted; @@ -788,7 +844,7 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, PgStat_TableStatus *pgstat_info; /* Find or create a tabstat entry for the rel */ - pgstat_info = pgstat_prep_relation_pending(rec->id, rec->shared); + pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); /* Same math as in AtEOXact_PgStat, abort case */ if (rec->truncdropped) @@ -922,12 +978,12 @@ pgstat_relation_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts) * initialized if not exists. */ static PgStat_TableStatus * -pgstat_prep_relation_pending(Oid rel_id, bool isshared) +pgstat_prep_relation_pending(PgStat_Kind kind, Oid rel_id, bool isshared) { PgStat_EntryRef *entry_ref; PgStat_TableStatus *pending; - entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_RELATION, + entry_ref = pgstat_prep_pending_entry(kind, isshared ? InvalidOid : MyDatabaseId, rel_id, NULL); pending = entry_ref->pending; diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 565d0e70768b..946025f39ed0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -108,6 +108,40 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated) /* pg_stat_get_vacuum_count */ PG_STAT_GET_RELENTRY_INT64(vacuum_count) +/* + * Accessor macro for index stats entries (PgStat_StatIdxEntry). + */ +#define PG_STAT_GET_IDXENTRY_INT64(stat) \ +Datum \ +CppConcat(pg_stat_get_idx_,stat)(PG_FUNCTION_ARGS) \ +{ \ + Oid relid = PG_GETARG_OID(0); \ + int64 result; \ + PgStat_StatIdxEntry *idxentry; \ + \ + if ((idxentry = pgstat_fetch_stat_idxentry(relid)) == NULL) \ + result = 0; \ + else \ + result = (int64) (idxentry->stat); \ + \ + PG_RETURN_INT64(result); \ +} + +/* pg_stat_get_idx_numscans */ +PG_STAT_GET_IDXENTRY_INT64(numscans) + +/* pg_stat_get_idx_tuples_returned */ +PG_STAT_GET_IDXENTRY_INT64(tuples_returned) + +/* pg_stat_get_idx_tuples_fetched */ +PG_STAT_GET_IDXENTRY_INT64(tuples_fetched) + +/* pg_stat_get_idx_blocks_fetched */ +PG_STAT_GET_IDXENTRY_INT64(blocks_fetched) + +/* pg_stat_get_idx_blocks_hit */ +PG_STAT_GET_IDXENTRY_INT64(blocks_hit) + #define PG_STAT_GET_RELENTRY_FLOAT8(stat) \ Datum \ CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \ @@ -173,6 +207,34 @@ PG_STAT_GET_RELENTRY_TIMESTAMPTZ(lastscan) /* pg_stat_get_stat_reset_time */ PG_STAT_GET_RELENTRY_TIMESTAMPTZ(stat_reset_time) +/* + * Accessor macro for index timestamp fields. + */ +#define PG_STAT_GET_IDXENTRY_TIMESTAMPTZ(stat) \ +Datum \ +CppConcat(pg_stat_get_idx_,stat)(PG_FUNCTION_ARGS) \ +{ \ + Oid relid = PG_GETARG_OID(0); \ + TimestampTz result; \ + PgStat_StatIdxEntry *idxentry; \ + \ + if ((idxentry = pgstat_fetch_stat_idxentry(relid)) == NULL) \ + result = 0; \ + else \ + result = idxentry->stat; \ + \ + if (result == 0) \ + PG_RETURN_NULL(); \ + else \ + PG_RETURN_TIMESTAMPTZ(result); \ +} + +/* pg_stat_get_idx_lastscan */ +PG_STAT_GET_IDXENTRY_TIMESTAMPTZ(lastscan) + +/* pg_stat_get_idx_stat_reset_time */ +PG_STAT_GET_IDXENTRY_TIMESTAMPTZ(stat_reset_time) + Datum pg_stat_get_function_calls(PG_FUNCTION_ARGS) { @@ -1906,6 +1968,41 @@ PG_STAT_GET_XACT_RELENTRY_INT64(tuples_updated) /* pg_stat_get_xact_tuples_deleted */ PG_STAT_GET_XACT_RELENTRY_INT64(tuples_deleted) +/* + * Accessor macro for in-transaction index stats. + */ +#define PG_STAT_GET_XACT_IDXENTRY_INT64(stat) \ +Datum \ +CppConcat(pg_stat_get_xact_idx_,stat)(PG_FUNCTION_ARGS) \ +{ \ + Oid relid = PG_GETARG_OID(0); \ + int64 result; \ + PgStat_TableStatus *tabentry; \ + \ + tabentry = find_tabstat_entry_kind(PGSTAT_KIND_INDEX, relid); \ + if (!tabentry) \ + result = 0; \ + else \ + result = (int64) (tabentry->counts.stat); \ + \ + PG_RETURN_INT64(result); \ +} + +/* pg_stat_get_xact_idx_numscans */ +PG_STAT_GET_XACT_IDXENTRY_INT64(numscans) + +/* pg_stat_get_xact_idx_tuples_returned */ +PG_STAT_GET_XACT_IDXENTRY_INT64(tuples_returned) + +/* pg_stat_get_xact_idx_tuples_fetched */ +PG_STAT_GET_XACT_IDXENTRY_INT64(tuples_fetched) + +/* pg_stat_get_xact_idx_blocks_fetched */ +PG_STAT_GET_XACT_IDXENTRY_INT64(blocks_fetched) + +/* pg_stat_get_xact_idx_blocks_hit */ +PG_STAT_GET_XACT_IDXENTRY_INT64(blocks_hit) + Datum pg_stat_get_xact_function_calls(PG_FUNCTION_ARGS) { @@ -2047,6 +2144,17 @@ pg_stat_reset_single_table_counters(PG_FUNCTION_ARGS) PG_RETURN_VOID(); } +Datum +pg_stat_reset_single_index_counters(PG_FUNCTION_ARGS) +{ + Oid idxoid = PG_GETARG_OID(0); + Oid dboid = (IsSharedRelation(idxoid) ? InvalidOid : MyDatabaseId); + + pgstat_reset(PGSTAT_KIND_INDEX, dboid, idxoid); + + PG_RETURN_VOID(); +} + Datum pg_stat_reset_single_function_counters(PG_FUNCTION_ARGS) { diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 39905c2de142..ffec0903f038 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1806,11 +1806,11 @@ pg_stat_all_indexes| SELECT c.oid AS relid, n.nspname AS schemaname, c.relname, i.relname AS indexrelname, - pg_stat_get_numscans(i.oid) AS idx_scan, - pg_stat_get_lastscan(i.oid) AS last_idx_scan, - pg_stat_get_tuples_returned(i.oid) AS idx_tup_read, - pg_stat_get_tuples_fetched(i.oid) AS idx_tup_fetch, - pg_stat_get_stat_reset_time(i.oid) AS stats_reset + pg_stat_get_idx_numscans(i.oid) AS idx_scan, + pg_stat_get_idx_lastscan(i.oid) AS last_idx_scan, + pg_stat_get_idx_tuples_returned(i.oid) AS idx_tup_read, + pg_stat_get_idx_tuples_fetched(i.oid) AS idx_tup_fetch, + pg_stat_get_idx_stat_reset_time(i.oid) AS stats_reset FROM (((pg_class c JOIN pg_index x ON ((c.oid = x.indrelid))) JOIN pg_class i ON ((i.oid = x.indexrelid))) @@ -1822,9 +1822,9 @@ pg_stat_all_tables| SELECT c.oid AS relid, pg_stat_get_numscans(c.oid) AS seq_scan, pg_stat_get_lastscan(c.oid) AS last_seq_scan, pg_stat_get_tuples_returned(c.oid) AS seq_tup_read, - (sum(pg_stat_get_numscans(i.indexrelid)))::bigint AS idx_scan, - max(pg_stat_get_lastscan(i.indexrelid)) AS last_idx_scan, - ((sum(pg_stat_get_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_tuples_fetched(c.oid)) AS idx_tup_fetch, + (sum(pg_stat_get_idx_numscans(i.indexrelid)))::bigint AS idx_scan, + max(pg_stat_get_idx_lastscan(i.indexrelid)) AS last_idx_scan, + ((sum(pg_stat_get_idx_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_tuples_fetched(c.oid)) AS idx_tup_fetch, pg_stat_get_tuples_inserted(c.oid) AS n_tup_ins, pg_stat_get_tuples_updated(c.oid) AS n_tup_upd, pg_stat_get_tuples_deleted(c.oid) AS n_tup_del, @@ -2452,8 +2452,8 @@ pg_stat_xact_all_tables| SELECT c.oid AS relid, c.relname, pg_stat_get_xact_numscans(c.oid) AS seq_scan, pg_stat_get_xact_tuples_returned(c.oid) AS seq_tup_read, - (sum(pg_stat_get_xact_numscans(i.indexrelid)))::bigint AS idx_scan, - ((sum(pg_stat_get_xact_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_xact_tuples_fetched(c.oid)) AS idx_tup_fetch, + (sum(pg_stat_get_xact_idx_numscans(i.indexrelid)))::bigint AS idx_scan, + ((sum(pg_stat_get_xact_idx_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_xact_tuples_fetched(c.oid)) AS idx_tup_fetch, pg_stat_get_xact_tuples_inserted(c.oid) AS n_tup_ins, pg_stat_get_xact_tuples_updated(c.oid) AS n_tup_upd, pg_stat_get_xact_tuples_deleted(c.oid) AS n_tup_del, @@ -2506,9 +2506,9 @@ pg_statio_all_indexes| SELECT c.oid AS relid, n.nspname AS schemaname, c.relname, i.relname AS indexrelname, - (pg_stat_get_blocks_fetched(i.oid) - pg_stat_get_blocks_hit(i.oid)) AS idx_blks_read, - pg_stat_get_blocks_hit(i.oid) AS idx_blks_hit, - pg_stat_get_stat_reset_time(i.oid) AS stats_reset + (pg_stat_get_idx_blocks_fetched(i.oid) - pg_stat_get_idx_blocks_hit(i.oid)) AS idx_blks_read, + pg_stat_get_idx_blocks_hit(i.oid) AS idx_blks_hit, + pg_stat_get_idx_stat_reset_time(i.oid) AS stats_reset FROM (((pg_class c JOIN pg_index x ON ((c.oid = x.indrelid))) JOIN pg_class i ON ((i.oid = x.indexrelid))) @@ -2538,12 +2538,12 @@ pg_statio_all_tables| SELECT c.oid AS relid, FROM ((((pg_class c LEFT JOIN pg_class t ON ((c.reltoastrelid = t.oid))) LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace))) - LEFT JOIN LATERAL ( SELECT (sum((pg_stat_get_blocks_fetched(pg_index.indexrelid) - pg_stat_get_blocks_hit(pg_index.indexrelid))))::bigint AS idx_blks_read, - (sum(pg_stat_get_blocks_hit(pg_index.indexrelid)))::bigint AS idx_blks_hit + LEFT JOIN LATERAL ( SELECT (sum((pg_stat_get_idx_blocks_fetched(pg_index.indexrelid) - pg_stat_get_idx_blocks_hit(pg_index.indexrelid))))::bigint AS idx_blks_read, + (sum(pg_stat_get_idx_blocks_hit(pg_index.indexrelid)))::bigint AS idx_blks_hit FROM pg_index WHERE (pg_index.indrelid = c.oid)) i ON (true)) - LEFT JOIN LATERAL ( SELECT (sum((pg_stat_get_blocks_fetched(pg_index.indexrelid) - pg_stat_get_blocks_hit(pg_index.indexrelid))))::bigint AS idx_blks_read, - (sum(pg_stat_get_blocks_hit(pg_index.indexrelid)))::bigint AS idx_blks_hit + LEFT JOIN LATERAL ( SELECT (sum((pg_stat_get_idx_blocks_fetched(pg_index.indexrelid) - pg_stat_get_idx_blocks_hit(pg_index.indexrelid))))::bigint AS idx_blks_read, + (sum(pg_stat_get_idx_blocks_hit(pg_index.indexrelid)))::bigint AS idx_blks_hit FROM pg_index WHERE (pg_index.indrelid = t.oid)) x ON (true)) WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"])); diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index 03cbc1cdef59..b44e30f557c9 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -123,18 +123,19 @@ SELECT id, name, fixed_amount, ----+--------------+--------------+-----------+--------------- 1 | database | f | t | t 2 | relation | f | f | t - 3 | function | f | f | t - 4 | replslot | f | t | t - 5 | subscription | f | t | t - 6 | backend | f | t | f - 7 | archiver | t | f | t - 8 | bgwriter | t | f | t - 9 | checkpointer | t | f | t - 10 | io | t | f | t - 11 | lock | t | f | t - 12 | slru | t | f | t - 13 | wal | t | f | t -(13 rows) + 3 | index | f | f | t + 4 | function | f | f | t + 5 | replslot | f | t | t + 6 | subscription | f | t | t + 7 | backend | f | t | f + 8 | archiver | t | f | t + 9 | bgwriter | t | f | t + 10 | checkpointer | t | f | t + 11 | io | t | f | t + 12 | lock | t | f | t + 13 | slru | t | f | t + 14 | wal | t | f | t +(14 rows) -- ensure that both seqscan and indexscan plans are allowed SET enable_seqscan TO on; @@ -907,8 +908,8 @@ SELECT idx_scan, :'test_last_idx' < last_idx_scan AS idx_ok, (1 row) -- check that the stats in pg_stat_all_indexes are reset -SELECT pg_stat_reset_single_table_counters('test_last_scan_pkey'::regclass); - pg_stat_reset_single_table_counters +SELECT pg_stat_reset_single_index_counters('test_last_scan_pkey'::regclass); + pg_stat_reset_single_index_counters ------------------------------------- (1 row) @@ -1392,21 +1393,21 @@ select a from stats_test_tab1 where a = 3; 3 (1 row) -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t (1 row) -- pg_stat_have_stats returns false for dropped index with stats -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t (1 row) DROP index stats_test_idx1; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- f @@ -1422,14 +1423,14 @@ select a from stats_test_tab1 where a = 3; 3 (1 row) -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t (1 row) ROLLBACK; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- f @@ -1444,7 +1445,7 @@ select a from stats_test_tab1 where a = 3; 3 (1 row) -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t @@ -1452,7 +1453,7 @@ SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); REINDEX index CONCURRENTLY stats_test_idx1; -- false for previous oid -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- f @@ -1460,7 +1461,7 @@ SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); -- true for new oid SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t @@ -1468,7 +1469,7 @@ SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns true for a rolled back drop index with stats BEGIN; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t @@ -1476,7 +1477,7 @@ SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); DROP index stats_test_idx1; ROLLBACK; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 4c265d1245c7..d4623c32cd32 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -398,7 +398,7 @@ SELECT idx_scan, :'test_last_idx' < last_idx_scan AS idx_ok, FROM pg_stat_all_indexes WHERE indexrelid = 'test_last_scan_pkey'::regclass; -- check that the stats in pg_stat_all_indexes are reset -SELECT pg_stat_reset_single_table_counters('test_last_scan_pkey'::regclass); +SELECT pg_stat_reset_single_index_counters('test_last_scan_pkey'::regclass); SELECT idx_scan, stats_reset IS NOT NULL AS has_stats_reset FROM pg_stat_all_indexes WHERE indexrelid = 'test_last_scan_pkey'::regclass; @@ -619,40 +619,40 @@ CREATE index stats_test_idx1 on stats_test_tab1(a); SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset SET enable_seqscan TO off; select a from stats_test_tab1 where a = 3; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns false for dropped index with stats -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); DROP index stats_test_idx1; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns false for rolled back index creation BEGIN; CREATE index stats_test_idx1 on stats_test_tab1(a); SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset select a from stats_test_tab1 where a = 3; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); ROLLBACK; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns true for reindex CONCURRENTLY CREATE index stats_test_idx1 on stats_test_tab1(a); SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset select a from stats_test_tab1 where a = 3; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); REINDEX index CONCURRENTLY stats_test_idx1; -- false for previous oid -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- true for new oid SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns true for a rolled back drop index with stats BEGIN; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); DROP index stats_test_idx1; ROLLBACK; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- put enable_seqscan back to on SET enable_seqscan TO on; diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 858788b227c4..f3024aa16bc1 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -5867,7 +5867,7 @@ description | Waiting for a newly initialized WAL file to reach durable storage <returnvalue>void</returnvalue> </para> <para> - Resets statistics for a single table or index in the current database + Resets statistics for a single table in the current database or shared across all databases in the cluster to zero. It also resets statistics for a single sequence or materialized view in the current database. @@ -5878,6 +5878,24 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_stat_reset_single_index_counters</primary> + </indexterm> + <function>pg_stat_reset_single_index_counters</function> ( <type>oid</type> ) + <returnvalue>void</returnvalue> + </para> + <para> + Resets statistics for a single index in the current database + or shared across all databases in the cluster to zero. + </para> + <para> + This function is restricted to superusers by default, but other users + can be granted EXECUTE to run the function. + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <indexterm> -- 2.55.0 From 5f8a69eb6e08a1c95ca9ba67e72324daf470075b Mon Sep 17 00:00:00 2001 From: Michael Paquier <[email protected]> Date: Fri, 10 Jul 2026 12:19:56 +0900 Subject: [PATCH v13 2/3] Refactor PgStat_TableStatus to new PgStat_RelationStatus This new structure is split depending on the relkind it deals with: - PGSTAT_KIND_RELATION, for tables. - PGSTAT_KIND_INDEX, for indexes. This change makes the barrier cleaner between the handling of tables and indexes, by being able to track precisely what are the counters used by one or the other for pending data. Using a common ground for both eases the tracking of Relations in the relcache. --- src/include/pgstat.h | 110 +++++-- src/include/utils/pgstat_internal.h | 4 +- src/include/utils/rel.h | 2 +- src/backend/utils/activity/pgstat.c | 4 +- src/backend/utils/activity/pgstat_index.c | 26 +- src/backend/utils/activity/pgstat_relation.c | 296 ++++++++++--------- src/backend/utils/adt/pgstatfuncs.c | 13 +- src/backend/utils/cache/relcache.c | 2 +- src/tools/pgindent/typedefs.list | 2 +- 9 files changed, 257 insertions(+), 202 deletions(-) diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 23b85fc0c223..892cdf24139b 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -125,14 +125,12 @@ typedef struct PgStat_BackendSubEntry * of pg_memory_is_all_zeros() to detect whether there are any stats updates * to apply. * - * It is a component of PgStat_TableStatus (within-backend state). + * It is a component of PgStat_RelationStatus (within-backend state, for + * table data). * - * Note: for a table, tuples_returned is the number of tuples successfully - * fetched by heap_getnext, while tuples_fetched is the number of tuples - * successfully fetched by heap_fetch under the control of bitmap indexscans. - * For an index, tuples_returned is the number of index entries returned by - * the index AM, while tuples_fetched is the number of tuples successfully - * fetched by heap_fetch under the control of simple indexscans for this index. + * Note: tuples_returned is the number of tuples successfully fetched by + * heap_getnext, while tuples_fetched is the number of tuples successfully + * fetched by heap_fetch under the control of bitmap indexscans. * * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted * actions, regardless of whether the transaction committed. delta_live_tuples, @@ -163,34 +161,69 @@ typedef struct PgStat_TableCounts } PgStat_TableCounts; /* ---------- - * PgStat_TableStatus Per-table status within a backend + * PgStat_IndexCounts Per-index pending event counters + * + * Note: tuples_returned is the number of index entries returned by + * the index AM, while tuples_fetched is the number of tuples successfully + * fetched by heap_fetch under the control of simple indexscans for this + * index. + * + * It is a component of PgStat_RelationStatus (within-backend state, for + * index data). + * ---------- + */ +typedef struct PgStat_IndexCounts +{ + PgStat_Counter numscans; + PgStat_Counter tuples_returned; + PgStat_Counter tuples_fetched; + PgStat_Counter blocks_fetched; + PgStat_Counter blocks_hit; +} PgStat_IndexCounts; + +/* ---------- + * PgStat_RelationStatus Per-relation pending status within a backend * * Many of the event counters are nontransactional, ie, we count events * in committed and aborted transactions alike. For these, we just count - * directly in the PgStat_TableStatus. However, delta_live_tuples, + * directly in the PgStat_RelationStatus. However, delta_live_tuples, * delta_dead_tuples, and changed_tuples must be derived from event counts * with awareness of whether the transaction or subtransaction committed or * aborted. Hence, we also keep a stack of per-(sub)transaction status * records for every table modified in the current transaction. At commit * or abort, we propagate tuples_inserted/updated/deleted up to the - * parent subtransaction level, or out to the parent PgStat_TableStatus, + * parent subtransaction level, or out to the parent PgStat_RelationStatus, * as appropriate. + * + * 'kind' tracks the stats kind we are dealing with, for table or index + * pending data. * ---------- */ -typedef struct PgStat_TableStatus +typedef struct PgStat_RelationStatus { - Oid id; /* table's OID */ - bool shared; /* is it a shared catalog? */ - struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */ - PgStat_TableCounts counts; /* event counts to be sent */ + PgStat_Kind kind; /* PGSTAT_KIND_RELATION or PGSTAT_KIND_INDEX */ Relation relation; /* rel that is using this entry */ -} PgStat_TableStatus; + union + { + /* table counters */ + struct + { + Oid id; /* table's OID */ + bool shared; /* is it a shared catalog? */ + struct PgStat_RelXactStatus *trans; /* lowest subxact's counts */ + PgStat_TableCounts counts; /* event counts to be sent */ + } tab; + + /* index counters */ + PgStat_IndexCounts idx; + }; +} PgStat_RelationStatus; /* ---------- - * PgStat_TableXactStatus Per-table, per-subtransaction status + * PgStat_RelXactStatus Per-relation, per-subtransaction status * ---------- */ -typedef struct PgStat_TableXactStatus +typedef struct PgStat_RelXactStatus { PgStat_Counter tuples_inserted; /* tuples inserted in (sub)xact */ PgStat_Counter tuples_updated; /* tuples updated in (sub)xact */ @@ -203,11 +236,11 @@ typedef struct PgStat_TableXactStatus PgStat_Counter deleted_pre_truncdrop; int nest_level; /* subtransaction nest level */ /* links to other structs for same relation: */ - struct PgStat_TableXactStatus *upper; /* next higher subxact if any */ - PgStat_TableStatus *parent; /* per-table status */ + struct PgStat_RelXactStatus *upper; /* next higher subxact if any */ + PgStat_RelationStatus *parent; /* per-table status */ /* structs of same subxact level are linked here: */ - struct PgStat_TableXactStatus *next; /* next of same subxact */ -} PgStat_TableXactStatus; + struct PgStat_RelXactStatus *next; /* next of same subxact */ +} PgStat_RelXactStatus; /* ------------------------------------------------------------ @@ -744,37 +777,52 @@ extern void pgstat_report_analyze(Relation rel, #define pgstat_count_heap_scan(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.numscans++; \ + (rel)->pgstat_info->tab.counts.numscans++; \ } while (0) #define pgstat_count_heap_getnext(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.tuples_returned++; \ + (rel)->pgstat_info->tab.counts.tuples_returned++; \ } while (0) #define pgstat_count_heap_fetch(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.tuples_fetched++; \ + { \ + if ((rel)->pgstat_info->kind == PGSTAT_KIND_INDEX) \ + (rel)->pgstat_info->idx.tuples_fetched++; \ + else \ + (rel)->pgstat_info->tab.counts.tuples_fetched++; \ + } \ } while (0) #define pgstat_count_index_scan(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.numscans++; \ + (rel)->pgstat_info->idx.numscans++; \ } while (0) #define pgstat_count_index_tuples(rel, n) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.tuples_returned += (n); \ + (rel)->pgstat_info->idx.tuples_returned += (n); \ } while (0) #define pgstat_count_buffer_read(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.blocks_fetched++; \ + { \ + if ((rel)->pgstat_info->kind == PGSTAT_KIND_INDEX) \ + (rel)->pgstat_info->idx.blocks_fetched++; \ + else \ + (rel)->pgstat_info->tab.counts.blocks_fetched++; \ + } \ } while (0) #define pgstat_count_buffer_hit(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.blocks_hit++; \ + { \ + if ((rel)->pgstat_info->kind == PGSTAT_KIND_INDEX) \ + (rel)->pgstat_info->idx.blocks_hit++; \ + else \ + (rel)->pgstat_info->tab.counts.blocks_hit++; \ + } \ } while (0) extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n); @@ -792,8 +840,8 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid); extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free); -extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id); -extern PgStat_TableStatus *find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id); +extern PgStat_RelationStatus *find_relstat_entry_kind(PgStat_Kind kind, + Oid rel_id); extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry(Oid relid); extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry_ext(bool shared, diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h index 94c94c103feb..4df3082f2b6e 100644 --- a/src/include/utils/pgstat_internal.h +++ b/src/include/utils/pgstat_internal.h @@ -215,13 +215,13 @@ typedef struct PgStat_SubXactStatus /* * Tuple insertion/deletion counts for an open transaction can't be - * propagated into PgStat_TableStatus counters until we know if it is + * propagated into PgStat_RelationStatus counters until we know if it is * going to commit or abort. Hence, we keep these counts in per-subxact * structs that live in TopTransactionContext. This data structure is * designed on the assumption that subxacts won't usually modify very many * tables. */ - PgStat_TableXactStatus *first; /* head of list for this subxact */ + PgStat_RelXactStatus *first; /* head of list for this subxact */ } PgStat_SubXactStatus; diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 89c159b133fa..6d3423b2ad47 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -252,7 +252,7 @@ typedef struct RelationData bool pgstat_enabled; /* should relation stats be counted */ /* use "struct" here to avoid needing to include pgstat.h: */ - struct PgStat_TableStatus *pgstat_info; /* statistics collection area */ + struct PgStat_RelationStatus *pgstat_info; /* statistics collection area */ } RelationData; diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 3aa10162f8ad..907d1f53a7ec 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -310,7 +310,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .shared_size = sizeof(PgStatShared_Relation), .shared_data_off = offsetof(PgStatShared_Relation, stats), .shared_data_len = sizeof(((PgStatShared_Relation *) 0)->stats), - .pending_size = sizeof(PgStat_TableStatus), + .pending_size = sizeof(PgStat_RelationStatus), .flush_pending_cb = pgstat_relation_flush_cb, .delete_pending_cb = pgstat_relation_delete_pending_cb, @@ -326,7 +326,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .shared_size = sizeof(PgStatShared_Index), .shared_data_off = offsetof(PgStatShared_Index, stats), .shared_data_len = sizeof(((PgStatShared_Index *) 0)->stats), - .pending_size = sizeof(PgStat_TableStatus), + .pending_size = sizeof(PgStat_RelationStatus), .flush_pending_cb = pgstat_index_flush_cb, .delete_pending_cb = pgstat_index_delete_pending_cb, diff --git a/src/backend/utils/activity/pgstat_index.c b/src/backend/utils/activity/pgstat_index.c index e9fc27f2e4b4..5d1746614368 100644 --- a/src/backend/utils/activity/pgstat_index.c +++ b/src/backend/utils/activity/pgstat_index.c @@ -35,21 +35,21 @@ bool pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) { Oid dboid; - PgStat_TableStatus *lstats; /* pending stats entry */ + PgStat_RelationStatus *lstats; /* pending stats entry */ PgStatShared_Index *shidxstats; PgStat_StatIdxEntry *idxentry; /* index entry of shared stats */ PgStat_StatDBEntry *dbentry; /* pending database entry */ dboid = entry_ref->shared_entry->key.dboid; - lstats = (PgStat_TableStatus *) entry_ref->pending; + lstats = (PgStat_RelationStatus *) entry_ref->pending; shidxstats = (PgStatShared_Index *) entry_ref->shared_stats; /* * Ignore entries that didn't accumulate any actual counts, such as * indexes that were opened by the planner but not used. */ - if (pg_memory_is_all_zeros(&lstats->counts, - sizeof(struct PgStat_TableCounts))) + if (pg_memory_is_all_zeros(&lstats->idx, + sizeof(PgStat_IndexCounts))) return true; if (!pgstat_lock_entry(entry_ref, nowait)) @@ -58,25 +58,25 @@ pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) /* Add the values to the shared entry. */ idxentry = &shidxstats->stats; - idxentry->numscans += lstats->counts.numscans; - if (lstats->counts.numscans) + idxentry->numscans += lstats->idx.numscans; + if (lstats->idx.numscans) { TimestampTz t = GetCurrentTransactionStopTimestamp(); if (t > idxentry->lastscan) idxentry->lastscan = t; } - idxentry->tuples_returned += lstats->counts.tuples_returned; - idxentry->tuples_fetched += lstats->counts.tuples_fetched; - idxentry->blocks_fetched += lstats->counts.blocks_fetched; - idxentry->blocks_hit += lstats->counts.blocks_hit; + idxentry->tuples_returned += lstats->idx.tuples_returned; + idxentry->tuples_fetched += lstats->idx.tuples_fetched; + idxentry->blocks_fetched += lstats->idx.blocks_fetched; + idxentry->blocks_hit += lstats->idx.blocks_hit; pgstat_unlock_entry(entry_ref); /* The entry was successfully flushed, add the same to database stats */ dbentry = pgstat_prep_database_pending(dboid); - dbentry->blocks_fetched += lstats->counts.blocks_fetched; - dbentry->blocks_hit += lstats->counts.blocks_hit; + dbentry->blocks_fetched += lstats->idx.blocks_fetched; + dbentry->blocks_hit += lstats->idx.blocks_hit; return true; } @@ -87,7 +87,7 @@ pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) void pgstat_index_delete_pending_cb(PgStat_EntryRef *entry_ref) { - PgStat_TableStatus *pending = (PgStat_TableStatus *) entry_ref->pending; + PgStat_RelationStatus *pending = (PgStat_RelationStatus *) entry_ref->pending; if (pending->relation) pgstat_unlink_relation(pending->relation); diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 379a65aa28fb..0cfa5be49cde 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -42,12 +42,12 @@ typedef struct TwoPhasePgStatRecord } TwoPhasePgStatRecord; -static PgStat_TableStatus *pgstat_prep_relation_pending(PgStat_Kind kind, +static PgStat_RelationStatus *pgstat_prep_relation_pending(PgStat_Kind kind, Oid rel_id, bool isshared); -static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level); -static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info); -static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop); -static void restore_truncdrop_counters(PgStat_TableXactStatus *trans); +static void add_tabstat_xact_level(PgStat_RelationStatus *pgstat_info, int nest_level); +static void ensure_tabstat_xact_level(PgStat_RelationStatus *pgstat_info); +static void save_truncdrop_counters(PgStat_RelXactStatus *trans, bool is_drop); +static void restore_truncdrop_counters(PgStat_RelXactStatus *trans); /* * Determine the stats kind for a relation based on its relkind. @@ -177,7 +177,7 @@ pgstat_assoc_relation(Relation rel) kind = pgstat_get_relation_kind(rel->rd_rel->relkind); - /* find or make the PgStat_TableStatus entry, and update link */ + /* find or make the PgStat_RelationStatus entry, and update link */ rel->pgstat_info = pgstat_prep_relation_pending(kind, RelationGetRelid(rel), rel->rd_rel->relisshared); @@ -226,7 +226,7 @@ void pgstat_drop_relation(Relation rel) { int nest_level = GetCurrentTransactionNestLevel(); - PgStat_TableStatus *pgstat_info; + PgStat_RelationStatus *pgstat_info; PgStat_Kind kind = pgstat_get_relation_kind(rel->rd_rel->relkind); pgstat_drop_transactional(kind, @@ -239,15 +239,20 @@ pgstat_drop_relation(Relation rel) /* * Transactionally set counters to 0. That ensures that accesses to * pg_stat_xact_all_tables inside the transaction show 0. + * + * Indexes have no transactional counters, so leave. */ pgstat_info = rel->pgstat_info; - if (pgstat_info->trans && - pgstat_info->trans->nest_level == nest_level) + if (pgstat_info->kind == PGSTAT_KIND_INDEX) + return; + + if (pgstat_info->tab.trans && + pgstat_info->tab.trans->nest_level == nest_level) { - save_truncdrop_counters(pgstat_info->trans, true); - pgstat_info->trans->tuples_inserted = 0; - pgstat_info->trans->tuples_updated = 0; - pgstat_info->trans->tuples_deleted = 0; + save_truncdrop_counters(pgstat_info->tab.trans, true); + pgstat_info->tab.trans->tuples_inserted = 0; + pgstat_info->tab.trans->tuples_updated = 0; + pgstat_info->tab.trans->tuples_deleted = 0; } } @@ -355,15 +360,15 @@ pgstat_report_analyze(Relation rel, if (pgstat_should_count_relation(rel) && rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE) { - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; - for (trans = rel->pgstat_info->trans; trans; trans = trans->upper) + for (trans = rel->pgstat_info->tab.trans; trans; trans = trans->upper) { livetuples -= trans->tuples_inserted - trans->tuples_deleted; deadtuples -= trans->tuples_updated + trans->tuples_deleted; } /* count stuff inserted by already-aborted subxacts, too */ - deadtuples -= rel->pgstat_info->counts.delta_dead_tuples; + deadtuples -= rel->pgstat_info->tab.counts.delta_dead_tuples; /* Since ANALYZE's counts are estimates, we could have underflowed */ livetuples = Max(livetuples, 0); deadtuples = Max(deadtuples, 0); @@ -422,10 +427,10 @@ pgstat_count_heap_insert(Relation rel, PgStat_Counter n) { if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; ensure_tabstat_xact_level(pgstat_info); - pgstat_info->trans->tuples_inserted += n; + pgstat_info->tab.trans->tuples_inserted += n; } } @@ -439,19 +444,19 @@ pgstat_count_heap_update(Relation rel, bool hot, bool newpage) if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; ensure_tabstat_xact_level(pgstat_info); - pgstat_info->trans->tuples_updated++; + pgstat_info->tab.trans->tuples_updated++; /* * tuples_hot_updated and tuples_newpage_updated counters are * nontransactional, so just advance them */ if (hot) - pgstat_info->counts.tuples_hot_updated++; + pgstat_info->tab.counts.tuples_hot_updated++; else if (newpage) - pgstat_info->counts.tuples_newpage_updated++; + pgstat_info->tab.counts.tuples_newpage_updated++; } } @@ -463,10 +468,10 @@ pgstat_count_heap_delete(Relation rel) { if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; ensure_tabstat_xact_level(pgstat_info); - pgstat_info->trans->tuples_deleted++; + pgstat_info->tab.trans->tuples_deleted++; } } @@ -478,13 +483,13 @@ pgstat_count_truncate(Relation rel) { if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; ensure_tabstat_xact_level(pgstat_info); - save_truncdrop_counters(pgstat_info->trans, false); - pgstat_info->trans->tuples_inserted = 0; - pgstat_info->trans->tuples_updated = 0; - pgstat_info->trans->tuples_deleted = 0; + save_truncdrop_counters(pgstat_info->tab.trans, false); + pgstat_info->tab.trans->tuples_inserted = 0; + pgstat_info->tab.trans->tuples_updated = 0; + pgstat_info->tab.trans->tuples_deleted = 0; } } @@ -501,9 +506,9 @@ pgstat_update_heap_dead_tuples(Relation rel, int delta) { if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; - pgstat_info->counts.delta_dead_tuples -= delta; + pgstat_info->tab.counts.delta_dead_tuples -= delta; } } @@ -534,9 +539,9 @@ pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free) } /* - * find any existing PgStat_TableStatus entry for rel + * find any existing PgStat_RelationStatus entry for rel and kind * - * Find any existing PgStat_TableStatus entry for rel_id in the current + * Find any existing PgStat_RelationStatus entry for rel_id in the current * database. If not found, try finding from shared tables. * * If an entry is found, copy it and increment the copy's counters with their @@ -545,22 +550,13 @@ pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free) * * If no entry found, return NULL, don't create a new one. */ -PgStat_TableStatus * -find_tabstat_entry(Oid rel_id) -{ - return find_tabstat_entry_kind(PGSTAT_KIND_RELATION, rel_id); -} - -/* - * Same as find_tabstat_entry but for a specific stats kind. - */ -PgStat_TableStatus * -find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id) +PgStat_RelationStatus * +find_relstat_entry_kind(PgStat_Kind kind, Oid rel_id) { PgStat_EntryRef *entry_ref; - PgStat_TableXactStatus *trans; - PgStat_TableStatus *tabentry = NULL; - PgStat_TableStatus *tablestatus = NULL; + PgStat_RelXactStatus *trans; + PgStat_RelationStatus *tabentry = NULL; + PgStat_RelationStatus *tablestatus = NULL; entry_ref = pgstat_fetch_pending_entry(kind, MyDatabaseId, rel_id); if (!entry_ref) @@ -570,27 +566,33 @@ find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id) return tablestatus; } - tabentry = (PgStat_TableStatus *) entry_ref->pending; - tablestatus = palloc_object(PgStat_TableStatus); + tabentry = (PgStat_RelationStatus *) entry_ref->pending; + tablestatus = palloc_object(PgStat_RelationStatus); *tablestatus = *tabentry; /* - * Reset tablestatus->trans in the copy of PgStat_TableStatus as it may + * For index entries, just return the copy — no transactional data. + */ + if (kind == PGSTAT_KIND_INDEX) + return tablestatus; + + /* + * Reset tablestatus->trans in the copy of PgStat_RelationStatus as it may * point to a shared memory area. Its data is saved below, so removing it * does not matter. */ - tablestatus->trans = NULL; + tablestatus->tab.trans = NULL; /* * Live subtransaction counts are not included yet. This is not a hot * code path so reconcile tuples_inserted, tuples_updated and * tuples_deleted even if the caller may not be interested in this data. */ - for (trans = tabentry->trans; trans != NULL; trans = trans->upper) + for (trans = tabentry->tab.trans; trans != NULL; trans = trans->upper) { - tablestatus->counts.tuples_inserted += trans->tuples_inserted; - tablestatus->counts.tuples_updated += trans->tuples_updated; - tablestatus->counts.tuples_deleted += trans->tuples_deleted; + tablestatus->tab.counts.tuples_inserted += trans->tuples_inserted; + tablestatus->tab.counts.tuples_updated += trans->tuples_updated; + tablestatus->tab.counts.tuples_deleted += trans->tuples_deleted; } return tablestatus; @@ -607,51 +609,51 @@ find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id) void AtEOXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit) { - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; for (trans = xact_state->first; trans != NULL; trans = trans->next) { - PgStat_TableStatus *tabstat; + PgStat_RelationStatus *tabstat; Assert(trans->nest_level == 1); Assert(trans->upper == NULL); tabstat = trans->parent; - Assert(tabstat->trans == trans); + Assert(tabstat->tab.trans == trans); /* restore pre-truncate/drop stats (if any) in case of aborted xact */ if (!isCommit) restore_truncdrop_counters(trans); /* count attempted actions regardless of commit/abort */ - tabstat->counts.tuples_inserted += trans->tuples_inserted; - tabstat->counts.tuples_updated += trans->tuples_updated; - tabstat->counts.tuples_deleted += trans->tuples_deleted; + tabstat->tab.counts.tuples_inserted += trans->tuples_inserted; + tabstat->tab.counts.tuples_updated += trans->tuples_updated; + tabstat->tab.counts.tuples_deleted += trans->tuples_deleted; if (isCommit) { - tabstat->counts.truncdropped = trans->truncdropped; + tabstat->tab.counts.truncdropped = trans->truncdropped; if (trans->truncdropped) { /* forget live/dead stats seen by backend thus far */ - tabstat->counts.delta_live_tuples = 0; - tabstat->counts.delta_dead_tuples = 0; + tabstat->tab.counts.delta_live_tuples = 0; + tabstat->tab.counts.delta_dead_tuples = 0; } /* insert adds a live tuple, delete removes one */ - tabstat->counts.delta_live_tuples += + tabstat->tab.counts.delta_live_tuples += trans->tuples_inserted - trans->tuples_deleted; /* update and delete each create a dead tuple */ - tabstat->counts.delta_dead_tuples += + tabstat->tab.counts.delta_dead_tuples += trans->tuples_updated + trans->tuples_deleted; /* insert, update, delete each count as one change event */ - tabstat->counts.changed_tuples += + tabstat->tab.counts.changed_tuples += trans->tuples_inserted + trans->tuples_updated + trans->tuples_deleted; } else { /* inserted tuples are dead, deleted tuples are unaffected */ - tabstat->counts.delta_dead_tuples += + tabstat->tab.counts.delta_dead_tuples += trans->tuples_inserted + trans->tuples_updated; /* an aborted xact generates no changed_tuple events */ } - tabstat->trans = NULL; + tabstat->tab.trans = NULL; } } @@ -665,17 +667,17 @@ AtEOXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit) void AtEOSubXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit, int nestDepth) { - PgStat_TableXactStatus *trans; - PgStat_TableXactStatus *next_trans; + PgStat_RelXactStatus *trans; + PgStat_RelXactStatus *next_trans; for (trans = xact_state->first; trans != NULL; trans = next_trans) { - PgStat_TableStatus *tabstat; + PgStat_RelationStatus *tabstat; next_trans = trans->next; Assert(trans->nest_level == nestDepth); tabstat = trans->parent; - Assert(tabstat->trans == trans); + Assert(tabstat->tab.trans == trans); if (isCommit) { @@ -696,7 +698,7 @@ AtEOSubXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit, in trans->upper->tuples_updated += trans->tuples_updated; trans->upper->tuples_deleted += trans->tuples_deleted; } - tabstat->trans = trans->upper; + tabstat->tab.trans = trans->upper; pfree(trans); } else @@ -727,13 +729,13 @@ AtEOSubXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit, in /* first restore values obliterated by truncate/drop */ restore_truncdrop_counters(trans); /* count attempted actions regardless of commit/abort */ - tabstat->counts.tuples_inserted += trans->tuples_inserted; - tabstat->counts.tuples_updated += trans->tuples_updated; - tabstat->counts.tuples_deleted += trans->tuples_deleted; + tabstat->tab.counts.tuples_inserted += trans->tuples_inserted; + tabstat->tab.counts.tuples_updated += trans->tuples_updated; + tabstat->tab.counts.tuples_deleted += trans->tuples_deleted; /* inserted tuples are dead, deleted tuples are unaffected */ - tabstat->counts.delta_dead_tuples += + tabstat->tab.counts.delta_dead_tuples += trans->tuples_inserted + trans->tuples_updated; - tabstat->trans = trans->upper; + tabstat->tab.trans = trans->upper; pfree(trans); } } @@ -746,17 +748,17 @@ AtEOSubXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit, in void AtPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) { - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; for (trans = xact_state->first; trans != NULL; trans = trans->next) { - PgStat_TableStatus *tabstat PG_USED_FOR_ASSERTS_ONLY; + PgStat_RelationStatus *tabstat PG_USED_FOR_ASSERTS_ONLY; TwoPhasePgStatRecord record; Assert(trans->nest_level == 1); Assert(trans->upper == NULL); tabstat = trans->parent; - Assert(tabstat->trans == trans); + Assert(tabstat->tab.trans == trans); record.tuples_inserted = trans->tuples_inserted; record.tuples_updated = trans->tuples_updated; @@ -764,8 +766,8 @@ AtPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) record.inserted_pre_truncdrop = trans->inserted_pre_truncdrop; record.updated_pre_truncdrop = trans->updated_pre_truncdrop; record.deleted_pre_truncdrop = trans->deleted_pre_truncdrop; - record.id = tabstat->id; - record.shared = tabstat->shared; + record.id = tabstat->tab.id; + record.shared = tabstat->tab.shared; record.truncdropped = trans->truncdropped; RegisterTwoPhaseRecord(TWOPHASE_RM_PGSTAT_ID, 0, @@ -784,14 +786,14 @@ AtPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) void PostPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) { - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; for (trans = xact_state->first; trans != NULL; trans = trans->next) { - PgStat_TableStatus *tabstat; + PgStat_RelationStatus *tabstat; tabstat = trans->parent; - tabstat->trans = NULL; + tabstat->tab.trans = NULL; } } @@ -805,27 +807,27 @@ pgstat_twophase_postcommit(FullTransactionId fxid, uint16 info, void *recdata, uint32 len) { TwoPhasePgStatRecord *rec = (TwoPhasePgStatRecord *) recdata; - PgStat_TableStatus *pgstat_info; + PgStat_RelationStatus *pgstat_info; /* Find or create a tabstat entry for the rel */ pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); /* Same math as in AtEOXact_PgStat, commit case */ - pgstat_info->counts.tuples_inserted += rec->tuples_inserted; - pgstat_info->counts.tuples_updated += rec->tuples_updated; - pgstat_info->counts.tuples_deleted += rec->tuples_deleted; - pgstat_info->counts.truncdropped = rec->truncdropped; + pgstat_info->tab.counts.tuples_inserted += rec->tuples_inserted; + pgstat_info->tab.counts.tuples_updated += rec->tuples_updated; + pgstat_info->tab.counts.tuples_deleted += rec->tuples_deleted; + pgstat_info->tab.counts.truncdropped = rec->truncdropped; if (rec->truncdropped) { /* forget live/dead stats seen by backend thus far */ - pgstat_info->counts.delta_live_tuples = 0; - pgstat_info->counts.delta_dead_tuples = 0; + pgstat_info->tab.counts.delta_live_tuples = 0; + pgstat_info->tab.counts.delta_dead_tuples = 0; } - pgstat_info->counts.delta_live_tuples += + pgstat_info->tab.counts.delta_live_tuples += rec->tuples_inserted - rec->tuples_deleted; - pgstat_info->counts.delta_dead_tuples += + pgstat_info->tab.counts.delta_dead_tuples += rec->tuples_updated + rec->tuples_deleted; - pgstat_info->counts.changed_tuples += + pgstat_info->tab.counts.changed_tuples += rec->tuples_inserted + rec->tuples_updated + rec->tuples_deleted; } @@ -841,7 +843,7 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, void *recdata, uint32 len) { TwoPhasePgStatRecord *rec = (TwoPhasePgStatRecord *) recdata; - PgStat_TableStatus *pgstat_info; + PgStat_RelationStatus *pgstat_info; /* Find or create a tabstat entry for the rel */ pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); @@ -853,10 +855,10 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, rec->tuples_updated = rec->updated_pre_truncdrop; rec->tuples_deleted = rec->deleted_pre_truncdrop; } - pgstat_info->counts.tuples_inserted += rec->tuples_inserted; - pgstat_info->counts.tuples_updated += rec->tuples_updated; - pgstat_info->counts.tuples_deleted += rec->tuples_deleted; - pgstat_info->counts.delta_dead_tuples += + pgstat_info->tab.counts.tuples_inserted += rec->tuples_inserted; + pgstat_info->tab.counts.tuples_updated += rec->tuples_updated; + pgstat_info->tab.counts.tuples_deleted += rec->tuples_deleted; + pgstat_info->tab.counts.delta_dead_tuples += rec->tuples_inserted + rec->tuples_updated; } @@ -873,20 +875,20 @@ bool pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) { Oid dboid; - PgStat_TableStatus *lstats; /* pending stats entry */ + PgStat_RelationStatus *lstats; /* pending stats entry */ PgStatShared_Relation *shtabstats; PgStat_StatTabEntry *tabentry; /* table entry of shared stats */ PgStat_StatDBEntry *dbentry; /* pending database entry */ dboid = entry_ref->shared_entry->key.dboid; - lstats = (PgStat_TableStatus *) entry_ref->pending; + lstats = (PgStat_RelationStatus *) entry_ref->pending; shtabstats = (PgStatShared_Relation *) entry_ref->shared_stats; /* * Ignore entries that didn't accumulate any actual counts, such as * indexes that were opened by the planner but not used. */ - if (pg_memory_is_all_zeros(&lstats->counts, + if (pg_memory_is_all_zeros(&lstats->tab.counts, sizeof(struct PgStat_TableCounts))) return true; @@ -896,35 +898,35 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) /* add the values to the shared entry. */ tabentry = &shtabstats->stats; - tabentry->numscans += lstats->counts.numscans; - if (lstats->counts.numscans) + tabentry->numscans += lstats->tab.counts.numscans; + if (lstats->tab.counts.numscans) { TimestampTz t = GetCurrentTransactionStopTimestamp(); if (t > tabentry->lastscan) tabentry->lastscan = t; } - tabentry->tuples_returned += lstats->counts.tuples_returned; - tabentry->tuples_fetched += lstats->counts.tuples_fetched; - tabentry->tuples_inserted += lstats->counts.tuples_inserted; - tabentry->tuples_updated += lstats->counts.tuples_updated; - tabentry->tuples_deleted += lstats->counts.tuples_deleted; - tabentry->tuples_hot_updated += lstats->counts.tuples_hot_updated; - tabentry->tuples_newpage_updated += lstats->counts.tuples_newpage_updated; + tabentry->tuples_returned += lstats->tab.counts.tuples_returned; + tabentry->tuples_fetched += lstats->tab.counts.tuples_fetched; + tabentry->tuples_inserted += lstats->tab.counts.tuples_inserted; + tabentry->tuples_updated += lstats->tab.counts.tuples_updated; + tabentry->tuples_deleted += lstats->tab.counts.tuples_deleted; + tabentry->tuples_hot_updated += lstats->tab.counts.tuples_hot_updated; + tabentry->tuples_newpage_updated += lstats->tab.counts.tuples_newpage_updated; /* * If table was truncated/dropped, first reset the live/dead counters. */ - if (lstats->counts.truncdropped) + if (lstats->tab.counts.truncdropped) { tabentry->live_tuples = 0; tabentry->dead_tuples = 0; tabentry->ins_since_vacuum = 0; } - tabentry->live_tuples += lstats->counts.delta_live_tuples; - tabentry->dead_tuples += lstats->counts.delta_dead_tuples; - tabentry->mod_since_analyze += lstats->counts.changed_tuples; + tabentry->live_tuples += lstats->tab.counts.delta_live_tuples; + tabentry->dead_tuples += lstats->tab.counts.delta_dead_tuples; + tabentry->mod_since_analyze += lstats->tab.counts.changed_tuples; /* * Using tuples_inserted to update ins_since_vacuum does mean that we'll @@ -933,10 +935,10 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) * triggering for inserts more often than they maybe should, which is * probably not going to be common enough to be too concerned about here. */ - tabentry->ins_since_vacuum += lstats->counts.tuples_inserted; + tabentry->ins_since_vacuum += lstats->tab.counts.tuples_inserted; - tabentry->blocks_fetched += lstats->counts.blocks_fetched; - tabentry->blocks_hit += lstats->counts.blocks_hit; + tabentry->blocks_fetched += lstats->tab.counts.blocks_fetched; + tabentry->blocks_hit += lstats->tab.counts.blocks_hit; /* Clamp live_tuples in case of negative delta_live_tuples */ tabentry->live_tuples = Max(tabentry->live_tuples, 0); @@ -947,13 +949,13 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) /* The entry was successfully flushed, add the same to database stats */ dbentry = pgstat_prep_database_pending(dboid); - dbentry->tuples_returned += lstats->counts.tuples_returned; - dbentry->tuples_fetched += lstats->counts.tuples_fetched; - dbentry->tuples_inserted += lstats->counts.tuples_inserted; - dbentry->tuples_updated += lstats->counts.tuples_updated; - dbentry->tuples_deleted += lstats->counts.tuples_deleted; - dbentry->blocks_fetched += lstats->counts.blocks_fetched; - dbentry->blocks_hit += lstats->counts.blocks_hit; + dbentry->tuples_returned += lstats->tab.counts.tuples_returned; + dbentry->tuples_fetched += lstats->tab.counts.tuples_fetched; + dbentry->tuples_inserted += lstats->tab.counts.tuples_inserted; + dbentry->tuples_updated += lstats->tab.counts.tuples_updated; + dbentry->tuples_deleted += lstats->tab.counts.tuples_deleted; + dbentry->blocks_fetched += lstats->tab.counts.blocks_fetched; + dbentry->blocks_hit += lstats->tab.counts.blocks_hit; return true; } @@ -961,7 +963,7 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) void pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref) { - PgStat_TableStatus *pending = (PgStat_TableStatus *) entry_ref->pending; + PgStat_RelationStatus *pending = (PgStat_RelationStatus *) entry_ref->pending; if (pending->relation) pgstat_unlink_relation(pending->relation); @@ -974,21 +976,25 @@ pgstat_relation_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts) } /* - * Find or create a PgStat_TableStatus entry for rel. New entry is created and + * Find or create a PgStat_RelationStatus entry for rel. New entry is created and * initialized if not exists. */ -static PgStat_TableStatus * +static PgStat_RelationStatus * pgstat_prep_relation_pending(PgStat_Kind kind, Oid rel_id, bool isshared) { PgStat_EntryRef *entry_ref; - PgStat_TableStatus *pending; + PgStat_RelationStatus *pending; entry_ref = pgstat_prep_pending_entry(kind, isshared ? InvalidOid : MyDatabaseId, rel_id, NULL); pending = entry_ref->pending; - pending->id = rel_id; - pending->shared = isshared; + pending->kind = kind; + if (kind != PGSTAT_KIND_INDEX) + { + pending->tab.id = rel_id; + pending->tab.shared = isshared; + } return pending; } @@ -997,10 +1003,10 @@ pgstat_prep_relation_pending(PgStat_Kind kind, Oid rel_id, bool isshared) * add a new (sub)transaction state record */ static void -add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level) +add_tabstat_xact_level(PgStat_RelationStatus *pgstat_info, int nest_level) { PgStat_SubXactStatus *xact_state; - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; /* * If this is the first rel to be modified at the current nest level, we @@ -1009,27 +1015,27 @@ add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level) xact_state = pgstat_get_xact_stack_level(nest_level); /* Now make a per-table stack entry */ - trans = (PgStat_TableXactStatus *) + trans = (PgStat_RelXactStatus *) MemoryContextAllocZero(TopTransactionContext, - sizeof(PgStat_TableXactStatus)); + sizeof(PgStat_RelXactStatus)); trans->nest_level = nest_level; - trans->upper = pgstat_info->trans; + trans->upper = pgstat_info->tab.trans; trans->parent = pgstat_info; trans->next = xact_state->first; xact_state->first = trans; - pgstat_info->trans = trans; + pgstat_info->tab.trans = trans; } /* * Add a new (sub)transaction record if needed. */ static void -ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info) +ensure_tabstat_xact_level(PgStat_RelationStatus *pgstat_info) { int nest_level = GetCurrentTransactionNestLevel(); - if (pgstat_info->trans == NULL || - pgstat_info->trans->nest_level != nest_level) + if (pgstat_info->tab.trans == NULL || + pgstat_info->tab.trans->nest_level != nest_level) add_tabstat_xact_level(pgstat_info, nest_level); } @@ -1043,7 +1049,7 @@ ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info) * subxact level only. */ static void -save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop) +save_truncdrop_counters(PgStat_RelXactStatus *trans, bool is_drop) { if (!trans->truncdropped || is_drop) { @@ -1058,7 +1064,7 @@ save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop) * restore counters when a truncate aborts */ static void -restore_truncdrop_counters(PgStat_TableXactStatus *trans) +restore_truncdrop_counters(PgStat_RelXactStatus *trans) { if (trans->truncdropped) { diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 946025f39ed0..0d47d745c18f 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1928,12 +1928,13 @@ CppConcat(pg_stat_get_xact_,stat)(PG_FUNCTION_ARGS) \ { \ Oid relid = PG_GETARG_OID(0); \ int64 result; \ - PgStat_TableStatus *tabentry; \ + PgStat_RelationStatus *tabentry; \ \ - if ((tabentry = find_tabstat_entry(relid)) == NULL) \ + if ((tabentry = find_relstat_entry_kind(PGSTAT_KIND_RELATION, \ + relid)) == NULL) \ result = 0; \ else \ - result = (int64) (tabentry->counts.stat); \ + result = (int64) (tabentry->tab.counts.stat); \ \ PG_RETURN_INT64(result); \ } @@ -1977,13 +1978,13 @@ CppConcat(pg_stat_get_xact_idx_,stat)(PG_FUNCTION_ARGS) \ { \ Oid relid = PG_GETARG_OID(0); \ int64 result; \ - PgStat_TableStatus *tabentry; \ + PgStat_RelationStatus *tabentry; \ \ - tabentry = find_tabstat_entry_kind(PGSTAT_KIND_INDEX, relid); \ + tabentry = find_relstat_entry_kind(PGSTAT_KIND_INDEX, relid); \ if (!tabentry) \ result = 0; \ else \ - result = (int64) (tabentry->counts.stat); \ + result = (int64) (tabentry->idx.stat); \ \ PG_RETURN_INT64(result); \ } diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index fb4e042be8ad..ce965e0db116 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -2758,7 +2758,7 @@ RelationRebuildRelation(Relation relation) /* toast OID override must be preserved */ SWAPFIELD(Oid, rd_toastoid); /* pgstat_info / enabled must be preserved */ - SWAPFIELD(struct PgStat_TableStatus *, pgstat_info); + SWAPFIELD(struct PgStat_RelationStatus *, pgstat_info); SWAPFIELD(bool, pgstat_enabled); /* preserve old partition key if we have one */ if (keep_partkey) diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88b..2a7049a92490 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2354,6 +2354,7 @@ PgStat_LockEntry PgStat_PendingDroppedStatsItem PgStat_PendingIO PgStat_PendingLock +PgStat_RelationStatus PgStat_SLRUStats PgStat_ShmemControl PgStat_Snapshot @@ -2368,7 +2369,6 @@ PgStat_StatTabEntry PgStat_StatsFileOp PgStat_SubXactStatus PgStat_TableCounts -PgStat_TableStatus PgStat_TableXactStatus PgStat_WalCounters PgStat_WalStats -- 2.55.0 From 7da7413fad7c38c39b396789511f0e910a16608b Mon Sep 17 00:00:00 2001 From: Michael Paquier <[email protected]> Date: Fri, 10 Jul 2026 14:53:50 +0900 Subject: [PATCH v13 3/3] Introduce relfilenode statistics Statistics data previously associated to relations are not moved under a new stats kind, for relfilenodes. This enables the reconstruction of the statistics after a system crash, as WAL replay is only able to know about the relfilenodes it works on. Shared catalogs are handled using their relmapping. A new set of fetch functions are used in the views, providing an equivalent of what existed previously. When a relation rewrite happens, the aggregated counters are not persisted, restarted at 0 as we start afresh. --- src/include/catalog/pg_proc.dat | 75 ++++--- src/include/pgstat.h | 39 +++- src/include/utils/pgstat_internal.h | 14 ++ src/include/utils/pgstat_kind.h | 15 +- src/backend/access/heap/heapam_xlog.c | 13 ++ src/backend/catalog/system_views.sql | 18 +- src/backend/postmaster/autovacuum.c | 40 ++-- src/backend/utils/activity/Makefile | 1 + src/backend/utils/activity/meson.build | 1 + src/backend/utils/activity/pgstat.c | 15 ++ src/backend/utils/activity/pgstat_relation.c | 190 +++++++++++------ .../utils/activity/pgstat_relfilenode.c | 179 ++++++++++++++++ src/backend/utils/adt/pgstatfuncs.c | 74 ++++--- src/test/isolation/expected/stats.out | 198 +++++++++--------- src/test/isolation/specs/stats.spec | 10 +- src/test/regress/expected/rules.out | 18 +- src/test/regress/expected/stats.out | 169 +++++++-------- src/test/regress/expected/stats_rewrite.out | 20 +- src/test/regress/expected/stats_rewrite_1.out | 8 +- src/test/regress/sql/stats.sql | 42 ++-- .../postgres_fdw/expected/postgres_fdw.out | 20 +- contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +- 22 files changed, 751 insertions(+), 416 deletions(-) create mode 100644 src/backend/utils/activity/pgstat_relfilenode.c diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index dc6c61e84e54..acb8b9a4d588 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5578,45 +5578,6 @@ proname => 'pg_stat_get_tuples_fetched', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_tuples_fetched' }, -{ oid => '1931', descr => 'statistics: number of tuples inserted', - proname => 'pg_stat_get_tuples_inserted', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_inserted' }, -{ oid => '1932', descr => 'statistics: number of tuples updated', - proname => 'pg_stat_get_tuples_updated', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_updated' }, -{ oid => '1933', descr => 'statistics: number of tuples deleted', - proname => 'pg_stat_get_tuples_deleted', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_deleted' }, -{ oid => '1972', descr => 'statistics: number of tuples hot updated', - proname => 'pg_stat_get_tuples_hot_updated', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_hot_updated' }, -{ oid => '6217', - descr => 'statistics: number of tuples updated onto a new page', - proname => 'pg_stat_get_tuples_newpage_updated', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_newpage_updated' }, -{ oid => '2878', descr => 'statistics: number of live tuples', - proname => 'pg_stat_get_live_tuples', provolatile => 's', proparallel => 'r', - prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_live_tuples' }, -{ oid => '2879', descr => 'statistics: number of dead tuples', - proname => 'pg_stat_get_dead_tuples', provolatile => 's', proparallel => 'r', - prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_dead_tuples' }, -{ oid => '3177', - descr => 'statistics: number of tuples changed since last analyze', - proname => 'pg_stat_get_mod_since_analyze', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_mod_since_analyze' }, -{ oid => '5053', - descr => 'statistics: number of tuples inserted since last vacuum', - proname => 'pg_stat_get_ins_since_vacuum', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_ins_since_vacuum' }, { oid => '1934', descr => 'statistics: number of blocks fetched', proname => 'pg_stat_get_blocks_fetched', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', @@ -6294,6 +6255,42 @@ proname => 'pg_stat_get_xact_idx_blocks_hit', provolatile => 'v', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_xact_idx_blocks_hit' }, +{ oid => '9055', descr => 'statistics: number of tuples inserted (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_inserted', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_inserted' }, +{ oid => '9056', descr => 'statistics: number of tuples updated (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_updated', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_updated' }, +{ oid => '9057', descr => 'statistics: number of tuples deleted (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_deleted', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_deleted' }, +{ oid => '9058', descr => 'statistics: number of tuples HOT updated (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_hot_updated', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_hot_updated' }, +{ oid => '9059', descr => 'statistics: number of tuples updated to new page (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_newpage_updated', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_newpage_updated' }, +{ oid => '9060', descr => 'statistics: number of live tuples (by relfilenode)', + proname => 'pg_stat_get_rfn_live_tuples', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_live_tuples' }, +{ oid => '9061', descr => 'statistics: number of dead tuples (by relfilenode)', + proname => 'pg_stat_get_rfn_dead_tuples', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_dead_tuples' }, +{ oid => '9062', descr => 'statistics: number of tuples changed since last analyze (by relfilenode)', + proname => 'pg_stat_get_rfn_mod_since_analyze', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_mod_since_analyze' }, +{ oid => '9063', descr => 'statistics: number of tuples inserted since last vacuum (by relfilenode)', + proname => 'pg_stat_get_rfn_ins_since_vacuum', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_ins_since_vacuum' }, { oid => '3777', descr => 'statistics: reset collected statistics for a single function in the current database', proname => 'pg_stat_reset_single_function_counters', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 892cdf24139b..f2094270a000 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -12,6 +12,7 @@ #define PGSTAT_H #include "datatype/timestamp.h" +#include "common/relpath.h" #include "portability/instr_time.h" #include "postmaster/pgarch.h" /* for MAX_XFN_CHARS */ #include "replication/conflict.h" @@ -210,6 +211,7 @@ typedef struct PgStat_RelationStatus { Oid id; /* table's OID */ bool shared; /* is it a shared catalog? */ + RelFileNumber relfilenode; /* table's relfilenode */ struct PgStat_RelXactStatus *trans; /* lowest subxact's counts */ PgStat_TableCounts counts; /* event counts to be sent */ } tab; @@ -489,17 +491,6 @@ typedef struct PgStat_StatTabEntry PgStat_Counter tuples_returned; PgStat_Counter tuples_fetched; - PgStat_Counter tuples_inserted; - PgStat_Counter tuples_updated; - PgStat_Counter tuples_deleted; - PgStat_Counter tuples_hot_updated; - PgStat_Counter tuples_newpage_updated; - - PgStat_Counter live_tuples; - PgStat_Counter dead_tuples; - PgStat_Counter mod_since_analyze; - PgStat_Counter ins_since_vacuum; - PgStat_Counter blocks_fetched; PgStat_Counter blocks_hit; @@ -534,6 +525,22 @@ typedef struct PgStat_StatIdxEntry TimestampTz stat_reset_time; } PgStat_StatIdxEntry; +typedef struct PgStat_StatRFNodeEntry +{ + PgStat_Counter tuples_inserted; + PgStat_Counter tuples_updated; + PgStat_Counter tuples_deleted; + PgStat_Counter tuples_hot_updated; + PgStat_Counter tuples_newpage_updated; + + PgStat_Counter live_tuples; + PgStat_Counter dead_tuples; + PgStat_Counter mod_since_analyze; + PgStat_Counter ins_since_vacuum; + + TimestampTz stat_reset_time; +} PgStat_StatRFNodeEntry; + /* ------ * PgStat_WalCounters WAL activity data gathered from WalUsage * @@ -848,6 +855,16 @@ extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry_ext(bool shared, Oid reloid, bool *may_free); +extern PgStat_StatRFNodeEntry *pgstat_fetch_stat_rfnodeentry(Oid dboid, + RelFileNumber rfn); + +/* WAL replay helpers for relfilenode stats */ +extern void pgstat_wal_replay_insert(Oid dboid, RelFileNumber relfilenode, + int ntuples); +extern void pgstat_wal_replay_update(Oid dboid, RelFileNumber relfilenode, + bool hot, bool newpage); +extern void pgstat_wal_replay_delete(Oid dboid, RelFileNumber relfilenode); + /* * Functions in pgstat_replslot.c diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h index 4df3082f2b6e..466b3e79e5bd 100644 --- a/src/include/utils/pgstat_internal.h +++ b/src/include/utils/pgstat_internal.h @@ -513,6 +513,12 @@ typedef struct PgStatShared_Index PgStat_StatIdxEntry stats; } PgStatShared_Index; +typedef struct PgStatShared_RelFileNode +{ + PgStatShared_Common header; + PgStat_StatRFNodeEntry stats; +} PgStatShared_RelFileNode; + typedef struct PgStatShared_Function { PgStatShared_Common header; @@ -802,6 +808,14 @@ extern void pgstat_index_delete_pending_cb(PgStat_EntryRef *entry_ref); extern void pgstat_index_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); +/* + * Functions in pgstat_relfilenode.c + */ + +extern bool pgstat_relfilenode_flush_cb(PgStat_EntryRef *entry_ref, bool nowait); +extern void pgstat_relfilenode_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); + + /* * Functions in pgstat_replslot.c */ diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h index 45ca599d0dd6..84ccaa068489 100644 --- a/src/include/utils/pgstat_kind.h +++ b/src/include/utils/pgstat_kind.h @@ -31,15 +31,16 @@ #define PGSTAT_KIND_REPLSLOT 5 /* per-slot statistics */ #define PGSTAT_KIND_SUBSCRIPTION 6 /* per-subscription statistics */ #define PGSTAT_KIND_BACKEND 7 /* per-backend statistics */ +#define PGSTAT_KIND_RELFILENODE 8 /* per-relfilenode statistics */ /* stats for fixed-numbered objects */ -#define PGSTAT_KIND_ARCHIVER 8 -#define PGSTAT_KIND_BGWRITER 9 -#define PGSTAT_KIND_CHECKPOINTER 10 -#define PGSTAT_KIND_IO 11 -#define PGSTAT_KIND_LOCK 12 -#define PGSTAT_KIND_SLRU 13 -#define PGSTAT_KIND_WAL 14 +#define PGSTAT_KIND_ARCHIVER 9 +#define PGSTAT_KIND_BGWRITER 10 +#define PGSTAT_KIND_CHECKPOINTER 11 +#define PGSTAT_KIND_IO 12 +#define PGSTAT_KIND_LOCK 13 +#define PGSTAT_KIND_SLRU 14 +#define PGSTAT_KIND_WAL 15 #define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE #define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL diff --git a/src/backend/access/heap/heapam_xlog.c b/src/backend/access/heap/heapam_xlog.c index 9ed7024e8147..f1379c699900 100644 --- a/src/backend/access/heap/heapam_xlog.c +++ b/src/backend/access/heap/heapam_xlog.c @@ -19,6 +19,7 @@ #include "access/visibilitymap.h" #include "access/xlog.h" #include "access/xlogutils.h" +#include "pgstat.h" #include "storage/freespace.h" #include "storage/standby.h" @@ -357,6 +358,9 @@ heap_xlog_delete(XLogReaderState *record) } if (BufferIsValid(buffer)) UnlockReleaseBuffer(buffer); + + pgstat_wal_replay_delete(target_locator.dbOid, + target_locator.relNumber); } /* @@ -483,6 +487,9 @@ heap_xlog_insert(XLogReaderState *record) */ if (action == BLK_NEEDS_REDO && freespace < BLCKSZ / 5) XLogRecordPageWithFreeSpace(target_locator, blkno, freespace); + + pgstat_wal_replay_insert(target_locator.dbOid, + target_locator.relNumber, 1); } /* @@ -688,6 +695,9 @@ heap_xlog_multi_insert(XLogReaderState *record) */ if (action == BLK_NEEDS_REDO && freespace < BLCKSZ / 5) XLogRecordPageWithFreeSpace(rlocator, blkno, freespace); + + pgstat_wal_replay_insert(rlocator.dbOid, rlocator.relNumber, + xlrec->ntuples); } /* @@ -967,6 +977,9 @@ heap_xlog_update(XLogReaderState *record, bool hot_update) */ if (newaction == BLK_NEEDS_REDO && !hot_update && freespace < BLCKSZ / 5) XLogRecordPageWithFreeSpace(rlocator, newblk, freespace); + + pgstat_wal_replay_update(rlocator.dbOid, rlocator.relNumber, + hot_update, (oldblk != newblk && !hot_update)); } /* diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index ce991e4269e7..2577fe608f23 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -726,15 +726,15 @@ CREATE VIEW pg_stat_all_tables AS max(pg_stat_get_idx_lastscan(I.indexrelid)) AS last_idx_scan, sum(pg_stat_get_idx_tuples_fetched(I.indexrelid))::bigint + pg_stat_get_tuples_fetched(C.oid) AS idx_tup_fetch, - pg_stat_get_tuples_inserted(C.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(C.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(C.oid) AS n_tup_del, - pg_stat_get_tuples_hot_updated(C.oid) AS n_tup_hot_upd, - pg_stat_get_tuples_newpage_updated(C.oid) AS n_tup_newpage_upd, - pg_stat_get_live_tuples(C.oid) AS n_live_tup, - pg_stat_get_dead_tuples(C.oid) AS n_dead_tup, - pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze, - pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(C.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(C.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(C.oid)) AS n_tup_del, + pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode(C.oid)) AS n_tup_hot_upd, + pg_stat_get_rfn_tuples_newpage_updated(pg_relation_filenode(C.oid)) AS n_tup_newpage_upd, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(C.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(C.oid)) AS n_dead_tup, + pg_stat_get_rfn_mod_since_analyze(pg_relation_filenode(C.oid)) AS n_mod_since_analyze, + pg_stat_get_rfn_ins_since_vacuum(pg_relation_filenode(C.oid)) AS n_ins_since_vacuum, pg_stat_get_last_vacuum_time(C.oid) as last_vacuum, pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum, pg_stat_get_last_analyze_time(C.oid) as last_analyze, diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index 45abf48768af..23e4637968f8 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -109,6 +109,7 @@ #include "utils/memutils.h" #include "utils/ps_status.h" #include "utils/rel.h" +#include "utils/relmapper.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timeout.h" @@ -3082,10 +3083,10 @@ relation_needs_vacanalyze(Oid relid, bool *wraparound, AutoVacuumScores *scores) { - PgStat_StatTabEntry *tabentry; + PgStat_StatRFNodeEntry *rfnentry; + RelFileNumber rfn; bool force_vacuum; bool av_enabled; - bool may_free = false; /* constants from reloptions or GUC variables */ int vac_base_thresh, @@ -3247,20 +3248,27 @@ relation_needs_vacanalyze(Oid relid, *dovacuum = true; /* - * If we found stats for the table, and autovacuum is currently enabled, - * make a threshold-based decision whether to vacuum and/or analyze. If - * autovacuum is currently disabled, we must be here for anti-wraparound - * vacuuming only, so don't vacuum (or analyze) anything that's not being - * forced. + * Fetch tuple counters from the relfilenode entry. For shared catalogs, + * use the relation map. */ - tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared, - relid, &may_free); - if (!tabentry) - return; + rfn = classForm->relfilenode; + if (!RelFileNumberIsValid(rfn)) + rfn = RelationMapOidToFilenumber(relid, classForm->relisshared); - vactuples = tabentry->dead_tuples; - instuples = tabentry->ins_since_vacuum; - anltuples = tabentry->mod_since_analyze; + rfnentry = pgstat_fetch_stat_rfnodeentry( + classForm->relisshared ? InvalidOid : MyDatabaseId, rfn); + if (rfnentry) + { + vactuples = rfnentry->dead_tuples; + instuples = rfnentry->ins_since_vacuum; + anltuples = rfnentry->mod_since_analyze; + } + else + { + vactuples = 0; + instuples = 0; + anltuples = 0; + } /* If the table hasn't yet been vacuumed, take reltuples as zero */ if (reltuples < 0) @@ -3334,10 +3342,6 @@ relation_needs_vacanalyze(Oid relid, vactuples, vacthresh, scores->vac, anltuples, anlthresh, scores->anl, scores->xid, scores->mxid); - - /* Avoid leaking pgstat entries until the end of autovacuum. */ - if (may_free) - pfree(tabentry); } /* diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile index 2e32d1485d64..7ee4c82e1b65 100644 --- a/src/backend/utils/activity/Makefile +++ b/src/backend/utils/activity/Makefile @@ -30,6 +30,7 @@ OBJS = \ pgstat_kind.o \ pgstat_lock.o \ pgstat_relation.o \ + pgstat_relfilenode.o \ pgstat_replslot.o \ pgstat_shmem.o \ pgstat_slru.o \ diff --git a/src/backend/utils/activity/meson.build b/src/backend/utils/activity/meson.build index e6dcb2e26fc5..754e120dd612 100644 --- a/src/backend/utils/activity/meson.build +++ b/src/backend/utils/activity/meson.build @@ -15,6 +15,7 @@ backend_sources += files( 'pgstat_kind.c', 'pgstat_lock.c', 'pgstat_relation.c', + 'pgstat_relfilenode.c', 'pgstat_replslot.c', 'pgstat_shmem.c', 'pgstat_slru.c', diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 907d1f53a7ec..e545b1c7aaac 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -333,6 +333,21 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .reset_timestamp_cb = pgstat_index_reset_timestamp_cb, }, + [PGSTAT_KIND_RELFILENODE] = { + .name = "relfilenode", + + .fixed_amount = false, + .write_to_file = true, + + .shared_size = sizeof(PgStatShared_RelFileNode), + .shared_data_off = offsetof(PgStatShared_RelFileNode, stats), + .shared_data_len = sizeof(((PgStatShared_RelFileNode *) 0)->stats), + .pending_size = sizeof(PgStat_RelationStatus), + + .flush_pending_cb = pgstat_relfilenode_flush_cb, + .reset_timestamp_cb = pgstat_relfilenode_reset_timestamp_cb, + }, + [PGSTAT_KIND_FUNCTION] = { .name = "function", diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 0cfa5be49cde..7dbcc22fa78f 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -37,6 +37,7 @@ typedef struct TwoPhasePgStatRecord PgStat_Counter updated_pre_truncdrop; PgStat_Counter deleted_pre_truncdrop; Oid id; /* table's OID */ + RelFileNumber relfilenode; /* table's relfilenode at prepare time */ bool shared; /* is it a shared catalog? */ bool truncdropped; /* was the relation truncated/dropped? */ } TwoPhasePgStatRecord; @@ -182,6 +183,13 @@ pgstat_assoc_relation(Relation rel) RelationGetRelid(rel), rel->rd_rel->relisshared); + /* Store relfilenode for PGSTAT_KIND_RELFILENODE flush */ + if (rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE && + rel->rd_rel->relkind != RELKIND_INDEX) + rel->pgstat_info->tab.relfilenode = rel->rd_locator.relNumber; + else + rel->pgstat_info->tab.relfilenode = InvalidRelFileNumber; + /* don't allow link a stats to multiple relcache entries */ Assert(rel->pgstat_info->relation == NULL); @@ -264,8 +272,11 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, PgStat_Counter deadtuples, TimestampTz starttime) { PgStat_EntryRef *entry_ref; + PgStat_EntryRef *rfn_ref; PgStatShared_Relation *shtabentry; + PgStatShared_RelFileNode *shrfnstats; PgStat_StatTabEntry *tabentry; + PgStat_StatRFNodeEntry *rfnentry; Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId); TimestampTz ts; PgStat_Counter elapsedtime; @@ -284,21 +295,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats; tabentry = &shtabentry->stats; - tabentry->live_tuples = livetuples; - tabentry->dead_tuples = deadtuples; - - /* - * It is quite possible that a non-aggressive VACUUM ended up skipping - * various pages, however, we'll zero the insert counter here regardless. - * It's currently used only to track when we need to perform an "insert" - * autovacuum, which are mainly intended to freeze newly inserted tuples. - * Zeroing this may just mean we'll not try to vacuum the table again - * until enough tuples have been inserted to trigger another insert - * autovacuum. An anti-wraparound autovacuum will catch any persistent - * stragglers. - */ - tabentry->ins_since_vacuum = 0; - if (AmAutoVacuumWorkerProcess()) { tabentry->last_autovacuum_time = ts; @@ -314,6 +310,34 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, pgstat_unlock_entry(entry_ref); + /* Update the relfilenode entry */ + rfn_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, + (uint64) rel->rd_locator.relNumber, + false); + if (rfn_ref) + { + shrfnstats = (PgStatShared_RelFileNode *) rfn_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->live_tuples = livetuples; + rfnentry->dead_tuples = deadtuples; + + /* + * It is quite possible that a non-aggressive VACUUM ended up + * skipping various pages, however, we'll zero the insert counter + * here regardless. It's currently used only to track when we need + * to perform an "insert" autovacuum, which are mainly intended to + * freeze newly inserted tuples. Zeroing this may just mean we'll + * not try to vacuum the table again until enough tuples have been + * inserted to trigger another insert autovacuum. An + * anti-wraparound autovacuum will catch any persistent stragglers. + */ + rfnentry->ins_since_vacuum = 0; + + pgstat_unlock_entry(rfn_ref); + } + /* * Flush IO statistics now. pgstat_report_stat() will flush IO stats, * however this will not be called until after an entire autovacuum cycle @@ -336,8 +360,11 @@ pgstat_report_analyze(Relation rel, bool resetcounter, TimestampTz starttime) { PgStat_EntryRef *entry_ref; + PgStat_EntryRef *rfn_ref; PgStatShared_Relation *shtabentry; + PgStatShared_RelFileNode *shrfnstats; PgStat_StatTabEntry *tabentry; + PgStat_StatRFNodeEntry *rfnentry; Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId); TimestampTz ts; PgStat_Counter elapsedtime; @@ -388,17 +415,6 @@ pgstat_report_analyze(Relation rel, shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats; tabentry = &shtabentry->stats; - tabentry->live_tuples = livetuples; - tabentry->dead_tuples = deadtuples; - - /* - * If commanded, reset mod_since_analyze to zero. This forgets any - * changes that were committed while the ANALYZE was in progress, but we - * have no good way to estimate how many of those there were. - */ - if (resetcounter) - tabentry->mod_since_analyze = 0; - if (AmAutoVacuumWorkerProcess()) { tabentry->last_autoanalyze_time = ts; @@ -414,6 +430,30 @@ pgstat_report_analyze(Relation rel, pgstat_unlock_entry(entry_ref); + /* Update the relfilenode entry */ + rfn_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, + (uint64) rel->rd_locator.relNumber, + false); + if (rfn_ref) + { + shrfnstats = (PgStatShared_RelFileNode *) rfn_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->live_tuples = livetuples; + rfnentry->dead_tuples = deadtuples; + + /* + * If commanded, reset mod_since_analyze to zero. This forgets any + * changes that were committed while the ANALYZE was in progress, but + * we have no good way to estimate how many of those there were. + */ + if (resetcounter) + rfnentry->mod_since_analyze = 0; + + pgstat_unlock_entry(rfn_ref); + } + /* see pgstat_report_vacuum() */ pgstat_flush_io(false); (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); @@ -767,6 +807,7 @@ AtPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) record.updated_pre_truncdrop = trans->updated_pre_truncdrop; record.deleted_pre_truncdrop = trans->deleted_pre_truncdrop; record.id = tabstat->tab.id; + record.relfilenode = tabstat->tab.relfilenode; record.shared = tabstat->tab.shared; record.truncdropped = trans->truncdropped; @@ -811,6 +852,7 @@ pgstat_twophase_postcommit(FullTransactionId fxid, uint16 info, /* Find or create a tabstat entry for the rel */ pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); + pgstat_info->tab.relfilenode = rec->relfilenode; /* Same math as in AtEOXact_PgStat, commit case */ pgstat_info->tab.counts.tuples_inserted += rec->tuples_inserted; @@ -847,6 +889,7 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, /* Find or create a tabstat entry for the rel */ pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); + pgstat_info->tab.relfilenode = rec->relfilenode; /* Same math as in AtEOXact_PgStat, abort case */ if (rec->truncdropped) @@ -875,10 +918,14 @@ bool pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) { Oid dboid; + RelFileNumber rfn; PgStat_RelationStatus *lstats; /* pending stats entry */ PgStatShared_Relation *shtabstats; PgStat_StatTabEntry *tabentry; /* table entry of shared stats */ PgStat_StatDBEntry *dbentry; /* pending database entry */ + PgStat_EntryRef *rfn_ref; + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; dboid = entry_ref->shared_entry->key.dboid; lstats = (PgStat_RelationStatus *) entry_ref->pending; @@ -908,45 +955,70 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) } tabentry->tuples_returned += lstats->tab.counts.tuples_returned; tabentry->tuples_fetched += lstats->tab.counts.tuples_fetched; - tabentry->tuples_inserted += lstats->tab.counts.tuples_inserted; - tabentry->tuples_updated += lstats->tab.counts.tuples_updated; - tabentry->tuples_deleted += lstats->tab.counts.tuples_deleted; - tabentry->tuples_hot_updated += lstats->tab.counts.tuples_hot_updated; - tabentry->tuples_newpage_updated += lstats->tab.counts.tuples_newpage_updated; - - /* - * If table was truncated/dropped, first reset the live/dead counters. - */ - if (lstats->tab.counts.truncdropped) - { - tabentry->live_tuples = 0; - tabentry->dead_tuples = 0; - tabentry->ins_since_vacuum = 0; - } - - tabentry->live_tuples += lstats->tab.counts.delta_live_tuples; - tabentry->dead_tuples += lstats->tab.counts.delta_dead_tuples; - tabentry->mod_since_analyze += lstats->tab.counts.changed_tuples; - - /* - * Using tuples_inserted to update ins_since_vacuum does mean that we'll - * track aborted inserts too. This isn't ideal, but otherwise probably - * not worth adding an extra field for. It may just amount to autovacuums - * triggering for inserts more often than they maybe should, which is - * probably not going to be common enough to be too concerned about here. - */ - tabentry->ins_since_vacuum += lstats->tab.counts.tuples_inserted; tabentry->blocks_fetched += lstats->tab.counts.blocks_fetched; tabentry->blocks_hit += lstats->tab.counts.blocks_hit; - /* Clamp live_tuples in case of negative delta_live_tuples */ - tabentry->live_tuples = Max(tabentry->live_tuples, 0); - /* Likewise for dead_tuples */ - tabentry->dead_tuples = Max(tabentry->dead_tuples, 0); - pgstat_unlock_entry(entry_ref); + /* + * Flush tuple counters to the relfilenode entry. + * + * Use the current relfilenode from the relation (if available) since it + * may have changed due to TRUNCATE. Fall back to the cached value. + */ + rfn = InvalidRelFileNumber; + + if (lstats->relation) + rfn = lstats->relation->rd_locator.relNumber; + else if (lstats->tab.relfilenode != InvalidRelFileNumber) + rfn = lstats->tab.relfilenode; + + if (rfn != InvalidRelFileNumber) + { + rfn_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, + (uint64) rfn, + nowait); + if (rfn_ref) + { + shrfnstats = (PgStatShared_RelFileNode *) rfn_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->tuples_inserted += lstats->tab.counts.tuples_inserted; + rfnentry->tuples_updated += lstats->tab.counts.tuples_updated; + rfnentry->tuples_deleted += lstats->tab.counts.tuples_deleted; + rfnentry->tuples_hot_updated += lstats->tab.counts.tuples_hot_updated; + rfnentry->tuples_newpage_updated += lstats->tab.counts.tuples_newpage_updated; + + if (lstats->tab.counts.truncdropped) + { + rfnentry->live_tuples = 0; + rfnentry->dead_tuples = 0; + rfnentry->ins_since_vacuum = 0; + } + + rfnentry->live_tuples += lstats->tab.counts.delta_live_tuples; + rfnentry->dead_tuples += lstats->tab.counts.delta_dead_tuples; + rfnentry->mod_since_analyze += lstats->tab.counts.changed_tuples; + + /* + * Using tuples_inserted to update ins_since_vacuum means we + * track aborted inserts too. This isn't ideal, but not worth + * adding an extra field for — it may just trigger autovacuums + * slightly more often than necessary. + */ + rfnentry->ins_since_vacuum += lstats->tab.counts.tuples_inserted; + + /* Clamp live_tuples in case of negative delta_live_tuples */ + rfnentry->live_tuples = Max(rfnentry->live_tuples, 0); + /* Likewise for dead_tuples */ + rfnentry->dead_tuples = Max(rfnentry->dead_tuples, 0); + + pgstat_unlock_entry(rfn_ref); + } + } + /* The entry was successfully flushed, add the same to database stats */ dbentry = pgstat_prep_database_pending(dboid); dbentry->tuples_returned += lstats->tab.counts.tuples_returned; diff --git a/src/backend/utils/activity/pgstat_relfilenode.c b/src/backend/utils/activity/pgstat_relfilenode.c new file mode 100644 index 000000000000..82613fb1a4ee --- /dev/null +++ b/src/backend/utils/activity/pgstat_relfilenode.c @@ -0,0 +1,179 @@ +/* ------------------------------------------------------------------------- + * + * pgstat_relfilenode.c + * Implementation of per-relfilenode statistics. + * + * This file contains the implementation of PGSTAT_KIND_RELFILENODE statistics, + * for tuple counters (inserts, updates, deletes, live/dead tuples, etc.) keyed + * by relfilenode. These counters are rebuilt during crash recovery, from WAL + * replayed. + * + * Copyright (c) 2001-2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/backend/utils/activity/pgstat_relfilenode.c + * ------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "utils/memutils.h" +#include "utils/pgstat_internal.h" + + +/* + * Flush out pending stats for a relfilenode entry. + * + * If nowait is true and the lock could not be immediately acquired, returns + * false without flushing the entry. Otherwise returns true. + */ +bool +pgstat_relfilenode_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) +{ + PgStat_RelationStatus *lstats; /* pending stats entry */ + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; + + lstats = (PgStat_RelationStatus *) entry_ref->pending; + shrfnstats = (PgStatShared_RelFileNode *) entry_ref->shared_stats; + + /* + * Ignore entries that didn't accumulate any actual counts, such as + * tables that were opened but not modified. + */ + if (pg_memory_is_all_zeros(&lstats->tab.counts, + sizeof(struct PgStat_TableCounts))) + return true; + + if (!pgstat_lock_entry(entry_ref, nowait)) + return false; + + /* Add the values to the shared entry. */ + rfnentry = &shrfnstats->stats; + + /* + * If table was truncated/dropped, first reset the live/dead counters. + */ + if (lstats->tab.counts.truncdropped) + { + rfnentry->live_tuples = 0; + rfnentry->dead_tuples = 0; + rfnentry->ins_since_vacuum = 0; + } + + rfnentry->live_tuples += lstats->tab.counts.delta_live_tuples; + rfnentry->dead_tuples += lstats->tab.counts.delta_dead_tuples; + rfnentry->mod_since_analyze += lstats->tab.counts.changed_tuples; + + /* + * Using tuples_inserted to update ins_since_vacuum means we track + * aborted inserts too. This isn't ideal, but not worth adding an extra + * field for — it may just trigger autovacuums slightly more often than + * necessary. + */ + rfnentry->ins_since_vacuum += lstats->tab.counts.tuples_inserted; + + /* Clamp live_tuples in case of negative delta_live_tuples */ + rfnentry->live_tuples = Max(rfnentry->live_tuples, 0); + /* Likewise for dead_tuples */ + rfnentry->dead_tuples = Max(rfnentry->dead_tuples, 0); + + pgstat_unlock_entry(entry_ref); + + return true; +} + +/* + * Callback to reset the timestamp on a relfilenode stats entry. + */ +void +pgstat_relfilenode_reset_timestamp_cb(PgStatShared_Common *header, + TimestampTz ts) +{ + ((PgStatShared_RelFileNode *) header)->stats.stat_reset_time = ts; +} + +/* + * Fetch the relfilenode stats entry for the given database and relfilenode. + */ +PgStat_StatRFNodeEntry * +pgstat_fetch_stat_rfnodeentry(Oid dboid, RelFileNumber rfn) +{ + return (PgStat_StatRFNodeEntry *) + pgstat_fetch_entry(PGSTAT_KIND_RELFILENODE, dboid, (uint64) rfn, NULL); +} + +/* + * WAL replay: increment tuple counters in the relfilenode entry. + * + * This is called during WAL replay to rebuild tuple counters from + * heap WAL records. It directly updates the shared stats entry without + * going through the pending/flush machinery. + */ +void +pgstat_wal_replay_insert(Oid dboid, RelFileNumber relfilenode, int ntuples) +{ + PgStat_EntryRef *entry_ref; + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; + + entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, (uint64) relfilenode, + false); + shrfnstats = (PgStatShared_RelFileNode *) entry_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->tuples_inserted += ntuples; + rfnentry->mod_since_analyze += ntuples; + rfnentry->ins_since_vacuum += ntuples; + rfnentry->live_tuples += ntuples; + + pgstat_unlock_entry(entry_ref); +} + +void +pgstat_wal_replay_update(Oid dboid, RelFileNumber relfilenode, + bool hot, bool newpage) +{ + PgStat_EntryRef *entry_ref; + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; + + entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, (uint64) relfilenode, + false); + shrfnstats = (PgStatShared_RelFileNode *) entry_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->tuples_updated++; + rfnentry->mod_since_analyze++; + rfnentry->dead_tuples++; + if (hot) + rfnentry->tuples_hot_updated++; + if (newpage) + rfnentry->tuples_newpage_updated++; + + pgstat_unlock_entry(entry_ref); +} + +void +pgstat_wal_replay_delete(Oid dboid, RelFileNumber relfilenode) +{ + PgStat_EntryRef *entry_ref; + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; + + entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, (uint64) relfilenode, + false); + shrfnstats = (PgStatShared_RelFileNode *) entry_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->tuples_deleted++; + rfnentry->mod_since_analyze++; + rfnentry->dead_tuples++; + rfnentry->live_tuples--; + rfnentry->live_tuples = Max(rfnentry->live_tuples, 0); + + pgstat_unlock_entry(entry_ref); +} diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 0d47d745c18f..8be2cb4d1293 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -69,45 +69,65 @@ PG_STAT_GET_RELENTRY_INT64(blocks_fetched) /* pg_stat_get_blocks_hit */ PG_STAT_GET_RELENTRY_INT64(blocks_hit) -/* pg_stat_get_dead_tuples */ -PG_STAT_GET_RELENTRY_INT64(dead_tuples) - -/* pg_stat_get_ins_since_vacuum */ -PG_STAT_GET_RELENTRY_INT64(ins_since_vacuum) - -/* pg_stat_get_live_tuples */ -PG_STAT_GET_RELENTRY_INT64(live_tuples) - -/* pg_stat_get_mod_since_analyze */ -PG_STAT_GET_RELENTRY_INT64(mod_since_analyze) - /* pg_stat_get_numscans */ PG_STAT_GET_RELENTRY_INT64(numscans) -/* pg_stat_get_tuples_deleted */ -PG_STAT_GET_RELENTRY_INT64(tuples_deleted) - /* pg_stat_get_tuples_fetched */ PG_STAT_GET_RELENTRY_INT64(tuples_fetched) -/* pg_stat_get_tuples_hot_updated */ -PG_STAT_GET_RELENTRY_INT64(tuples_hot_updated) - -/* pg_stat_get_tuples_newpage_updated */ -PG_STAT_GET_RELENTRY_INT64(tuples_newpage_updated) - -/* pg_stat_get_tuples_inserted */ -PG_STAT_GET_RELENTRY_INT64(tuples_inserted) - /* pg_stat_get_tuples_returned */ PG_STAT_GET_RELENTRY_INT64(tuples_returned) -/* pg_stat_get_tuples_updated */ -PG_STAT_GET_RELENTRY_INT64(tuples_updated) - /* pg_stat_get_vacuum_count */ PG_STAT_GET_RELENTRY_INT64(vacuum_count) +/* + * Accessor macro for relfilenode stats entries (PgStat_StatRFNodeEntry). + * Takes a relfilenode number as input (from pg_class.relfilenode). + */ +#define PG_STAT_GET_RFNENTRY_INT64(stat) \ +Datum \ +CppConcat(pg_stat_get_rfn_,stat)(PG_FUNCTION_ARGS) \ +{ \ + Oid rfn = PG_GETARG_OID(0); \ + int64 result; \ + PgStat_StatRFNodeEntry *rfnentry; \ + \ + if ((rfnentry = pgstat_fetch_stat_rfnodeentry(MyDatabaseId, rfn)) == NULL) \ + result = 0; \ + else \ + result = (int64) (rfnentry->stat); \ + \ + PG_RETURN_INT64(result); \ +} + +/* pg_stat_get_rfn_tuples_inserted */ +PG_STAT_GET_RFNENTRY_INT64(tuples_inserted) + +/* pg_stat_get_rfn_tuples_updated */ +PG_STAT_GET_RFNENTRY_INT64(tuples_updated) + +/* pg_stat_get_rfn_tuples_deleted */ +PG_STAT_GET_RFNENTRY_INT64(tuples_deleted) + +/* pg_stat_get_rfn_tuples_hot_updated */ +PG_STAT_GET_RFNENTRY_INT64(tuples_hot_updated) + +/* pg_stat_get_rfn_tuples_newpage_updated */ +PG_STAT_GET_RFNENTRY_INT64(tuples_newpage_updated) + +/* pg_stat_get_rfn_live_tuples */ +PG_STAT_GET_RFNENTRY_INT64(live_tuples) + +/* pg_stat_get_rfn_dead_tuples */ +PG_STAT_GET_RFNENTRY_INT64(dead_tuples) + +/* pg_stat_get_rfn_mod_since_analyze */ +PG_STAT_GET_RFNENTRY_INT64(mod_since_analyze) + +/* pg_stat_get_rfn_ins_since_vacuum */ +PG_STAT_GET_RFNENTRY_INT64(ins_since_vacuum) + /* * Accessor macro for index stats entries (PgStat_StatIdxEntry). */ diff --git a/src/test/isolation/expected/stats.out b/src/test/isolation/expected/stats.out index cfad309ccf34..1ede5af1cf81 100644 --- a/src/test/isolation/expected/stats.out +++ b/src/test/isolation/expected/stats.out @@ -2186,18 +2186,18 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count --------+------------+---------+---------+---------+----------+----------+------------ - 0| 0| 0| 0| 0| 0| 0| 0 + 0| 0| | | | | | 0 (1 row) @@ -2230,18 +2230,18 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count --------+------------+---------+---------+---------+----------+----------+------------ - 0| 0| 0| 0| 0| 0| 0| 0 + 0| 0| | | | | | 0 (1 row) @@ -2256,11 +2256,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2295,11 +2295,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2334,11 +2334,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2389,11 +2389,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2421,11 +2421,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2483,11 +2483,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2515,11 +2515,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2578,11 +2578,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2647,11 +2647,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2708,11 +2708,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2775,11 +2775,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2815,18 +2815,18 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count --------+------------+---------+---------+---------+----------+----------+------------ - 3| 9| 5| 1| 0| 1| 1| 0 + 3| 9| 0| 0| 0| 0| 0| 0 (1 row) @@ -2861,18 +2861,18 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count --------+------------+---------+---------+---------+----------+----------+------------ - 3| 9| 5| 1| 0| 1| 1| 0 + 3| 9| 0| 0| 0| 0| 0| 0 (1 row) @@ -2901,11 +2901,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2947,11 +2947,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2988,11 +2988,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -3035,11 +3035,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' diff --git a/src/test/isolation/specs/stats.spec b/src/test/isolation/specs/stats.spec index da16710da0fb..6eadc90463f2 100644 --- a/src/test/isolation/specs/stats.spec +++ b/src/test/isolation/specs/stats.spec @@ -102,11 +102,11 @@ step s1_table_stats { SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index ffec0903f038..3e3ef4f189b2 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1825,15 +1825,15 @@ pg_stat_all_tables| SELECT c.oid AS relid, (sum(pg_stat_get_idx_numscans(i.indexrelid)))::bigint AS idx_scan, max(pg_stat_get_idx_lastscan(i.indexrelid)) AS last_idx_scan, ((sum(pg_stat_get_idx_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_tuples_fetched(c.oid)) AS idx_tup_fetch, - pg_stat_get_tuples_inserted(c.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(c.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(c.oid) AS n_tup_del, - pg_stat_get_tuples_hot_updated(c.oid) AS n_tup_hot_upd, - pg_stat_get_tuples_newpage_updated(c.oid) AS n_tup_newpage_upd, - pg_stat_get_live_tuples(c.oid) AS n_live_tup, - pg_stat_get_dead_tuples(c.oid) AS n_dead_tup, - pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze, - pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode((c.oid)::regclass)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode((c.oid)::regclass)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode((c.oid)::regclass)) AS n_tup_del, + pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode((c.oid)::regclass)) AS n_tup_hot_upd, + pg_stat_get_rfn_tuples_newpage_updated(pg_relation_filenode((c.oid)::regclass)) AS n_tup_newpage_upd, + pg_stat_get_rfn_live_tuples(pg_relation_filenode((c.oid)::regclass)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode((c.oid)::regclass)) AS n_dead_tup, + pg_stat_get_rfn_mod_since_analyze(pg_relation_filenode((c.oid)::regclass)) AS n_mod_since_analyze, + pg_stat_get_rfn_ins_since_vacuum(pg_relation_filenode((c.oid)::regclass)) AS n_ins_since_vacuum, pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum, pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum, pg_stat_get_last_analyze_time(c.oid) AS last_analyze, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index b44e30f557c9..f7d573b890aa 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -128,14 +128,15 @@ SELECT id, name, fixed_amount, 5 | replslot | f | t | t 6 | subscription | f | t | t 7 | backend | f | t | f - 8 | archiver | t | f | t - 9 | bgwriter | t | f | t - 10 | checkpointer | t | f | t - 11 | io | t | f | t - 12 | lock | t | f | t - 13 | slru | t | f | t - 14 | wal | t | f | t -(14 rows) + 8 | relfilenode | f | f | t + 9 | archiver | t | f | t + 10 | bgwriter | t | f | t + 11 | checkpointer | t | f | t + 12 | io | t | f | t + 13 | lock | t | f | t + 14 | slru | t | f | t + 15 | wal | t | f | t +(15 rows) -- ensure that both seqscan and indexscan plans are allowed SET enable_seqscan TO on; @@ -241,9 +242,9 @@ SELECT relname, n_tup_ins, n_tup_upd, n_tup_del, n_live_tup, n_dead_tup WHERE relname like 'trunc_stats_test%' order by relname; relname | n_tup_ins | n_tup_upd | n_tup_del | n_live_tup | n_dead_tup -------------------+-----------+-----------+-----------+------------+------------ - trunc_stats_test | 3 | 0 | 0 | 0 | 0 - trunc_stats_test1 | 4 | 2 | 1 | 1 | 0 - trunc_stats_test2 | 1 | 0 | 0 | 1 | 0 + trunc_stats_test | 0 | 0 | 0 | 0 | 0 + trunc_stats_test1 | 0 | 0 | 0 | 0 | 0 + trunc_stats_test2 | 0 | 0 | 0 | 0 | 0 trunc_stats_test3 | 4 | 0 | 0 | 2 | 2 trunc_stats_test4 | 2 | 0 | 0 | 0 | 2 (5 rows) @@ -470,30 +471,30 @@ SELECT pg_stat_get_function_calls(:stats_test_func2_oid); -- by oid after the DROP TABLE. Save oids. CREATE TABLE drop_stats_test(); INSERT INTO drop_stats_test DEFAULT VALUES; -SELECT 'drop_stats_test'::regclass::oid AS drop_stats_test_oid \gset +SELECT pg_relation_filenode('drop_stats_test'::regclass) AS drop_stats_test_oid \gset CREATE TABLE drop_stats_test_xact(); INSERT INTO drop_stats_test_xact DEFAULT VALUES; -SELECT 'drop_stats_test_xact'::regclass::oid AS drop_stats_test_xact_oid \gset +SELECT pg_relation_filenode('drop_stats_test_xact'::regclass) AS drop_stats_test_xact_oid \gset CREATE TABLE drop_stats_test_subxact(); INSERT INTO drop_stats_test_subxact DEFAULT VALUES; -SELECT 'drop_stats_test_subxact'::regclass::oid AS drop_stats_test_subxact_oid \gset +SELECT pg_relation_filenode('drop_stats_test_subxact'::regclass) AS drop_stats_test_subxact_oid \gset SELECT pg_stat_force_next_flush(); pg_stat_force_next_flush -------------------------- (1 row) -SELECT pg_stat_get_live_tuples(:drop_stats_test_oid); - pg_stat_get_live_tuples -------------------------- - 1 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) DROP TABLE drop_stats_test; -SELECT pg_stat_get_live_tuples(:drop_stats_test_oid); - pg_stat_get_live_tuples -------------------------- - 0 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_oid); @@ -504,18 +505,18 @@ SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_oid); -- check that rollback protects against having stats dropped and that local -- modifications don't pose a problem -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); - pg_stat_get_live_tuples -------------------------- - 1 -(1 row) - -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); - pg_stat_get_tuples_inserted +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); + pg_stat_get_rfn_live_tuples ----------------------------- 1 (1 row) +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); + pg_stat_get_rfn_tuples_inserted +--------------------------------- + 1 +(1 row) + SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); pg_stat_get_xact_tuples_inserted ---------------------------------- @@ -544,29 +545,29 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); - pg_stat_get_live_tuples -------------------------- - 1 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); - pg_stat_get_tuples_inserted ------------------------------ - 2 +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); + pg_stat_get_rfn_tuples_inserted +--------------------------------- + 2 (1 row) -- transactional drop -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); - pg_stat_get_live_tuples -------------------------- - 1 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); - pg_stat_get_tuples_inserted ------------------------------ - 2 +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); + pg_stat_get_rfn_tuples_inserted +--------------------------------- + 2 (1 row) BEGIN; @@ -591,23 +592,23 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); - pg_stat_get_live_tuples -------------------------- - 0 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); - pg_stat_get_tuples_inserted ------------------------------ - 0 +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); + pg_stat_get_rfn_tuples_inserted +--------------------------------- + 2 (1 row) -- savepoint rollback (2 levels) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 1 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) BEGIN; @@ -636,17 +637,17 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 3 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) -- savepoint rolback (1 level) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 3 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) BEGIN; @@ -655,17 +656,17 @@ DROP TABLE drop_stats_test_subxact; SAVEPOINT sp2; ROLLBACK TO SAVEPOINT sp1; COMMIT; -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 3 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) -- and now actually drop -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 3 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) BEGIN; @@ -674,10 +675,10 @@ DROP TABLE drop_stats_test_subxact; SAVEPOINT sp2; RELEASE SAVEPOINT sp1; COMMIT; -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 0 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) DROP TABLE trunc_stats_test, trunc_stats_test1, trunc_stats_test2, trunc_stats_test3, trunc_stats_test4; @@ -945,7 +946,7 @@ SELECT (n_tup_ins + n_tup_upd) > 0 AS has_data FROM pg_stat_all_tables WHERE relid = 'pg_shdescription'::regclass; has_data ---------- - t + f (1 row) -- stats_reset may not be set for datid=0 and shared objects in @@ -1902,7 +1903,7 @@ DECLARE BEGIN -- we don't want to wait forever; loop will exit after 30 seconds FOR i IN 1 .. 300 LOOP - SELECT (pg_stat_get_tuples_hot_updated('brin_hot'::regclass::oid) > 0) INTO updated; + SELECT (pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode('brin_hot'::regclass::oid)) > 0) INTO updated; EXIT WHEN updated; -- wait a little @@ -1929,10 +1930,10 @@ SELECT wait_for_hot_stats(); (1 row) -SELECT pg_stat_get_tuples_hot_updated('brin_hot'::regclass::oid); - pg_stat_get_tuples_hot_updated --------------------------------- - 1 +SELECT pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode('brin_hot'::regclass::oid)); + pg_stat_get_rfn_tuples_hot_updated +------------------------------------ + 1 (1 row) DROP TABLE brin_hot; diff --git a/src/test/regress/expected/stats_rewrite.out b/src/test/regress/expected/stats_rewrite.out index 93752bab9cb3..4f341672e5f1 100644 --- a/src/test/regress/expected/stats_rewrite.out +++ b/src/test/regress/expected/stats_rewrite.out @@ -42,7 +42,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_rewrite_alone'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 1 | 1 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_2pc_rewrite_alone; @@ -66,7 +66,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 4 | 4 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_2pc; @@ -93,7 +93,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_multi'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 6 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_2pc_multi; @@ -114,7 +114,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_rewrite_alone_abort'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 1 | 1 | 0 + 2 | 1 | 1 (1 row) DROP TABLE test_2pc_rewrite_alone_abort; @@ -138,7 +138,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_abort'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 4 | 1 | 3 + 7 | 1 | 6 (1 row) DROP TABLE test_2pc_abort; @@ -169,7 +169,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_savepoint'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 3 | 3 + 0 | 0 | 0 (1 row) DROP TABLE test_2pc_savepoint; @@ -209,7 +209,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_alone'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 1 | 1 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_alone; @@ -232,7 +232,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 4 | 4 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test; @@ -258,7 +258,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_multi'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 6 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_multi; @@ -331,7 +331,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_savepoint'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 3 | 3 + 0 | 0 | 0 (1 row) DROP TABLE test_savepoint; diff --git a/src/test/regress/expected/stats_rewrite_1.out b/src/test/regress/expected/stats_rewrite_1.out index 909188b18fd7..32c8cf803ebe 100644 --- a/src/test/regress/expected/stats_rewrite_1.out +++ b/src/test/regress/expected/stats_rewrite_1.out @@ -230,7 +230,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_alone'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 1 | 1 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_alone; @@ -253,7 +253,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 4 | 4 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test; @@ -279,7 +279,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_multi'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 6 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_multi; @@ -352,7 +352,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_savepoint'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 3 | 3 + 0 | 0 | 0 (1 row) DROP TABLE test_savepoint; diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index d4623c32cd32..9188429e03ca 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -219,27 +219,27 @@ SELECT pg_stat_get_function_calls(:stats_test_func2_oid); -- by oid after the DROP TABLE. Save oids. CREATE TABLE drop_stats_test(); INSERT INTO drop_stats_test DEFAULT VALUES; -SELECT 'drop_stats_test'::regclass::oid AS drop_stats_test_oid \gset +SELECT pg_relation_filenode('drop_stats_test'::regclass) AS drop_stats_test_oid \gset CREATE TABLE drop_stats_test_xact(); INSERT INTO drop_stats_test_xact DEFAULT VALUES; -SELECT 'drop_stats_test_xact'::regclass::oid AS drop_stats_test_xact_oid \gset +SELECT pg_relation_filenode('drop_stats_test_xact'::regclass) AS drop_stats_test_xact_oid \gset CREATE TABLE drop_stats_test_subxact(); INSERT INTO drop_stats_test_subxact DEFAULT VALUES; -SELECT 'drop_stats_test_subxact'::regclass::oid AS drop_stats_test_subxact_oid \gset +SELECT pg_relation_filenode('drop_stats_test_subxact'::regclass) AS drop_stats_test_subxact_oid \gset SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples(:drop_stats_test_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_oid); DROP TABLE drop_stats_test; -SELECT pg_stat_get_live_tuples(:drop_stats_test_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_oid); SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_oid); -- check that rollback protects against having stats dropped and that local -- modifications don't pose a problem -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); BEGIN; INSERT INTO drop_stats_test_xact DEFAULT VALUES; @@ -248,12 +248,12 @@ DROP TABLE drop_stats_test_xact; SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); ROLLBACK; SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); -- transactional drop -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); BEGIN; INSERT INTO drop_stats_test_xact DEFAULT VALUES; SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); @@ -261,11 +261,11 @@ DROP TABLE drop_stats_test_xact; SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); COMMIT; SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); -- savepoint rollback (2 levels) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); BEGIN; INSERT INTO drop_stats_test_subxact DEFAULT VALUES; SAVEPOINT sp1; @@ -277,27 +277,27 @@ ROLLBACK TO SAVEPOINT sp2; SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_subxact_oid); COMMIT; SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); -- savepoint rolback (1 level) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); BEGIN; SAVEPOINT sp1; DROP TABLE drop_stats_test_subxact; SAVEPOINT sp2; ROLLBACK TO SAVEPOINT sp1; COMMIT; -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); -- and now actually drop -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); BEGIN; SAVEPOINT sp1; DROP TABLE drop_stats_test_subxact; SAVEPOINT sp2; RELEASE SAVEPOINT sp1; COMMIT; -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); DROP TABLE trunc_stats_test, trunc_stats_test1, trunc_stats_test2, trunc_stats_test3, trunc_stats_test4; DROP TABLE prevstats; @@ -898,7 +898,7 @@ DECLARE BEGIN -- we don't want to wait forever; loop will exit after 30 seconds FOR i IN 1 .. 300 LOOP - SELECT (pg_stat_get_tuples_hot_updated('brin_hot'::regclass::oid) > 0) INTO updated; + SELECT (pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode('brin_hot'::regclass::oid)) > 0) INTO updated; EXIT WHEN updated; -- wait a little @@ -923,7 +923,7 @@ UPDATE brin_hot SET val = -3 WHERE id = 42; \c - SELECT wait_for_hot_stats(); -SELECT pg_stat_get_tuples_hot_updated('brin_hot'::regclass::oid); +SELECT pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode('brin_hot'::regclass::oid)); DROP TABLE brin_hot; DROP FUNCTION wait_for_hot_stats(); diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index 5ebae1cedc2f..eed714f7d588 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -12988,12 +12988,12 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples('public.simport_ftable'::regclass), - pg_stat_get_dead_tuples('public.simport_ftable'::regclass), +SELECT pg_stat_get_rfn_live_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), + pg_stat_get_rfn_dead_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), pg_stat_get_analyze_count('public.simport_ftable'::regclass); - pg_stat_get_live_tuples | pg_stat_get_dead_tuples | pg_stat_get_analyze_count --------------------------+-------------------------+--------------------------- - 0 | 0 | 1 + pg_stat_get_rfn_live_tuples | pg_stat_get_rfn_dead_tuples | pg_stat_get_analyze_count +-----------------------------+-----------------------------+--------------------------- + | | 1 (1 row) ALTER TABLE simport_table ALTER COLUMN c1 SET STATISTICS 0; @@ -13023,12 +13023,12 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples('public.simport_ftable'::regclass), - pg_stat_get_dead_tuples('public.simport_ftable'::regclass), +SELECT pg_stat_get_rfn_live_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), + pg_stat_get_rfn_dead_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), pg_stat_get_analyze_count('public.simport_ftable'::regclass); - pg_stat_get_live_tuples | pg_stat_get_dead_tuples | pg_stat_get_analyze_count --------------------------+-------------------------+--------------------------- - 4 | 0 | 1 + pg_stat_get_rfn_live_tuples | pg_stat_get_rfn_dead_tuples | pg_stat_get_analyze_count +-----------------------------+-----------------------------+--------------------------- + | | 1 (1 row) ANALYZE VERBOSE simport_ftable (c1); -- should work diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql index e868da00ace7..a3be2f260280 100644 --- a/contrib/postgres_fdw/sql/postgres_fdw.sql +++ b/contrib/postgres_fdw/sql/postgres_fdw.sql @@ -4590,8 +4590,8 @@ ANALYZE simport_table; SELECT pg_stat_reset_single_table_counters('public.simport_ftable'::regclass); ANALYZE VERBOSE simport_ftable; -- should work SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples('public.simport_ftable'::regclass), - pg_stat_get_dead_tuples('public.simport_ftable'::regclass), +SELECT pg_stat_get_rfn_live_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), + pg_stat_get_rfn_dead_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), pg_stat_get_analyze_count('public.simport_ftable'::regclass); ALTER TABLE simport_table ALTER COLUMN c1 SET STATISTICS 0; @@ -4612,8 +4612,8 @@ ANALYZE simport_table; SELECT pg_stat_reset_single_table_counters('public.simport_ftable'::regclass); ANALYZE VERBOSE simport_ftable; -- should work SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples('public.simport_ftable'::regclass), - pg_stat_get_dead_tuples('public.simport_ftable'::regclass), +SELECT pg_stat_get_rfn_live_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), + pg_stat_get_rfn_dead_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), pg_stat_get_analyze_count('public.simport_ftable'::regclass); ANALYZE VERBOSE simport_ftable (c1); -- should work -- 2.55.0 Attachments: [text/plain] v13-0001-Split-PgStat_StatTabEntry-into-separate-table-an.patch (46.5K, ../../[email protected]/2-v13-0001-Split-PgStat_StatTabEntry-into-separate-table-an.patch) download | inline diff: From 0fd23ab6f58fc43a9086975d9d2d5a3ad67c466b Mon Sep 17 00:00:00 2001 From: Michael Paquier <[email protected]> Date: Fri, 10 Jul 2026 12:52:03 +0900 Subject: [PATCH v13 1/3] Split PgStat_StatTabEntry into separate table and index structures Table and index stats structures are now split, with PgStat_StatIdxEntry containing only the fields relevant to index statistics. Index entries do not care about most of the fields saved for tables, so this saves up to 200 bytes for each stats entry in shared memory. A new stats kind called PGSTAT_KIND_INDEX is created, with the fetch functions for the fields, that are plugged into the system views. XXX: Bump of PGSTAT_FILE_FORMAT_ID required. --- src/include/catalog/pg_proc.dat | 59 ++++++++- src/include/pgstat.h | 20 +++ src/include/utils/pgstat_internal.h | 15 +++ src/include/utils/pgstat_kind.h | 23 ++-- src/backend/catalog/system_views.sql | 40 +++--- src/backend/utils/activity/Makefile | 1 + src/backend/utils/activity/meson.build | 1 + src/backend/utils/activity/pgstat.c | 16 +++ src/backend/utils/activity/pgstat_index.c | 129 +++++++++++++++++++ src/backend/utils/activity/pgstat_relation.c | 106 +++++++++++---- src/backend/utils/adt/pgstatfuncs.c | 108 ++++++++++++++++ src/test/regress/expected/rules.out | 34 ++--- src/test/regress/expected/stats.out | 49 +++---- src/test/regress/sql/stats.sql | 22 ++-- doc/src/sgml/monitoring.sgml | 20 ++- 15 files changed, 531 insertions(+), 112 deletions(-) create mode 100644 src/backend/utils/activity/pgstat_index.c diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 3cb84359adf0..dc6c61e84e54 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5558,11 +5558,11 @@ proargnames => '{mcv_list,index,values,nulls,frequency,base_frequency}', prosrc => 'pg_stats_ext_mcvlist_items' }, -{ oid => '1928', descr => 'statistics: number of scans done for table/index', +{ oid => '1928', descr => 'statistics: number of scans done for table', proname => 'pg_stat_get_numscans', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_numscans' }, -{ oid => '6310', descr => 'statistics: time of the last scan for table/index', +{ oid => '6310', descr => 'statistics: time of the last scan for table', proname => 'pg_stat_get_lastscan', provolatile => 's', proparallel => 'r', prorettype => 'timestamptz', proargtypes => 'oid', prosrc => 'pg_stat_get_lastscan' }, @@ -6237,10 +6237,63 @@ proargdefaults => '{NULL}', prosrc => 'pg_stat_reset_shared', proacl => '{POSTGRES=X}' }, { oid => '3776', - descr => 'statistics: reset collected statistics for a single table or index in the current database or shared across all databases in the cluster', + descr => 'statistics: reset collected statistics for a single table in the current database or shared across all databases in the cluster', proname => 'pg_stat_reset_single_table_counters', provolatile => 'v', prorettype => 'void', proargtypes => 'oid', prosrc => 'pg_stat_reset_single_table_counters', proacl => '{POSTGRES=X}' }, +{ oid => '8450', + descr => 'statistics: reset collected statistics for a single index in the current database or shared across all databases in the cluster', + proname => 'pg_stat_reset_single_index_counters', provolatile => 'v', + prorettype => 'void', proargtypes => 'oid', + prosrc => 'pg_stat_reset_single_index_counters', proacl => '{POSTGRES=X}' }, +{ oid => '8451', descr => 'statistics: number of scans done for index', + proname => 'pg_stat_get_idx_numscans', provolatile => 's', proparallel => 'r', + prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_numscans' }, +{ oid => '8452', descr => 'statistics: number of index entries returned by index', + proname => 'pg_stat_get_idx_tuples_returned', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_tuples_returned' }, +{ oid => '8453', descr => 'statistics: number of tuples fetched from table by index', + proname => 'pg_stat_get_idx_tuples_fetched', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_tuples_fetched' }, +{ oid => '8454', descr => 'statistics: number of blocks fetched for index', + proname => 'pg_stat_get_idx_blocks_fetched', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_blocks_fetched' }, +{ oid => '8455', descr => 'statistics: number of blocks found in cache for index', + proname => 'pg_stat_get_idx_blocks_hit', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_blocks_hit' }, +{ oid => '8456', descr => 'statistics: time of the last scan for index', + proname => 'pg_stat_get_idx_lastscan', provolatile => 's', proparallel => 'r', + prorettype => 'timestamptz', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_lastscan' }, +{ oid => '8457', descr => 'statistics: last reset for an index', + proname => 'pg_stat_get_idx_stat_reset_time', provolatile => 's', + proparallel => 'r', prorettype => 'timestamptz', proargtypes => 'oid', + prosrc => 'pg_stat_get_idx_stat_reset_time' }, +{ oid => '8458', descr => 'statistics: number of scans done for index in current transaction', + proname => 'pg_stat_get_xact_idx_numscans', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_numscans' }, +{ oid => '8459', descr => 'statistics: number of index entries returned in current transaction', + proname => 'pg_stat_get_xact_idx_tuples_returned', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_tuples_returned' }, +{ oid => '8460', descr => 'statistics: number of tuples fetched by index in current transaction', + proname => 'pg_stat_get_xact_idx_tuples_fetched', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_tuples_fetched' }, +{ oid => '8461', descr => 'statistics: number of blocks fetched for index in current transaction', + proname => 'pg_stat_get_xact_idx_blocks_fetched', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_blocks_fetched' }, +{ oid => '8462', descr => 'statistics: number of blocks found in cache for index in current transaction', + proname => 'pg_stat_get_xact_idx_blocks_hit', provolatile => 'v', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_xact_idx_blocks_hit' }, { oid => '3777', descr => 'statistics: reset collected statistics for a single function in the current database', proname => 'pg_stat_reset_single_function_counters', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 58a44857f131..23b85fc0c223 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -487,6 +487,20 @@ typedef struct PgStat_StatTabEntry TimestampTz stat_reset_time; } PgStat_StatTabEntry; +typedef struct PgStat_StatIdxEntry +{ + PgStat_Counter numscans; + TimestampTz lastscan; + + PgStat_Counter tuples_returned; + PgStat_Counter tuples_fetched; + + PgStat_Counter blocks_fetched; + PgStat_Counter blocks_hit; + + TimestampTz stat_reset_time; +} PgStat_StatIdxEntry; + /* ------ * PgStat_WalCounters WAL activity data gathered from WalUsage * @@ -779,6 +793,12 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free); extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id); +extern PgStat_TableStatus *find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id); + +extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry(Oid relid); +extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry_ext(bool shared, + Oid reloid, + bool *may_free); /* diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h index b3dc3ff7d8bb..94c94c103feb 100644 --- a/src/include/utils/pgstat_internal.h +++ b/src/include/utils/pgstat_internal.h @@ -507,6 +507,12 @@ typedef struct PgStatShared_Relation PgStat_StatTabEntry stats; } PgStatShared_Relation; +typedef struct PgStatShared_Index +{ + PgStatShared_Common header; + PgStat_StatIdxEntry stats; +} PgStatShared_Index; + typedef struct PgStatShared_Function { PgStatShared_Common header; @@ -787,6 +793,15 @@ extern void pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref); extern void pgstat_relation_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); +/* + * Functions in pgstat_index.c + */ + +extern bool pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait); +extern void pgstat_index_delete_pending_cb(PgStat_EntryRef *entry_ref); +extern void pgstat_index_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); + + /* * Functions in pgstat_replslot.c */ diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h index 2d78a0296834..45ca599d0dd6 100644 --- a/src/include/utils/pgstat_kind.h +++ b/src/include/utils/pgstat_kind.h @@ -26,19 +26,20 @@ /* stats for variable-numbered objects */ #define PGSTAT_KIND_DATABASE 1 /* database-wide statistics */ #define PGSTAT_KIND_RELATION 2 /* per-table statistics */ -#define PGSTAT_KIND_FUNCTION 3 /* per-function statistics */ -#define PGSTAT_KIND_REPLSLOT 4 /* per-slot statistics */ -#define PGSTAT_KIND_SUBSCRIPTION 5 /* per-subscription statistics */ -#define PGSTAT_KIND_BACKEND 6 /* per-backend statistics */ +#define PGSTAT_KIND_INDEX 3 /* per-index statistics */ +#define PGSTAT_KIND_FUNCTION 4 /* per-function statistics */ +#define PGSTAT_KIND_REPLSLOT 5 /* per-slot statistics */ +#define PGSTAT_KIND_SUBSCRIPTION 6 /* per-subscription statistics */ +#define PGSTAT_KIND_BACKEND 7 /* per-backend statistics */ /* stats for fixed-numbered objects */ -#define PGSTAT_KIND_ARCHIVER 7 -#define PGSTAT_KIND_BGWRITER 8 -#define PGSTAT_KIND_CHECKPOINTER 9 -#define PGSTAT_KIND_IO 10 -#define PGSTAT_KIND_LOCK 11 -#define PGSTAT_KIND_SLRU 12 -#define PGSTAT_KIND_WAL 13 +#define PGSTAT_KIND_ARCHIVER 8 +#define PGSTAT_KIND_BGWRITER 9 +#define PGSTAT_KIND_CHECKPOINTER 10 +#define PGSTAT_KIND_IO 11 +#define PGSTAT_KIND_LOCK 12 +#define PGSTAT_KIND_SLRU 13 +#define PGSTAT_KIND_WAL 14 #define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE #define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index 6c1c5545cb56..ce991e4269e7 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -722,9 +722,9 @@ CREATE VIEW pg_stat_all_tables AS pg_stat_get_numscans(C.oid) AS seq_scan, pg_stat_get_lastscan(C.oid) AS last_seq_scan, pg_stat_get_tuples_returned(C.oid) AS seq_tup_read, - sum(pg_stat_get_numscans(I.indexrelid))::bigint AS idx_scan, - max(pg_stat_get_lastscan(I.indexrelid)) AS last_idx_scan, - sum(pg_stat_get_tuples_fetched(I.indexrelid))::bigint + + sum(pg_stat_get_idx_numscans(I.indexrelid))::bigint AS idx_scan, + max(pg_stat_get_idx_lastscan(I.indexrelid)) AS last_idx_scan, + sum(pg_stat_get_idx_tuples_fetched(I.indexrelid))::bigint + pg_stat_get_tuples_fetched(C.oid) AS idx_tup_fetch, pg_stat_get_tuples_inserted(C.oid) AS n_tup_ins, pg_stat_get_tuples_updated(C.oid) AS n_tup_upd, @@ -761,8 +761,8 @@ CREATE VIEW pg_stat_xact_all_tables AS C.relname AS relname, pg_stat_get_xact_numscans(C.oid) AS seq_scan, pg_stat_get_xact_tuples_returned(C.oid) AS seq_tup_read, - sum(pg_stat_get_xact_numscans(I.indexrelid))::bigint AS idx_scan, - sum(pg_stat_get_xact_tuples_fetched(I.indexrelid))::bigint + + sum(pg_stat_get_xact_idx_numscans(I.indexrelid))::bigint AS idx_scan, + sum(pg_stat_get_xact_idx_tuples_fetched(I.indexrelid))::bigint + pg_stat_get_xact_tuples_fetched(C.oid) AS idx_tup_fetch, pg_stat_get_xact_tuples_inserted(C.oid) AS n_tup_ins, pg_stat_get_xact_tuples_updated(C.oid) AS n_tup_upd, @@ -833,17 +833,17 @@ CREATE VIEW pg_statio_all_tables AS pg_class T ON C.reltoastrelid = T.oid LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) LEFT JOIN LATERAL ( - SELECT sum(pg_stat_get_blocks_fetched(indexrelid) - - pg_stat_get_blocks_hit(indexrelid))::bigint + SELECT sum(pg_stat_get_idx_blocks_fetched(indexrelid) - + pg_stat_get_idx_blocks_hit(indexrelid))::bigint AS idx_blks_read, - sum(pg_stat_get_blocks_hit(indexrelid))::bigint + sum(pg_stat_get_idx_blocks_hit(indexrelid))::bigint AS idx_blks_hit FROM pg_index WHERE indrelid = C.oid ) I ON true LEFT JOIN LATERAL ( - SELECT sum(pg_stat_get_blocks_fetched(indexrelid) - - pg_stat_get_blocks_hit(indexrelid))::bigint + SELECT sum(pg_stat_get_idx_blocks_fetched(indexrelid) - + pg_stat_get_idx_blocks_hit(indexrelid))::bigint AS idx_blks_read, - sum(pg_stat_get_blocks_hit(indexrelid))::bigint + sum(pg_stat_get_idx_blocks_hit(indexrelid))::bigint AS idx_blks_hit FROM pg_index WHERE indrelid = T.oid ) X ON true WHERE C.relkind IN ('r', 't', 'm'); @@ -865,11 +865,11 @@ CREATE VIEW pg_stat_all_indexes AS N.nspname AS schemaname, C.relname AS relname, I.relname AS indexrelname, - pg_stat_get_numscans(I.oid) AS idx_scan, - pg_stat_get_lastscan(I.oid) AS last_idx_scan, - pg_stat_get_tuples_returned(I.oid) AS idx_tup_read, - pg_stat_get_tuples_fetched(I.oid) AS idx_tup_fetch, - pg_stat_get_stat_reset_time(I.oid) AS stats_reset + pg_stat_get_idx_numscans(I.oid) AS idx_scan, + pg_stat_get_idx_lastscan(I.oid) AS last_idx_scan, + pg_stat_get_idx_tuples_returned(I.oid) AS idx_tup_read, + pg_stat_get_idx_tuples_fetched(I.oid) AS idx_tup_fetch, + pg_stat_get_idx_stat_reset_time(I.oid) AS stats_reset FROM pg_class C JOIN pg_index X ON C.oid = X.indrelid JOIN pg_class I ON I.oid = X.indexrelid @@ -893,10 +893,10 @@ CREATE VIEW pg_statio_all_indexes AS N.nspname AS schemaname, C.relname AS relname, I.relname AS indexrelname, - pg_stat_get_blocks_fetched(I.oid) - - pg_stat_get_blocks_hit(I.oid) AS idx_blks_read, - pg_stat_get_blocks_hit(I.oid) AS idx_blks_hit, - pg_stat_get_stat_reset_time(I.oid) AS stats_reset + pg_stat_get_idx_blocks_fetched(I.oid) - + pg_stat_get_idx_blocks_hit(I.oid) AS idx_blks_read, + pg_stat_get_idx_blocks_hit(I.oid) AS idx_blks_hit, + pg_stat_get_idx_stat_reset_time(I.oid) AS stats_reset FROM pg_class C JOIN pg_index X ON C.oid = X.indrelid JOIN pg_class I ON I.oid = X.indexrelid diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile index 5fed953c28a7..2e32d1485d64 100644 --- a/src/backend/utils/activity/Makefile +++ b/src/backend/utils/activity/Makefile @@ -25,6 +25,7 @@ OBJS = \ pgstat_checkpointer.o \ pgstat_database.o \ pgstat_function.o \ + pgstat_index.o \ pgstat_io.o \ pgstat_kind.o \ pgstat_lock.o \ diff --git a/src/backend/utils/activity/meson.build b/src/backend/utils/activity/meson.build index 470b5dac402b..e6dcb2e26fc5 100644 --- a/src/backend/utils/activity/meson.build +++ b/src/backend/utils/activity/meson.build @@ -10,6 +10,7 @@ backend_sources += files( 'pgstat_checkpointer.c', 'pgstat_database.c', 'pgstat_function.c', + 'pgstat_index.c', 'pgstat_io.c', 'pgstat_kind.c', 'pgstat_lock.c', diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 8b2a5ec36750..3aa10162f8ad 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -317,6 +317,22 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .reset_timestamp_cb = pgstat_relation_reset_timestamp_cb, }, + [PGSTAT_KIND_INDEX] = { + .name = "index", + + .fixed_amount = false, + .write_to_file = true, + + .shared_size = sizeof(PgStatShared_Index), + .shared_data_off = offsetof(PgStatShared_Index, stats), + .shared_data_len = sizeof(((PgStatShared_Index *) 0)->stats), + .pending_size = sizeof(PgStat_TableStatus), + + .flush_pending_cb = pgstat_index_flush_cb, + .delete_pending_cb = pgstat_index_delete_pending_cb, + .reset_timestamp_cb = pgstat_index_reset_timestamp_cb, + }, + [PGSTAT_KIND_FUNCTION] = { .name = "function", diff --git a/src/backend/utils/activity/pgstat_index.c b/src/backend/utils/activity/pgstat_index.c new file mode 100644 index 000000000000..e9fc27f2e4b4 --- /dev/null +++ b/src/backend/utils/activity/pgstat_index.c @@ -0,0 +1,129 @@ +/* ------------------------------------------------------------------------- + * + * pgstat_index.c + * Implementation of index statistics. + * + * This file contains the implementation of index statistics. + * + * Copyright (c) 2001-2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/backend/utils/activity/pgstat_index.c + * ------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "access/xact.h" +#include "catalog/catalog.h" +#include "utils/memutils.h" +#include "utils/pgstat_internal.h" +#include "utils/rel.h" +#include "utils/timestamp.h" + + +/* + * Flush out pending stats for an index entry. + * + * If nowait is true and the lock could not be immediately acquired, returns + * false without flushing the entry. Otherwise returns true. + * + * Some of the stats are copied to the corresponding pending database stats + * entry when successfully flushing. + */ +bool +pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) +{ + Oid dboid; + PgStat_TableStatus *lstats; /* pending stats entry */ + PgStatShared_Index *shidxstats; + PgStat_StatIdxEntry *idxentry; /* index entry of shared stats */ + PgStat_StatDBEntry *dbentry; /* pending database entry */ + + dboid = entry_ref->shared_entry->key.dboid; + lstats = (PgStat_TableStatus *) entry_ref->pending; + shidxstats = (PgStatShared_Index *) entry_ref->shared_stats; + + /* + * Ignore entries that didn't accumulate any actual counts, such as + * indexes that were opened by the planner but not used. + */ + if (pg_memory_is_all_zeros(&lstats->counts, + sizeof(struct PgStat_TableCounts))) + return true; + + if (!pgstat_lock_entry(entry_ref, nowait)) + return false; + + /* Add the values to the shared entry. */ + idxentry = &shidxstats->stats; + + idxentry->numscans += lstats->counts.numscans; + if (lstats->counts.numscans) + { + TimestampTz t = GetCurrentTransactionStopTimestamp(); + + if (t > idxentry->lastscan) + idxentry->lastscan = t; + } + idxentry->tuples_returned += lstats->counts.tuples_returned; + idxentry->tuples_fetched += lstats->counts.tuples_fetched; + idxentry->blocks_fetched += lstats->counts.blocks_fetched; + idxentry->blocks_hit += lstats->counts.blocks_hit; + + pgstat_unlock_entry(entry_ref); + + /* The entry was successfully flushed, add the same to database stats */ + dbentry = pgstat_prep_database_pending(dboid); + dbentry->blocks_fetched += lstats->counts.blocks_fetched; + dbentry->blocks_hit += lstats->counts.blocks_hit; + + return true; +} + +/* + * Callback to delete pending index stats. + */ +void +pgstat_index_delete_pending_cb(PgStat_EntryRef *entry_ref) +{ + PgStat_TableStatus *pending = (PgStat_TableStatus *) entry_ref->pending; + + if (pending->relation) + pgstat_unlink_relation(pending->relation); +} + +/* + * Callback to reset the timestamp on an index stats entry. + */ +void +pgstat_index_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts) +{ + ((PgStatShared_Index *) header)->stats.stat_reset_time = ts; +} + +/* + * Support function for the SQL-callable pgstat* functions. Returns + * the collected statistics for one index or NULL. NULL doesn't mean + * that the index doesn't exist, just that there are no statistics, so the + * caller is better off to report ZERO instead. + */ +PgStat_StatIdxEntry * +pgstat_fetch_stat_idxentry(Oid relid) +{ + return pgstat_fetch_stat_idxentry_ext(IsSharedRelation(relid), relid, NULL); +} + +/* + * More efficient version of pgstat_fetch_stat_idxentry(), allowing to specify + * whether the to-be-accessed index is a shared relation or not. This version + * also returns whether the caller can pfree() the result if desired. + */ +PgStat_StatIdxEntry * +pgstat_fetch_stat_idxentry_ext(bool shared, Oid reloid, bool *may_free) +{ + Oid dboid = (shared ? InvalidOid : MyDatabaseId); + + return (PgStat_StatIdxEntry *) + pgstat_fetch_entry(PGSTAT_KIND_INDEX, dboid, reloid, may_free); +} diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 04f2eb21d0bb..379a65aa28fb 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -42,12 +42,24 @@ typedef struct TwoPhasePgStatRecord } TwoPhasePgStatRecord; -static PgStat_TableStatus *pgstat_prep_relation_pending(Oid rel_id, bool isshared); +static PgStat_TableStatus *pgstat_prep_relation_pending(PgStat_Kind kind, + Oid rel_id, bool isshared); static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level); static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info); static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop); static void restore_truncdrop_counters(PgStat_TableXactStatus *trans); +/* + * Determine the stats kind for a relation based on its relkind. + */ +static inline PgStat_Kind +pgstat_get_relation_kind(char relkind) +{ + if (relkind == RELKIND_INDEX) + return PGSTAT_KIND_INDEX; + return PGSTAT_KIND_RELATION; +} + /* * Copy stats between relations. This is used for things like REINDEX @@ -56,25 +68,52 @@ static void restore_truncdrop_counters(PgStat_TableXactStatus *trans); void pgstat_copy_relation_stats(Relation dst, Relation src) { - PgStat_StatTabEntry *srcstats; - PgStatShared_Relation *dstshstats; - PgStat_EntryRef *dst_ref; + PgStat_Kind kind = pgstat_get_relation_kind(src->rd_rel->relkind); - srcstats = pgstat_fetch_stat_tabentry_ext(src->rd_rel->relisshared, - RelationGetRelid(src), - NULL); - if (!srcstats) - return; + if (kind == PGSTAT_KIND_INDEX) + { + PgStat_StatIdxEntry *srcstats; + PgStatShared_Index *dstshstats; + PgStat_EntryRef *dst_ref; - dst_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION, - dst->rd_rel->relisshared ? InvalidOid : MyDatabaseId, - RelationGetRelid(dst), - false); + srcstats = pgstat_fetch_stat_idxentry_ext(src->rd_rel->relisshared, + RelationGetRelid(src), + NULL); + if (!srcstats) + return; - dstshstats = (PgStatShared_Relation *) dst_ref->shared_stats; - dstshstats->stats = *srcstats; + dst_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_INDEX, + dst->rd_rel->relisshared ? InvalidOid : MyDatabaseId, + RelationGetRelid(dst), + false); - pgstat_unlock_entry(dst_ref); + dstshstats = (PgStatShared_Index *) dst_ref->shared_stats; + dstshstats->stats = *srcstats; + + pgstat_unlock_entry(dst_ref); + } + else + { + PgStat_StatTabEntry *srcstats; + PgStatShared_Relation *dstshstats; + PgStat_EntryRef *dst_ref; + + srcstats = pgstat_fetch_stat_tabentry_ext(src->rd_rel->relisshared, + RelationGetRelid(src), + NULL); + if (!srcstats) + return; + + dst_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION, + dst->rd_rel->relisshared ? InvalidOid : MyDatabaseId, + RelationGetRelid(dst), + false); + + dstshstats = (PgStatShared_Relation *) dst_ref->shared_stats; + dstshstats->stats = *srcstats; + + pgstat_unlock_entry(dst_ref); + } } /* @@ -131,11 +170,16 @@ pgstat_init_relation(Relation rel) void pgstat_assoc_relation(Relation rel) { + PgStat_Kind kind; + Assert(rel->pgstat_enabled); Assert(rel->pgstat_info == NULL); + kind = pgstat_get_relation_kind(rel->rd_rel->relkind); + /* find or make the PgStat_TableStatus entry, and update link */ - rel->pgstat_info = pgstat_prep_relation_pending(RelationGetRelid(rel), + rel->pgstat_info = pgstat_prep_relation_pending(kind, + RelationGetRelid(rel), rel->rd_rel->relisshared); /* don't allow link a stats to multiple relcache entries */ @@ -168,7 +212,9 @@ pgstat_unlink_relation(Relation rel) void pgstat_create_relation(Relation rel) { - pgstat_create_transactional(PGSTAT_KIND_RELATION, + PgStat_Kind kind = pgstat_get_relation_kind(rel->rd_rel->relkind); + + pgstat_create_transactional(kind, rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId, RelationGetRelid(rel)); } @@ -181,8 +227,9 @@ pgstat_drop_relation(Relation rel) { int nest_level = GetCurrentTransactionNestLevel(); PgStat_TableStatus *pgstat_info; + PgStat_Kind kind = pgstat_get_relation_kind(rel->rd_rel->relkind); - pgstat_drop_transactional(PGSTAT_KIND_RELATION, + pgstat_drop_transactional(kind, rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId, RelationGetRelid(rel)); @@ -500,16 +547,25 @@ pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free) */ PgStat_TableStatus * find_tabstat_entry(Oid rel_id) +{ + return find_tabstat_entry_kind(PGSTAT_KIND_RELATION, rel_id); +} + +/* + * Same as find_tabstat_entry but for a specific stats kind. + */ +PgStat_TableStatus * +find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id) { PgStat_EntryRef *entry_ref; PgStat_TableXactStatus *trans; PgStat_TableStatus *tabentry = NULL; PgStat_TableStatus *tablestatus = NULL; - entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, MyDatabaseId, rel_id); + entry_ref = pgstat_fetch_pending_entry(kind, MyDatabaseId, rel_id); if (!entry_ref) { - entry_ref = pgstat_fetch_pending_entry(PGSTAT_KIND_RELATION, InvalidOid, rel_id); + entry_ref = pgstat_fetch_pending_entry(kind, InvalidOid, rel_id); if (!entry_ref) return tablestatus; } @@ -752,7 +808,7 @@ pgstat_twophase_postcommit(FullTransactionId fxid, uint16 info, PgStat_TableStatus *pgstat_info; /* Find or create a tabstat entry for the rel */ - pgstat_info = pgstat_prep_relation_pending(rec->id, rec->shared); + pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); /* Same math as in AtEOXact_PgStat, commit case */ pgstat_info->counts.tuples_inserted += rec->tuples_inserted; @@ -788,7 +844,7 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, PgStat_TableStatus *pgstat_info; /* Find or create a tabstat entry for the rel */ - pgstat_info = pgstat_prep_relation_pending(rec->id, rec->shared); + pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); /* Same math as in AtEOXact_PgStat, abort case */ if (rec->truncdropped) @@ -922,12 +978,12 @@ pgstat_relation_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts) * initialized if not exists. */ static PgStat_TableStatus * -pgstat_prep_relation_pending(Oid rel_id, bool isshared) +pgstat_prep_relation_pending(PgStat_Kind kind, Oid rel_id, bool isshared) { PgStat_EntryRef *entry_ref; PgStat_TableStatus *pending; - entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_RELATION, + entry_ref = pgstat_prep_pending_entry(kind, isshared ? InvalidOid : MyDatabaseId, rel_id, NULL); pending = entry_ref->pending; diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 565d0e70768b..946025f39ed0 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -108,6 +108,40 @@ PG_STAT_GET_RELENTRY_INT64(tuples_updated) /* pg_stat_get_vacuum_count */ PG_STAT_GET_RELENTRY_INT64(vacuum_count) +/* + * Accessor macro for index stats entries (PgStat_StatIdxEntry). + */ +#define PG_STAT_GET_IDXENTRY_INT64(stat) \ +Datum \ +CppConcat(pg_stat_get_idx_,stat)(PG_FUNCTION_ARGS) \ +{ \ + Oid relid = PG_GETARG_OID(0); \ + int64 result; \ + PgStat_StatIdxEntry *idxentry; \ + \ + if ((idxentry = pgstat_fetch_stat_idxentry(relid)) == NULL) \ + result = 0; \ + else \ + result = (int64) (idxentry->stat); \ + \ + PG_RETURN_INT64(result); \ +} + +/* pg_stat_get_idx_numscans */ +PG_STAT_GET_IDXENTRY_INT64(numscans) + +/* pg_stat_get_idx_tuples_returned */ +PG_STAT_GET_IDXENTRY_INT64(tuples_returned) + +/* pg_stat_get_idx_tuples_fetched */ +PG_STAT_GET_IDXENTRY_INT64(tuples_fetched) + +/* pg_stat_get_idx_blocks_fetched */ +PG_STAT_GET_IDXENTRY_INT64(blocks_fetched) + +/* pg_stat_get_idx_blocks_hit */ +PG_STAT_GET_IDXENTRY_INT64(blocks_hit) + #define PG_STAT_GET_RELENTRY_FLOAT8(stat) \ Datum \ CppConcat(pg_stat_get_,stat)(PG_FUNCTION_ARGS) \ @@ -173,6 +207,34 @@ PG_STAT_GET_RELENTRY_TIMESTAMPTZ(lastscan) /* pg_stat_get_stat_reset_time */ PG_STAT_GET_RELENTRY_TIMESTAMPTZ(stat_reset_time) +/* + * Accessor macro for index timestamp fields. + */ +#define PG_STAT_GET_IDXENTRY_TIMESTAMPTZ(stat) \ +Datum \ +CppConcat(pg_stat_get_idx_,stat)(PG_FUNCTION_ARGS) \ +{ \ + Oid relid = PG_GETARG_OID(0); \ + TimestampTz result; \ + PgStat_StatIdxEntry *idxentry; \ + \ + if ((idxentry = pgstat_fetch_stat_idxentry(relid)) == NULL) \ + result = 0; \ + else \ + result = idxentry->stat; \ + \ + if (result == 0) \ + PG_RETURN_NULL(); \ + else \ + PG_RETURN_TIMESTAMPTZ(result); \ +} + +/* pg_stat_get_idx_lastscan */ +PG_STAT_GET_IDXENTRY_TIMESTAMPTZ(lastscan) + +/* pg_stat_get_idx_stat_reset_time */ +PG_STAT_GET_IDXENTRY_TIMESTAMPTZ(stat_reset_time) + Datum pg_stat_get_function_calls(PG_FUNCTION_ARGS) { @@ -1906,6 +1968,41 @@ PG_STAT_GET_XACT_RELENTRY_INT64(tuples_updated) /* pg_stat_get_xact_tuples_deleted */ PG_STAT_GET_XACT_RELENTRY_INT64(tuples_deleted) +/* + * Accessor macro for in-transaction index stats. + */ +#define PG_STAT_GET_XACT_IDXENTRY_INT64(stat) \ +Datum \ +CppConcat(pg_stat_get_xact_idx_,stat)(PG_FUNCTION_ARGS) \ +{ \ + Oid relid = PG_GETARG_OID(0); \ + int64 result; \ + PgStat_TableStatus *tabentry; \ + \ + tabentry = find_tabstat_entry_kind(PGSTAT_KIND_INDEX, relid); \ + if (!tabentry) \ + result = 0; \ + else \ + result = (int64) (tabentry->counts.stat); \ + \ + PG_RETURN_INT64(result); \ +} + +/* pg_stat_get_xact_idx_numscans */ +PG_STAT_GET_XACT_IDXENTRY_INT64(numscans) + +/* pg_stat_get_xact_idx_tuples_returned */ +PG_STAT_GET_XACT_IDXENTRY_INT64(tuples_returned) + +/* pg_stat_get_xact_idx_tuples_fetched */ +PG_STAT_GET_XACT_IDXENTRY_INT64(tuples_fetched) + +/* pg_stat_get_xact_idx_blocks_fetched */ +PG_STAT_GET_XACT_IDXENTRY_INT64(blocks_fetched) + +/* pg_stat_get_xact_idx_blocks_hit */ +PG_STAT_GET_XACT_IDXENTRY_INT64(blocks_hit) + Datum pg_stat_get_xact_function_calls(PG_FUNCTION_ARGS) { @@ -2047,6 +2144,17 @@ pg_stat_reset_single_table_counters(PG_FUNCTION_ARGS) PG_RETURN_VOID(); } +Datum +pg_stat_reset_single_index_counters(PG_FUNCTION_ARGS) +{ + Oid idxoid = PG_GETARG_OID(0); + Oid dboid = (IsSharedRelation(idxoid) ? InvalidOid : MyDatabaseId); + + pgstat_reset(PGSTAT_KIND_INDEX, dboid, idxoid); + + PG_RETURN_VOID(); +} + Datum pg_stat_reset_single_function_counters(PG_FUNCTION_ARGS) { diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index 39905c2de142..ffec0903f038 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1806,11 +1806,11 @@ pg_stat_all_indexes| SELECT c.oid AS relid, n.nspname AS schemaname, c.relname, i.relname AS indexrelname, - pg_stat_get_numscans(i.oid) AS idx_scan, - pg_stat_get_lastscan(i.oid) AS last_idx_scan, - pg_stat_get_tuples_returned(i.oid) AS idx_tup_read, - pg_stat_get_tuples_fetched(i.oid) AS idx_tup_fetch, - pg_stat_get_stat_reset_time(i.oid) AS stats_reset + pg_stat_get_idx_numscans(i.oid) AS idx_scan, + pg_stat_get_idx_lastscan(i.oid) AS last_idx_scan, + pg_stat_get_idx_tuples_returned(i.oid) AS idx_tup_read, + pg_stat_get_idx_tuples_fetched(i.oid) AS idx_tup_fetch, + pg_stat_get_idx_stat_reset_time(i.oid) AS stats_reset FROM (((pg_class c JOIN pg_index x ON ((c.oid = x.indrelid))) JOIN pg_class i ON ((i.oid = x.indexrelid))) @@ -1822,9 +1822,9 @@ pg_stat_all_tables| SELECT c.oid AS relid, pg_stat_get_numscans(c.oid) AS seq_scan, pg_stat_get_lastscan(c.oid) AS last_seq_scan, pg_stat_get_tuples_returned(c.oid) AS seq_tup_read, - (sum(pg_stat_get_numscans(i.indexrelid)))::bigint AS idx_scan, - max(pg_stat_get_lastscan(i.indexrelid)) AS last_idx_scan, - ((sum(pg_stat_get_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_tuples_fetched(c.oid)) AS idx_tup_fetch, + (sum(pg_stat_get_idx_numscans(i.indexrelid)))::bigint AS idx_scan, + max(pg_stat_get_idx_lastscan(i.indexrelid)) AS last_idx_scan, + ((sum(pg_stat_get_idx_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_tuples_fetched(c.oid)) AS idx_tup_fetch, pg_stat_get_tuples_inserted(c.oid) AS n_tup_ins, pg_stat_get_tuples_updated(c.oid) AS n_tup_upd, pg_stat_get_tuples_deleted(c.oid) AS n_tup_del, @@ -2452,8 +2452,8 @@ pg_stat_xact_all_tables| SELECT c.oid AS relid, c.relname, pg_stat_get_xact_numscans(c.oid) AS seq_scan, pg_stat_get_xact_tuples_returned(c.oid) AS seq_tup_read, - (sum(pg_stat_get_xact_numscans(i.indexrelid)))::bigint AS idx_scan, - ((sum(pg_stat_get_xact_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_xact_tuples_fetched(c.oid)) AS idx_tup_fetch, + (sum(pg_stat_get_xact_idx_numscans(i.indexrelid)))::bigint AS idx_scan, + ((sum(pg_stat_get_xact_idx_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_xact_tuples_fetched(c.oid)) AS idx_tup_fetch, pg_stat_get_xact_tuples_inserted(c.oid) AS n_tup_ins, pg_stat_get_xact_tuples_updated(c.oid) AS n_tup_upd, pg_stat_get_xact_tuples_deleted(c.oid) AS n_tup_del, @@ -2506,9 +2506,9 @@ pg_statio_all_indexes| SELECT c.oid AS relid, n.nspname AS schemaname, c.relname, i.relname AS indexrelname, - (pg_stat_get_blocks_fetched(i.oid) - pg_stat_get_blocks_hit(i.oid)) AS idx_blks_read, - pg_stat_get_blocks_hit(i.oid) AS idx_blks_hit, - pg_stat_get_stat_reset_time(i.oid) AS stats_reset + (pg_stat_get_idx_blocks_fetched(i.oid) - pg_stat_get_idx_blocks_hit(i.oid)) AS idx_blks_read, + pg_stat_get_idx_blocks_hit(i.oid) AS idx_blks_hit, + pg_stat_get_idx_stat_reset_time(i.oid) AS stats_reset FROM (((pg_class c JOIN pg_index x ON ((c.oid = x.indrelid))) JOIN pg_class i ON ((i.oid = x.indexrelid))) @@ -2538,12 +2538,12 @@ pg_statio_all_tables| SELECT c.oid AS relid, FROM ((((pg_class c LEFT JOIN pg_class t ON ((c.reltoastrelid = t.oid))) LEFT JOIN pg_namespace n ON ((n.oid = c.relnamespace))) - LEFT JOIN LATERAL ( SELECT (sum((pg_stat_get_blocks_fetched(pg_index.indexrelid) - pg_stat_get_blocks_hit(pg_index.indexrelid))))::bigint AS idx_blks_read, - (sum(pg_stat_get_blocks_hit(pg_index.indexrelid)))::bigint AS idx_blks_hit + LEFT JOIN LATERAL ( SELECT (sum((pg_stat_get_idx_blocks_fetched(pg_index.indexrelid) - pg_stat_get_idx_blocks_hit(pg_index.indexrelid))))::bigint AS idx_blks_read, + (sum(pg_stat_get_idx_blocks_hit(pg_index.indexrelid)))::bigint AS idx_blks_hit FROM pg_index WHERE (pg_index.indrelid = c.oid)) i ON (true)) - LEFT JOIN LATERAL ( SELECT (sum((pg_stat_get_blocks_fetched(pg_index.indexrelid) - pg_stat_get_blocks_hit(pg_index.indexrelid))))::bigint AS idx_blks_read, - (sum(pg_stat_get_blocks_hit(pg_index.indexrelid)))::bigint AS idx_blks_hit + LEFT JOIN LATERAL ( SELECT (sum((pg_stat_get_idx_blocks_fetched(pg_index.indexrelid) - pg_stat_get_idx_blocks_hit(pg_index.indexrelid))))::bigint AS idx_blks_read, + (sum(pg_stat_get_idx_blocks_hit(pg_index.indexrelid)))::bigint AS idx_blks_hit FROM pg_index WHERE (pg_index.indrelid = t.oid)) x ON (true)) WHERE (c.relkind = ANY (ARRAY['r'::"char", 't'::"char", 'm'::"char"])); diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index 03cbc1cdef59..b44e30f557c9 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -123,18 +123,19 @@ SELECT id, name, fixed_amount, ----+--------------+--------------+-----------+--------------- 1 | database | f | t | t 2 | relation | f | f | t - 3 | function | f | f | t - 4 | replslot | f | t | t - 5 | subscription | f | t | t - 6 | backend | f | t | f - 7 | archiver | t | f | t - 8 | bgwriter | t | f | t - 9 | checkpointer | t | f | t - 10 | io | t | f | t - 11 | lock | t | f | t - 12 | slru | t | f | t - 13 | wal | t | f | t -(13 rows) + 3 | index | f | f | t + 4 | function | f | f | t + 5 | replslot | f | t | t + 6 | subscription | f | t | t + 7 | backend | f | t | f + 8 | archiver | t | f | t + 9 | bgwriter | t | f | t + 10 | checkpointer | t | f | t + 11 | io | t | f | t + 12 | lock | t | f | t + 13 | slru | t | f | t + 14 | wal | t | f | t +(14 rows) -- ensure that both seqscan and indexscan plans are allowed SET enable_seqscan TO on; @@ -907,8 +908,8 @@ SELECT idx_scan, :'test_last_idx' < last_idx_scan AS idx_ok, (1 row) -- check that the stats in pg_stat_all_indexes are reset -SELECT pg_stat_reset_single_table_counters('test_last_scan_pkey'::regclass); - pg_stat_reset_single_table_counters +SELECT pg_stat_reset_single_index_counters('test_last_scan_pkey'::regclass); + pg_stat_reset_single_index_counters ------------------------------------- (1 row) @@ -1392,21 +1393,21 @@ select a from stats_test_tab1 where a = 3; 3 (1 row) -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t (1 row) -- pg_stat_have_stats returns false for dropped index with stats -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t (1 row) DROP index stats_test_idx1; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- f @@ -1422,14 +1423,14 @@ select a from stats_test_tab1 where a = 3; 3 (1 row) -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t (1 row) ROLLBACK; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- f @@ -1444,7 +1445,7 @@ select a from stats_test_tab1 where a = 3; 3 (1 row) -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t @@ -1452,7 +1453,7 @@ SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); REINDEX index CONCURRENTLY stats_test_idx1; -- false for previous oid -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- f @@ -1460,7 +1461,7 @@ SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); -- true for new oid SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t @@ -1468,7 +1469,7 @@ SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns true for a rolled back drop index with stats BEGIN; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t @@ -1476,7 +1477,7 @@ SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); DROP index stats_test_idx1; ROLLBACK; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); pg_stat_have_stats -------------------- t diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index 4c265d1245c7..d4623c32cd32 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -398,7 +398,7 @@ SELECT idx_scan, :'test_last_idx' < last_idx_scan AS idx_ok, FROM pg_stat_all_indexes WHERE indexrelid = 'test_last_scan_pkey'::regclass; -- check that the stats in pg_stat_all_indexes are reset -SELECT pg_stat_reset_single_table_counters('test_last_scan_pkey'::regclass); +SELECT pg_stat_reset_single_index_counters('test_last_scan_pkey'::regclass); SELECT idx_scan, stats_reset IS NOT NULL AS has_stats_reset FROM pg_stat_all_indexes WHERE indexrelid = 'test_last_scan_pkey'::regclass; @@ -619,40 +619,40 @@ CREATE index stats_test_idx1 on stats_test_tab1(a); SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset SET enable_seqscan TO off; select a from stats_test_tab1 where a = 3; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns false for dropped index with stats -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); DROP index stats_test_idx1; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns false for rolled back index creation BEGIN; CREATE index stats_test_idx1 on stats_test_tab1(a); SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset select a from stats_test_tab1 where a = 3; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); ROLLBACK; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns true for reindex CONCURRENTLY CREATE index stats_test_idx1 on stats_test_tab1(a); SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset select a from stats_test_tab1 where a = 3; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); REINDEX index CONCURRENTLY stats_test_idx1; -- false for previous oid -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- true for new oid SELECT 'stats_test_idx1'::regclass::oid AS stats_test_idx1_oid \gset -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- pg_stat_have_stats returns true for a rolled back drop index with stats BEGIN; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); DROP index stats_test_idx1; ROLLBACK; -SELECT pg_stat_have_stats('relation', :dboid, :stats_test_idx1_oid); +SELECT pg_stat_have_stats('index', :dboid, :stats_test_idx1_oid); -- put enable_seqscan back to on SET enable_seqscan TO on; diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml index 858788b227c4..f3024aa16bc1 100644 --- a/doc/src/sgml/monitoring.sgml +++ b/doc/src/sgml/monitoring.sgml @@ -5867,7 +5867,7 @@ description | Waiting for a newly initialized WAL file to reach durable storage <returnvalue>void</returnvalue> </para> <para> - Resets statistics for a single table or index in the current database + Resets statistics for a single table in the current database or shared across all databases in the cluster to zero. It also resets statistics for a single sequence or materialized view in the current database. @@ -5878,6 +5878,24 @@ description | Waiting for a newly initialized WAL file to reach durable storage </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_stat_reset_single_index_counters</primary> + </indexterm> + <function>pg_stat_reset_single_index_counters</function> ( <type>oid</type> ) + <returnvalue>void</returnvalue> + </para> + <para> + Resets statistics for a single index in the current database + or shared across all databases in the cluster to zero. + </para> + <para> + This function is restricted to superusers by default, but other users + can be granted EXECUTE to run the function. + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <indexterm> -- 2.55.0 [text/plain] v13-0002-Refactor-PgStat_TableStatus-to-new-PgStat_Relati.patch (43.0K, ../../[email protected]/3-v13-0002-Refactor-PgStat_TableStatus-to-new-PgStat_Relati.patch) download | inline diff: From 5f8a69eb6e08a1c95ca9ba67e72324daf470075b Mon Sep 17 00:00:00 2001 From: Michael Paquier <[email protected]> Date: Fri, 10 Jul 2026 12:19:56 +0900 Subject: [PATCH v13 2/3] Refactor PgStat_TableStatus to new PgStat_RelationStatus This new structure is split depending on the relkind it deals with: - PGSTAT_KIND_RELATION, for tables. - PGSTAT_KIND_INDEX, for indexes. This change makes the barrier cleaner between the handling of tables and indexes, by being able to track precisely what are the counters used by one or the other for pending data. Using a common ground for both eases the tracking of Relations in the relcache. --- src/include/pgstat.h | 110 +++++-- src/include/utils/pgstat_internal.h | 4 +- src/include/utils/rel.h | 2 +- src/backend/utils/activity/pgstat.c | 4 +- src/backend/utils/activity/pgstat_index.c | 26 +- src/backend/utils/activity/pgstat_relation.c | 296 ++++++++++--------- src/backend/utils/adt/pgstatfuncs.c | 13 +- src/backend/utils/cache/relcache.c | 2 +- src/tools/pgindent/typedefs.list | 2 +- 9 files changed, 257 insertions(+), 202 deletions(-) diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 23b85fc0c223..892cdf24139b 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -125,14 +125,12 @@ typedef struct PgStat_BackendSubEntry * of pg_memory_is_all_zeros() to detect whether there are any stats updates * to apply. * - * It is a component of PgStat_TableStatus (within-backend state). + * It is a component of PgStat_RelationStatus (within-backend state, for + * table data). * - * Note: for a table, tuples_returned is the number of tuples successfully - * fetched by heap_getnext, while tuples_fetched is the number of tuples - * successfully fetched by heap_fetch under the control of bitmap indexscans. - * For an index, tuples_returned is the number of index entries returned by - * the index AM, while tuples_fetched is the number of tuples successfully - * fetched by heap_fetch under the control of simple indexscans for this index. + * Note: tuples_returned is the number of tuples successfully fetched by + * heap_getnext, while tuples_fetched is the number of tuples successfully + * fetched by heap_fetch under the control of bitmap indexscans. * * tuples_inserted/updated/deleted/hot_updated/newpage_updated count attempted * actions, regardless of whether the transaction committed. delta_live_tuples, @@ -163,34 +161,69 @@ typedef struct PgStat_TableCounts } PgStat_TableCounts; /* ---------- - * PgStat_TableStatus Per-table status within a backend + * PgStat_IndexCounts Per-index pending event counters + * + * Note: tuples_returned is the number of index entries returned by + * the index AM, while tuples_fetched is the number of tuples successfully + * fetched by heap_fetch under the control of simple indexscans for this + * index. + * + * It is a component of PgStat_RelationStatus (within-backend state, for + * index data). + * ---------- + */ +typedef struct PgStat_IndexCounts +{ + PgStat_Counter numscans; + PgStat_Counter tuples_returned; + PgStat_Counter tuples_fetched; + PgStat_Counter blocks_fetched; + PgStat_Counter blocks_hit; +} PgStat_IndexCounts; + +/* ---------- + * PgStat_RelationStatus Per-relation pending status within a backend * * Many of the event counters are nontransactional, ie, we count events * in committed and aborted transactions alike. For these, we just count - * directly in the PgStat_TableStatus. However, delta_live_tuples, + * directly in the PgStat_RelationStatus. However, delta_live_tuples, * delta_dead_tuples, and changed_tuples must be derived from event counts * with awareness of whether the transaction or subtransaction committed or * aborted. Hence, we also keep a stack of per-(sub)transaction status * records for every table modified in the current transaction. At commit * or abort, we propagate tuples_inserted/updated/deleted up to the - * parent subtransaction level, or out to the parent PgStat_TableStatus, + * parent subtransaction level, or out to the parent PgStat_RelationStatus, * as appropriate. + * + * 'kind' tracks the stats kind we are dealing with, for table or index + * pending data. * ---------- */ -typedef struct PgStat_TableStatus +typedef struct PgStat_RelationStatus { - Oid id; /* table's OID */ - bool shared; /* is it a shared catalog? */ - struct PgStat_TableXactStatus *trans; /* lowest subxact's counts */ - PgStat_TableCounts counts; /* event counts to be sent */ + PgStat_Kind kind; /* PGSTAT_KIND_RELATION or PGSTAT_KIND_INDEX */ Relation relation; /* rel that is using this entry */ -} PgStat_TableStatus; + union + { + /* table counters */ + struct + { + Oid id; /* table's OID */ + bool shared; /* is it a shared catalog? */ + struct PgStat_RelXactStatus *trans; /* lowest subxact's counts */ + PgStat_TableCounts counts; /* event counts to be sent */ + } tab; + + /* index counters */ + PgStat_IndexCounts idx; + }; +} PgStat_RelationStatus; /* ---------- - * PgStat_TableXactStatus Per-table, per-subtransaction status + * PgStat_RelXactStatus Per-relation, per-subtransaction status * ---------- */ -typedef struct PgStat_TableXactStatus +typedef struct PgStat_RelXactStatus { PgStat_Counter tuples_inserted; /* tuples inserted in (sub)xact */ PgStat_Counter tuples_updated; /* tuples updated in (sub)xact */ @@ -203,11 +236,11 @@ typedef struct PgStat_TableXactStatus PgStat_Counter deleted_pre_truncdrop; int nest_level; /* subtransaction nest level */ /* links to other structs for same relation: */ - struct PgStat_TableXactStatus *upper; /* next higher subxact if any */ - PgStat_TableStatus *parent; /* per-table status */ + struct PgStat_RelXactStatus *upper; /* next higher subxact if any */ + PgStat_RelationStatus *parent; /* per-table status */ /* structs of same subxact level are linked here: */ - struct PgStat_TableXactStatus *next; /* next of same subxact */ -} PgStat_TableXactStatus; + struct PgStat_RelXactStatus *next; /* next of same subxact */ +} PgStat_RelXactStatus; /* ------------------------------------------------------------ @@ -744,37 +777,52 @@ extern void pgstat_report_analyze(Relation rel, #define pgstat_count_heap_scan(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.numscans++; \ + (rel)->pgstat_info->tab.counts.numscans++; \ } while (0) #define pgstat_count_heap_getnext(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.tuples_returned++; \ + (rel)->pgstat_info->tab.counts.tuples_returned++; \ } while (0) #define pgstat_count_heap_fetch(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.tuples_fetched++; \ + { \ + if ((rel)->pgstat_info->kind == PGSTAT_KIND_INDEX) \ + (rel)->pgstat_info->idx.tuples_fetched++; \ + else \ + (rel)->pgstat_info->tab.counts.tuples_fetched++; \ + } \ } while (0) #define pgstat_count_index_scan(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.numscans++; \ + (rel)->pgstat_info->idx.numscans++; \ } while (0) #define pgstat_count_index_tuples(rel, n) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.tuples_returned += (n); \ + (rel)->pgstat_info->idx.tuples_returned += (n); \ } while (0) #define pgstat_count_buffer_read(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.blocks_fetched++; \ + { \ + if ((rel)->pgstat_info->kind == PGSTAT_KIND_INDEX) \ + (rel)->pgstat_info->idx.blocks_fetched++; \ + else \ + (rel)->pgstat_info->tab.counts.blocks_fetched++; \ + } \ } while (0) #define pgstat_count_buffer_hit(rel) \ do { \ if (pgstat_should_count_relation(rel)) \ - (rel)->pgstat_info->counts.blocks_hit++; \ + { \ + if ((rel)->pgstat_info->kind == PGSTAT_KIND_INDEX) \ + (rel)->pgstat_info->idx.blocks_hit++; \ + else \ + (rel)->pgstat_info->tab.counts.blocks_hit++; \ + } \ } while (0) extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n); @@ -792,8 +840,8 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid); extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free); -extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id); -extern PgStat_TableStatus *find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id); +extern PgStat_RelationStatus *find_relstat_entry_kind(PgStat_Kind kind, + Oid rel_id); extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry(Oid relid); extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry_ext(bool shared, diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h index 94c94c103feb..4df3082f2b6e 100644 --- a/src/include/utils/pgstat_internal.h +++ b/src/include/utils/pgstat_internal.h @@ -215,13 +215,13 @@ typedef struct PgStat_SubXactStatus /* * Tuple insertion/deletion counts for an open transaction can't be - * propagated into PgStat_TableStatus counters until we know if it is + * propagated into PgStat_RelationStatus counters until we know if it is * going to commit or abort. Hence, we keep these counts in per-subxact * structs that live in TopTransactionContext. This data structure is * designed on the assumption that subxacts won't usually modify very many * tables. */ - PgStat_TableXactStatus *first; /* head of list for this subxact */ + PgStat_RelXactStatus *first; /* head of list for this subxact */ } PgStat_SubXactStatus; diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h index 89c159b133fa..6d3423b2ad47 100644 --- a/src/include/utils/rel.h +++ b/src/include/utils/rel.h @@ -252,7 +252,7 @@ typedef struct RelationData bool pgstat_enabled; /* should relation stats be counted */ /* use "struct" here to avoid needing to include pgstat.h: */ - struct PgStat_TableStatus *pgstat_info; /* statistics collection area */ + struct PgStat_RelationStatus *pgstat_info; /* statistics collection area */ } RelationData; diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 3aa10162f8ad..907d1f53a7ec 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -310,7 +310,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .shared_size = sizeof(PgStatShared_Relation), .shared_data_off = offsetof(PgStatShared_Relation, stats), .shared_data_len = sizeof(((PgStatShared_Relation *) 0)->stats), - .pending_size = sizeof(PgStat_TableStatus), + .pending_size = sizeof(PgStat_RelationStatus), .flush_pending_cb = pgstat_relation_flush_cb, .delete_pending_cb = pgstat_relation_delete_pending_cb, @@ -326,7 +326,7 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .shared_size = sizeof(PgStatShared_Index), .shared_data_off = offsetof(PgStatShared_Index, stats), .shared_data_len = sizeof(((PgStatShared_Index *) 0)->stats), - .pending_size = sizeof(PgStat_TableStatus), + .pending_size = sizeof(PgStat_RelationStatus), .flush_pending_cb = pgstat_index_flush_cb, .delete_pending_cb = pgstat_index_delete_pending_cb, diff --git a/src/backend/utils/activity/pgstat_index.c b/src/backend/utils/activity/pgstat_index.c index e9fc27f2e4b4..5d1746614368 100644 --- a/src/backend/utils/activity/pgstat_index.c +++ b/src/backend/utils/activity/pgstat_index.c @@ -35,21 +35,21 @@ bool pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) { Oid dboid; - PgStat_TableStatus *lstats; /* pending stats entry */ + PgStat_RelationStatus *lstats; /* pending stats entry */ PgStatShared_Index *shidxstats; PgStat_StatIdxEntry *idxentry; /* index entry of shared stats */ PgStat_StatDBEntry *dbentry; /* pending database entry */ dboid = entry_ref->shared_entry->key.dboid; - lstats = (PgStat_TableStatus *) entry_ref->pending; + lstats = (PgStat_RelationStatus *) entry_ref->pending; shidxstats = (PgStatShared_Index *) entry_ref->shared_stats; /* * Ignore entries that didn't accumulate any actual counts, such as * indexes that were opened by the planner but not used. */ - if (pg_memory_is_all_zeros(&lstats->counts, - sizeof(struct PgStat_TableCounts))) + if (pg_memory_is_all_zeros(&lstats->idx, + sizeof(PgStat_IndexCounts))) return true; if (!pgstat_lock_entry(entry_ref, nowait)) @@ -58,25 +58,25 @@ pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) /* Add the values to the shared entry. */ idxentry = &shidxstats->stats; - idxentry->numscans += lstats->counts.numscans; - if (lstats->counts.numscans) + idxentry->numscans += lstats->idx.numscans; + if (lstats->idx.numscans) { TimestampTz t = GetCurrentTransactionStopTimestamp(); if (t > idxentry->lastscan) idxentry->lastscan = t; } - idxentry->tuples_returned += lstats->counts.tuples_returned; - idxentry->tuples_fetched += lstats->counts.tuples_fetched; - idxentry->blocks_fetched += lstats->counts.blocks_fetched; - idxentry->blocks_hit += lstats->counts.blocks_hit; + idxentry->tuples_returned += lstats->idx.tuples_returned; + idxentry->tuples_fetched += lstats->idx.tuples_fetched; + idxentry->blocks_fetched += lstats->idx.blocks_fetched; + idxentry->blocks_hit += lstats->idx.blocks_hit; pgstat_unlock_entry(entry_ref); /* The entry was successfully flushed, add the same to database stats */ dbentry = pgstat_prep_database_pending(dboid); - dbentry->blocks_fetched += lstats->counts.blocks_fetched; - dbentry->blocks_hit += lstats->counts.blocks_hit; + dbentry->blocks_fetched += lstats->idx.blocks_fetched; + dbentry->blocks_hit += lstats->idx.blocks_hit; return true; } @@ -87,7 +87,7 @@ pgstat_index_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) void pgstat_index_delete_pending_cb(PgStat_EntryRef *entry_ref) { - PgStat_TableStatus *pending = (PgStat_TableStatus *) entry_ref->pending; + PgStat_RelationStatus *pending = (PgStat_RelationStatus *) entry_ref->pending; if (pending->relation) pgstat_unlink_relation(pending->relation); diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 379a65aa28fb..0cfa5be49cde 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -42,12 +42,12 @@ typedef struct TwoPhasePgStatRecord } TwoPhasePgStatRecord; -static PgStat_TableStatus *pgstat_prep_relation_pending(PgStat_Kind kind, +static PgStat_RelationStatus *pgstat_prep_relation_pending(PgStat_Kind kind, Oid rel_id, bool isshared); -static void add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level); -static void ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info); -static void save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop); -static void restore_truncdrop_counters(PgStat_TableXactStatus *trans); +static void add_tabstat_xact_level(PgStat_RelationStatus *pgstat_info, int nest_level); +static void ensure_tabstat_xact_level(PgStat_RelationStatus *pgstat_info); +static void save_truncdrop_counters(PgStat_RelXactStatus *trans, bool is_drop); +static void restore_truncdrop_counters(PgStat_RelXactStatus *trans); /* * Determine the stats kind for a relation based on its relkind. @@ -177,7 +177,7 @@ pgstat_assoc_relation(Relation rel) kind = pgstat_get_relation_kind(rel->rd_rel->relkind); - /* find or make the PgStat_TableStatus entry, and update link */ + /* find or make the PgStat_RelationStatus entry, and update link */ rel->pgstat_info = pgstat_prep_relation_pending(kind, RelationGetRelid(rel), rel->rd_rel->relisshared); @@ -226,7 +226,7 @@ void pgstat_drop_relation(Relation rel) { int nest_level = GetCurrentTransactionNestLevel(); - PgStat_TableStatus *pgstat_info; + PgStat_RelationStatus *pgstat_info; PgStat_Kind kind = pgstat_get_relation_kind(rel->rd_rel->relkind); pgstat_drop_transactional(kind, @@ -239,15 +239,20 @@ pgstat_drop_relation(Relation rel) /* * Transactionally set counters to 0. That ensures that accesses to * pg_stat_xact_all_tables inside the transaction show 0. + * + * Indexes have no transactional counters, so leave. */ pgstat_info = rel->pgstat_info; - if (pgstat_info->trans && - pgstat_info->trans->nest_level == nest_level) + if (pgstat_info->kind == PGSTAT_KIND_INDEX) + return; + + if (pgstat_info->tab.trans && + pgstat_info->tab.trans->nest_level == nest_level) { - save_truncdrop_counters(pgstat_info->trans, true); - pgstat_info->trans->tuples_inserted = 0; - pgstat_info->trans->tuples_updated = 0; - pgstat_info->trans->tuples_deleted = 0; + save_truncdrop_counters(pgstat_info->tab.trans, true); + pgstat_info->tab.trans->tuples_inserted = 0; + pgstat_info->tab.trans->tuples_updated = 0; + pgstat_info->tab.trans->tuples_deleted = 0; } } @@ -355,15 +360,15 @@ pgstat_report_analyze(Relation rel, if (pgstat_should_count_relation(rel) && rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE) { - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; - for (trans = rel->pgstat_info->trans; trans; trans = trans->upper) + for (trans = rel->pgstat_info->tab.trans; trans; trans = trans->upper) { livetuples -= trans->tuples_inserted - trans->tuples_deleted; deadtuples -= trans->tuples_updated + trans->tuples_deleted; } /* count stuff inserted by already-aborted subxacts, too */ - deadtuples -= rel->pgstat_info->counts.delta_dead_tuples; + deadtuples -= rel->pgstat_info->tab.counts.delta_dead_tuples; /* Since ANALYZE's counts are estimates, we could have underflowed */ livetuples = Max(livetuples, 0); deadtuples = Max(deadtuples, 0); @@ -422,10 +427,10 @@ pgstat_count_heap_insert(Relation rel, PgStat_Counter n) { if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; ensure_tabstat_xact_level(pgstat_info); - pgstat_info->trans->tuples_inserted += n; + pgstat_info->tab.trans->tuples_inserted += n; } } @@ -439,19 +444,19 @@ pgstat_count_heap_update(Relation rel, bool hot, bool newpage) if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; ensure_tabstat_xact_level(pgstat_info); - pgstat_info->trans->tuples_updated++; + pgstat_info->tab.trans->tuples_updated++; /* * tuples_hot_updated and tuples_newpage_updated counters are * nontransactional, so just advance them */ if (hot) - pgstat_info->counts.tuples_hot_updated++; + pgstat_info->tab.counts.tuples_hot_updated++; else if (newpage) - pgstat_info->counts.tuples_newpage_updated++; + pgstat_info->tab.counts.tuples_newpage_updated++; } } @@ -463,10 +468,10 @@ pgstat_count_heap_delete(Relation rel) { if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; ensure_tabstat_xact_level(pgstat_info); - pgstat_info->trans->tuples_deleted++; + pgstat_info->tab.trans->tuples_deleted++; } } @@ -478,13 +483,13 @@ pgstat_count_truncate(Relation rel) { if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; ensure_tabstat_xact_level(pgstat_info); - save_truncdrop_counters(pgstat_info->trans, false); - pgstat_info->trans->tuples_inserted = 0; - pgstat_info->trans->tuples_updated = 0; - pgstat_info->trans->tuples_deleted = 0; + save_truncdrop_counters(pgstat_info->tab.trans, false); + pgstat_info->tab.trans->tuples_inserted = 0; + pgstat_info->tab.trans->tuples_updated = 0; + pgstat_info->tab.trans->tuples_deleted = 0; } } @@ -501,9 +506,9 @@ pgstat_update_heap_dead_tuples(Relation rel, int delta) { if (pgstat_should_count_relation(rel)) { - PgStat_TableStatus *pgstat_info = rel->pgstat_info; + PgStat_RelationStatus *pgstat_info = rel->pgstat_info; - pgstat_info->counts.delta_dead_tuples -= delta; + pgstat_info->tab.counts.delta_dead_tuples -= delta; } } @@ -534,9 +539,9 @@ pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free) } /* - * find any existing PgStat_TableStatus entry for rel + * find any existing PgStat_RelationStatus entry for rel and kind * - * Find any existing PgStat_TableStatus entry for rel_id in the current + * Find any existing PgStat_RelationStatus entry for rel_id in the current * database. If not found, try finding from shared tables. * * If an entry is found, copy it and increment the copy's counters with their @@ -545,22 +550,13 @@ pgstat_fetch_stat_tabentry_ext(bool shared, Oid reloid, bool *may_free) * * If no entry found, return NULL, don't create a new one. */ -PgStat_TableStatus * -find_tabstat_entry(Oid rel_id) -{ - return find_tabstat_entry_kind(PGSTAT_KIND_RELATION, rel_id); -} - -/* - * Same as find_tabstat_entry but for a specific stats kind. - */ -PgStat_TableStatus * -find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id) +PgStat_RelationStatus * +find_relstat_entry_kind(PgStat_Kind kind, Oid rel_id) { PgStat_EntryRef *entry_ref; - PgStat_TableXactStatus *trans; - PgStat_TableStatus *tabentry = NULL; - PgStat_TableStatus *tablestatus = NULL; + PgStat_RelXactStatus *trans; + PgStat_RelationStatus *tabentry = NULL; + PgStat_RelationStatus *tablestatus = NULL; entry_ref = pgstat_fetch_pending_entry(kind, MyDatabaseId, rel_id); if (!entry_ref) @@ -570,27 +566,33 @@ find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id) return tablestatus; } - tabentry = (PgStat_TableStatus *) entry_ref->pending; - tablestatus = palloc_object(PgStat_TableStatus); + tabentry = (PgStat_RelationStatus *) entry_ref->pending; + tablestatus = palloc_object(PgStat_RelationStatus); *tablestatus = *tabentry; /* - * Reset tablestatus->trans in the copy of PgStat_TableStatus as it may + * For index entries, just return the copy — no transactional data. + */ + if (kind == PGSTAT_KIND_INDEX) + return tablestatus; + + /* + * Reset tablestatus->trans in the copy of PgStat_RelationStatus as it may * point to a shared memory area. Its data is saved below, so removing it * does not matter. */ - tablestatus->trans = NULL; + tablestatus->tab.trans = NULL; /* * Live subtransaction counts are not included yet. This is not a hot * code path so reconcile tuples_inserted, tuples_updated and * tuples_deleted even if the caller may not be interested in this data. */ - for (trans = tabentry->trans; trans != NULL; trans = trans->upper) + for (trans = tabentry->tab.trans; trans != NULL; trans = trans->upper) { - tablestatus->counts.tuples_inserted += trans->tuples_inserted; - tablestatus->counts.tuples_updated += trans->tuples_updated; - tablestatus->counts.tuples_deleted += trans->tuples_deleted; + tablestatus->tab.counts.tuples_inserted += trans->tuples_inserted; + tablestatus->tab.counts.tuples_updated += trans->tuples_updated; + tablestatus->tab.counts.tuples_deleted += trans->tuples_deleted; } return tablestatus; @@ -607,51 +609,51 @@ find_tabstat_entry_kind(PgStat_Kind kind, Oid rel_id) void AtEOXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit) { - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; for (trans = xact_state->first; trans != NULL; trans = trans->next) { - PgStat_TableStatus *tabstat; + PgStat_RelationStatus *tabstat; Assert(trans->nest_level == 1); Assert(trans->upper == NULL); tabstat = trans->parent; - Assert(tabstat->trans == trans); + Assert(tabstat->tab.trans == trans); /* restore pre-truncate/drop stats (if any) in case of aborted xact */ if (!isCommit) restore_truncdrop_counters(trans); /* count attempted actions regardless of commit/abort */ - tabstat->counts.tuples_inserted += trans->tuples_inserted; - tabstat->counts.tuples_updated += trans->tuples_updated; - tabstat->counts.tuples_deleted += trans->tuples_deleted; + tabstat->tab.counts.tuples_inserted += trans->tuples_inserted; + tabstat->tab.counts.tuples_updated += trans->tuples_updated; + tabstat->tab.counts.tuples_deleted += trans->tuples_deleted; if (isCommit) { - tabstat->counts.truncdropped = trans->truncdropped; + tabstat->tab.counts.truncdropped = trans->truncdropped; if (trans->truncdropped) { /* forget live/dead stats seen by backend thus far */ - tabstat->counts.delta_live_tuples = 0; - tabstat->counts.delta_dead_tuples = 0; + tabstat->tab.counts.delta_live_tuples = 0; + tabstat->tab.counts.delta_dead_tuples = 0; } /* insert adds a live tuple, delete removes one */ - tabstat->counts.delta_live_tuples += + tabstat->tab.counts.delta_live_tuples += trans->tuples_inserted - trans->tuples_deleted; /* update and delete each create a dead tuple */ - tabstat->counts.delta_dead_tuples += + tabstat->tab.counts.delta_dead_tuples += trans->tuples_updated + trans->tuples_deleted; /* insert, update, delete each count as one change event */ - tabstat->counts.changed_tuples += + tabstat->tab.counts.changed_tuples += trans->tuples_inserted + trans->tuples_updated + trans->tuples_deleted; } else { /* inserted tuples are dead, deleted tuples are unaffected */ - tabstat->counts.delta_dead_tuples += + tabstat->tab.counts.delta_dead_tuples += trans->tuples_inserted + trans->tuples_updated; /* an aborted xact generates no changed_tuple events */ } - tabstat->trans = NULL; + tabstat->tab.trans = NULL; } } @@ -665,17 +667,17 @@ AtEOXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit) void AtEOSubXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit, int nestDepth) { - PgStat_TableXactStatus *trans; - PgStat_TableXactStatus *next_trans; + PgStat_RelXactStatus *trans; + PgStat_RelXactStatus *next_trans; for (trans = xact_state->first; trans != NULL; trans = next_trans) { - PgStat_TableStatus *tabstat; + PgStat_RelationStatus *tabstat; next_trans = trans->next; Assert(trans->nest_level == nestDepth); tabstat = trans->parent; - Assert(tabstat->trans == trans); + Assert(tabstat->tab.trans == trans); if (isCommit) { @@ -696,7 +698,7 @@ AtEOSubXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit, in trans->upper->tuples_updated += trans->tuples_updated; trans->upper->tuples_deleted += trans->tuples_deleted; } - tabstat->trans = trans->upper; + tabstat->tab.trans = trans->upper; pfree(trans); } else @@ -727,13 +729,13 @@ AtEOSubXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit, in /* first restore values obliterated by truncate/drop */ restore_truncdrop_counters(trans); /* count attempted actions regardless of commit/abort */ - tabstat->counts.tuples_inserted += trans->tuples_inserted; - tabstat->counts.tuples_updated += trans->tuples_updated; - tabstat->counts.tuples_deleted += trans->tuples_deleted; + tabstat->tab.counts.tuples_inserted += trans->tuples_inserted; + tabstat->tab.counts.tuples_updated += trans->tuples_updated; + tabstat->tab.counts.tuples_deleted += trans->tuples_deleted; /* inserted tuples are dead, deleted tuples are unaffected */ - tabstat->counts.delta_dead_tuples += + tabstat->tab.counts.delta_dead_tuples += trans->tuples_inserted + trans->tuples_updated; - tabstat->trans = trans->upper; + tabstat->tab.trans = trans->upper; pfree(trans); } } @@ -746,17 +748,17 @@ AtEOSubXact_PgStat_Relations(PgStat_SubXactStatus *xact_state, bool isCommit, in void AtPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) { - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; for (trans = xact_state->first; trans != NULL; trans = trans->next) { - PgStat_TableStatus *tabstat PG_USED_FOR_ASSERTS_ONLY; + PgStat_RelationStatus *tabstat PG_USED_FOR_ASSERTS_ONLY; TwoPhasePgStatRecord record; Assert(trans->nest_level == 1); Assert(trans->upper == NULL); tabstat = trans->parent; - Assert(tabstat->trans == trans); + Assert(tabstat->tab.trans == trans); record.tuples_inserted = trans->tuples_inserted; record.tuples_updated = trans->tuples_updated; @@ -764,8 +766,8 @@ AtPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) record.inserted_pre_truncdrop = trans->inserted_pre_truncdrop; record.updated_pre_truncdrop = trans->updated_pre_truncdrop; record.deleted_pre_truncdrop = trans->deleted_pre_truncdrop; - record.id = tabstat->id; - record.shared = tabstat->shared; + record.id = tabstat->tab.id; + record.shared = tabstat->tab.shared; record.truncdropped = trans->truncdropped; RegisterTwoPhaseRecord(TWOPHASE_RM_PGSTAT_ID, 0, @@ -784,14 +786,14 @@ AtPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) void PostPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) { - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; for (trans = xact_state->first; trans != NULL; trans = trans->next) { - PgStat_TableStatus *tabstat; + PgStat_RelationStatus *tabstat; tabstat = trans->parent; - tabstat->trans = NULL; + tabstat->tab.trans = NULL; } } @@ -805,27 +807,27 @@ pgstat_twophase_postcommit(FullTransactionId fxid, uint16 info, void *recdata, uint32 len) { TwoPhasePgStatRecord *rec = (TwoPhasePgStatRecord *) recdata; - PgStat_TableStatus *pgstat_info; + PgStat_RelationStatus *pgstat_info; /* Find or create a tabstat entry for the rel */ pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); /* Same math as in AtEOXact_PgStat, commit case */ - pgstat_info->counts.tuples_inserted += rec->tuples_inserted; - pgstat_info->counts.tuples_updated += rec->tuples_updated; - pgstat_info->counts.tuples_deleted += rec->tuples_deleted; - pgstat_info->counts.truncdropped = rec->truncdropped; + pgstat_info->tab.counts.tuples_inserted += rec->tuples_inserted; + pgstat_info->tab.counts.tuples_updated += rec->tuples_updated; + pgstat_info->tab.counts.tuples_deleted += rec->tuples_deleted; + pgstat_info->tab.counts.truncdropped = rec->truncdropped; if (rec->truncdropped) { /* forget live/dead stats seen by backend thus far */ - pgstat_info->counts.delta_live_tuples = 0; - pgstat_info->counts.delta_dead_tuples = 0; + pgstat_info->tab.counts.delta_live_tuples = 0; + pgstat_info->tab.counts.delta_dead_tuples = 0; } - pgstat_info->counts.delta_live_tuples += + pgstat_info->tab.counts.delta_live_tuples += rec->tuples_inserted - rec->tuples_deleted; - pgstat_info->counts.delta_dead_tuples += + pgstat_info->tab.counts.delta_dead_tuples += rec->tuples_updated + rec->tuples_deleted; - pgstat_info->counts.changed_tuples += + pgstat_info->tab.counts.changed_tuples += rec->tuples_inserted + rec->tuples_updated + rec->tuples_deleted; } @@ -841,7 +843,7 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, void *recdata, uint32 len) { TwoPhasePgStatRecord *rec = (TwoPhasePgStatRecord *) recdata; - PgStat_TableStatus *pgstat_info; + PgStat_RelationStatus *pgstat_info; /* Find or create a tabstat entry for the rel */ pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); @@ -853,10 +855,10 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, rec->tuples_updated = rec->updated_pre_truncdrop; rec->tuples_deleted = rec->deleted_pre_truncdrop; } - pgstat_info->counts.tuples_inserted += rec->tuples_inserted; - pgstat_info->counts.tuples_updated += rec->tuples_updated; - pgstat_info->counts.tuples_deleted += rec->tuples_deleted; - pgstat_info->counts.delta_dead_tuples += + pgstat_info->tab.counts.tuples_inserted += rec->tuples_inserted; + pgstat_info->tab.counts.tuples_updated += rec->tuples_updated; + pgstat_info->tab.counts.tuples_deleted += rec->tuples_deleted; + pgstat_info->tab.counts.delta_dead_tuples += rec->tuples_inserted + rec->tuples_updated; } @@ -873,20 +875,20 @@ bool pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) { Oid dboid; - PgStat_TableStatus *lstats; /* pending stats entry */ + PgStat_RelationStatus *lstats; /* pending stats entry */ PgStatShared_Relation *shtabstats; PgStat_StatTabEntry *tabentry; /* table entry of shared stats */ PgStat_StatDBEntry *dbentry; /* pending database entry */ dboid = entry_ref->shared_entry->key.dboid; - lstats = (PgStat_TableStatus *) entry_ref->pending; + lstats = (PgStat_RelationStatus *) entry_ref->pending; shtabstats = (PgStatShared_Relation *) entry_ref->shared_stats; /* * Ignore entries that didn't accumulate any actual counts, such as * indexes that were opened by the planner but not used. */ - if (pg_memory_is_all_zeros(&lstats->counts, + if (pg_memory_is_all_zeros(&lstats->tab.counts, sizeof(struct PgStat_TableCounts))) return true; @@ -896,35 +898,35 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) /* add the values to the shared entry. */ tabentry = &shtabstats->stats; - tabentry->numscans += lstats->counts.numscans; - if (lstats->counts.numscans) + tabentry->numscans += lstats->tab.counts.numscans; + if (lstats->tab.counts.numscans) { TimestampTz t = GetCurrentTransactionStopTimestamp(); if (t > tabentry->lastscan) tabentry->lastscan = t; } - tabentry->tuples_returned += lstats->counts.tuples_returned; - tabentry->tuples_fetched += lstats->counts.tuples_fetched; - tabentry->tuples_inserted += lstats->counts.tuples_inserted; - tabentry->tuples_updated += lstats->counts.tuples_updated; - tabentry->tuples_deleted += lstats->counts.tuples_deleted; - tabentry->tuples_hot_updated += lstats->counts.tuples_hot_updated; - tabentry->tuples_newpage_updated += lstats->counts.tuples_newpage_updated; + tabentry->tuples_returned += lstats->tab.counts.tuples_returned; + tabentry->tuples_fetched += lstats->tab.counts.tuples_fetched; + tabentry->tuples_inserted += lstats->tab.counts.tuples_inserted; + tabentry->tuples_updated += lstats->tab.counts.tuples_updated; + tabentry->tuples_deleted += lstats->tab.counts.tuples_deleted; + tabentry->tuples_hot_updated += lstats->tab.counts.tuples_hot_updated; + tabentry->tuples_newpage_updated += lstats->tab.counts.tuples_newpage_updated; /* * If table was truncated/dropped, first reset the live/dead counters. */ - if (lstats->counts.truncdropped) + if (lstats->tab.counts.truncdropped) { tabentry->live_tuples = 0; tabentry->dead_tuples = 0; tabentry->ins_since_vacuum = 0; } - tabentry->live_tuples += lstats->counts.delta_live_tuples; - tabentry->dead_tuples += lstats->counts.delta_dead_tuples; - tabentry->mod_since_analyze += lstats->counts.changed_tuples; + tabentry->live_tuples += lstats->tab.counts.delta_live_tuples; + tabentry->dead_tuples += lstats->tab.counts.delta_dead_tuples; + tabentry->mod_since_analyze += lstats->tab.counts.changed_tuples; /* * Using tuples_inserted to update ins_since_vacuum does mean that we'll @@ -933,10 +935,10 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) * triggering for inserts more often than they maybe should, which is * probably not going to be common enough to be too concerned about here. */ - tabentry->ins_since_vacuum += lstats->counts.tuples_inserted; + tabentry->ins_since_vacuum += lstats->tab.counts.tuples_inserted; - tabentry->blocks_fetched += lstats->counts.blocks_fetched; - tabentry->blocks_hit += lstats->counts.blocks_hit; + tabentry->blocks_fetched += lstats->tab.counts.blocks_fetched; + tabentry->blocks_hit += lstats->tab.counts.blocks_hit; /* Clamp live_tuples in case of negative delta_live_tuples */ tabentry->live_tuples = Max(tabentry->live_tuples, 0); @@ -947,13 +949,13 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) /* The entry was successfully flushed, add the same to database stats */ dbentry = pgstat_prep_database_pending(dboid); - dbentry->tuples_returned += lstats->counts.tuples_returned; - dbentry->tuples_fetched += lstats->counts.tuples_fetched; - dbentry->tuples_inserted += lstats->counts.tuples_inserted; - dbentry->tuples_updated += lstats->counts.tuples_updated; - dbentry->tuples_deleted += lstats->counts.tuples_deleted; - dbentry->blocks_fetched += lstats->counts.blocks_fetched; - dbentry->blocks_hit += lstats->counts.blocks_hit; + dbentry->tuples_returned += lstats->tab.counts.tuples_returned; + dbentry->tuples_fetched += lstats->tab.counts.tuples_fetched; + dbentry->tuples_inserted += lstats->tab.counts.tuples_inserted; + dbentry->tuples_updated += lstats->tab.counts.tuples_updated; + dbentry->tuples_deleted += lstats->tab.counts.tuples_deleted; + dbentry->blocks_fetched += lstats->tab.counts.blocks_fetched; + dbentry->blocks_hit += lstats->tab.counts.blocks_hit; return true; } @@ -961,7 +963,7 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) void pgstat_relation_delete_pending_cb(PgStat_EntryRef *entry_ref) { - PgStat_TableStatus *pending = (PgStat_TableStatus *) entry_ref->pending; + PgStat_RelationStatus *pending = (PgStat_RelationStatus *) entry_ref->pending; if (pending->relation) pgstat_unlink_relation(pending->relation); @@ -974,21 +976,25 @@ pgstat_relation_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts) } /* - * Find or create a PgStat_TableStatus entry for rel. New entry is created and + * Find or create a PgStat_RelationStatus entry for rel. New entry is created and * initialized if not exists. */ -static PgStat_TableStatus * +static PgStat_RelationStatus * pgstat_prep_relation_pending(PgStat_Kind kind, Oid rel_id, bool isshared) { PgStat_EntryRef *entry_ref; - PgStat_TableStatus *pending; + PgStat_RelationStatus *pending; entry_ref = pgstat_prep_pending_entry(kind, isshared ? InvalidOid : MyDatabaseId, rel_id, NULL); pending = entry_ref->pending; - pending->id = rel_id; - pending->shared = isshared; + pending->kind = kind; + if (kind != PGSTAT_KIND_INDEX) + { + pending->tab.id = rel_id; + pending->tab.shared = isshared; + } return pending; } @@ -997,10 +1003,10 @@ pgstat_prep_relation_pending(PgStat_Kind kind, Oid rel_id, bool isshared) * add a new (sub)transaction state record */ static void -add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level) +add_tabstat_xact_level(PgStat_RelationStatus *pgstat_info, int nest_level) { PgStat_SubXactStatus *xact_state; - PgStat_TableXactStatus *trans; + PgStat_RelXactStatus *trans; /* * If this is the first rel to be modified at the current nest level, we @@ -1009,27 +1015,27 @@ add_tabstat_xact_level(PgStat_TableStatus *pgstat_info, int nest_level) xact_state = pgstat_get_xact_stack_level(nest_level); /* Now make a per-table stack entry */ - trans = (PgStat_TableXactStatus *) + trans = (PgStat_RelXactStatus *) MemoryContextAllocZero(TopTransactionContext, - sizeof(PgStat_TableXactStatus)); + sizeof(PgStat_RelXactStatus)); trans->nest_level = nest_level; - trans->upper = pgstat_info->trans; + trans->upper = pgstat_info->tab.trans; trans->parent = pgstat_info; trans->next = xact_state->first; xact_state->first = trans; - pgstat_info->trans = trans; + pgstat_info->tab.trans = trans; } /* * Add a new (sub)transaction record if needed. */ static void -ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info) +ensure_tabstat_xact_level(PgStat_RelationStatus *pgstat_info) { int nest_level = GetCurrentTransactionNestLevel(); - if (pgstat_info->trans == NULL || - pgstat_info->trans->nest_level != nest_level) + if (pgstat_info->tab.trans == NULL || + pgstat_info->tab.trans->nest_level != nest_level) add_tabstat_xact_level(pgstat_info, nest_level); } @@ -1043,7 +1049,7 @@ ensure_tabstat_xact_level(PgStat_TableStatus *pgstat_info) * subxact level only. */ static void -save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop) +save_truncdrop_counters(PgStat_RelXactStatus *trans, bool is_drop) { if (!trans->truncdropped || is_drop) { @@ -1058,7 +1064,7 @@ save_truncdrop_counters(PgStat_TableXactStatus *trans, bool is_drop) * restore counters when a truncate aborts */ static void -restore_truncdrop_counters(PgStat_TableXactStatus *trans) +restore_truncdrop_counters(PgStat_RelXactStatus *trans) { if (trans->truncdropped) { diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 946025f39ed0..0d47d745c18f 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -1928,12 +1928,13 @@ CppConcat(pg_stat_get_xact_,stat)(PG_FUNCTION_ARGS) \ { \ Oid relid = PG_GETARG_OID(0); \ int64 result; \ - PgStat_TableStatus *tabentry; \ + PgStat_RelationStatus *tabentry; \ \ - if ((tabentry = find_tabstat_entry(relid)) == NULL) \ + if ((tabentry = find_relstat_entry_kind(PGSTAT_KIND_RELATION, \ + relid)) == NULL) \ result = 0; \ else \ - result = (int64) (tabentry->counts.stat); \ + result = (int64) (tabentry->tab.counts.stat); \ \ PG_RETURN_INT64(result); \ } @@ -1977,13 +1978,13 @@ CppConcat(pg_stat_get_xact_idx_,stat)(PG_FUNCTION_ARGS) \ { \ Oid relid = PG_GETARG_OID(0); \ int64 result; \ - PgStat_TableStatus *tabentry; \ + PgStat_RelationStatus *tabentry; \ \ - tabentry = find_tabstat_entry_kind(PGSTAT_KIND_INDEX, relid); \ + tabentry = find_relstat_entry_kind(PGSTAT_KIND_INDEX, relid); \ if (!tabentry) \ result = 0; \ else \ - result = (int64) (tabentry->counts.stat); \ + result = (int64) (tabentry->idx.stat); \ \ PG_RETURN_INT64(result); \ } diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index fb4e042be8ad..ce965e0db116 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -2758,7 +2758,7 @@ RelationRebuildRelation(Relation relation) /* toast OID override must be preserved */ SWAPFIELD(Oid, rd_toastoid); /* pgstat_info / enabled must be preserved */ - SWAPFIELD(struct PgStat_TableStatus *, pgstat_info); + SWAPFIELD(struct PgStat_RelationStatus *, pgstat_info); SWAPFIELD(bool, pgstat_enabled); /* preserve old partition key if we have one */ if (keep_partkey) diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88b..2a7049a92490 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2354,6 +2354,7 @@ PgStat_LockEntry PgStat_PendingDroppedStatsItem PgStat_PendingIO PgStat_PendingLock +PgStat_RelationStatus PgStat_SLRUStats PgStat_ShmemControl PgStat_Snapshot @@ -2368,7 +2369,6 @@ PgStat_StatTabEntry PgStat_StatsFileOp PgStat_SubXactStatus PgStat_TableCounts -PgStat_TableStatus PgStat_TableXactStatus PgStat_WalCounters PgStat_WalStats -- 2.55.0 [text/plain] v13-0003-Introduce-relfilenode-statistics.patch (87.4K, ../../[email protected]/4-v13-0003-Introduce-relfilenode-statistics.patch) download | inline diff: From 7da7413fad7c38c39b396789511f0e910a16608b Mon Sep 17 00:00:00 2001 From: Michael Paquier <[email protected]> Date: Fri, 10 Jul 2026 14:53:50 +0900 Subject: [PATCH v13 3/3] Introduce relfilenode statistics Statistics data previously associated to relations are not moved under a new stats kind, for relfilenodes. This enables the reconstruction of the statistics after a system crash, as WAL replay is only able to know about the relfilenodes it works on. Shared catalogs are handled using their relmapping. A new set of fetch functions are used in the views, providing an equivalent of what existed previously. When a relation rewrite happens, the aggregated counters are not persisted, restarted at 0 as we start afresh. --- src/include/catalog/pg_proc.dat | 75 ++++--- src/include/pgstat.h | 39 +++- src/include/utils/pgstat_internal.h | 14 ++ src/include/utils/pgstat_kind.h | 15 +- src/backend/access/heap/heapam_xlog.c | 13 ++ src/backend/catalog/system_views.sql | 18 +- src/backend/postmaster/autovacuum.c | 40 ++-- src/backend/utils/activity/Makefile | 1 + src/backend/utils/activity/meson.build | 1 + src/backend/utils/activity/pgstat.c | 15 ++ src/backend/utils/activity/pgstat_relation.c | 190 +++++++++++------ .../utils/activity/pgstat_relfilenode.c | 179 ++++++++++++++++ src/backend/utils/adt/pgstatfuncs.c | 74 ++++--- src/test/isolation/expected/stats.out | 198 +++++++++--------- src/test/isolation/specs/stats.spec | 10 +- src/test/regress/expected/rules.out | 18 +- src/test/regress/expected/stats.out | 169 +++++++-------- src/test/regress/expected/stats_rewrite.out | 20 +- src/test/regress/expected/stats_rewrite_1.out | 8 +- src/test/regress/sql/stats.sql | 42 ++-- .../postgres_fdw/expected/postgres_fdw.out | 20 +- contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +- 22 files changed, 751 insertions(+), 416 deletions(-) create mode 100644 src/backend/utils/activity/pgstat_relfilenode.c diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index dc6c61e84e54..acb8b9a4d588 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -5578,45 +5578,6 @@ proname => 'pg_stat_get_tuples_fetched', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_tuples_fetched' }, -{ oid => '1931', descr => 'statistics: number of tuples inserted', - proname => 'pg_stat_get_tuples_inserted', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_inserted' }, -{ oid => '1932', descr => 'statistics: number of tuples updated', - proname => 'pg_stat_get_tuples_updated', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_updated' }, -{ oid => '1933', descr => 'statistics: number of tuples deleted', - proname => 'pg_stat_get_tuples_deleted', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_deleted' }, -{ oid => '1972', descr => 'statistics: number of tuples hot updated', - proname => 'pg_stat_get_tuples_hot_updated', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_hot_updated' }, -{ oid => '6217', - descr => 'statistics: number of tuples updated onto a new page', - proname => 'pg_stat_get_tuples_newpage_updated', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_tuples_newpage_updated' }, -{ oid => '2878', descr => 'statistics: number of live tuples', - proname => 'pg_stat_get_live_tuples', provolatile => 's', proparallel => 'r', - prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_live_tuples' }, -{ oid => '2879', descr => 'statistics: number of dead tuples', - proname => 'pg_stat_get_dead_tuples', provolatile => 's', proparallel => 'r', - prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_dead_tuples' }, -{ oid => '3177', - descr => 'statistics: number of tuples changed since last analyze', - proname => 'pg_stat_get_mod_since_analyze', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_mod_since_analyze' }, -{ oid => '5053', - descr => 'statistics: number of tuples inserted since last vacuum', - proname => 'pg_stat_get_ins_since_vacuum', provolatile => 's', - proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', - prosrc => 'pg_stat_get_ins_since_vacuum' }, { oid => '1934', descr => 'statistics: number of blocks fetched', proname => 'pg_stat_get_blocks_fetched', provolatile => 's', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', @@ -6294,6 +6255,42 @@ proname => 'pg_stat_get_xact_idx_blocks_hit', provolatile => 'v', proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', prosrc => 'pg_stat_get_xact_idx_blocks_hit' }, +{ oid => '9055', descr => 'statistics: number of tuples inserted (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_inserted', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_inserted' }, +{ oid => '9056', descr => 'statistics: number of tuples updated (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_updated', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_updated' }, +{ oid => '9057', descr => 'statistics: number of tuples deleted (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_deleted', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_deleted' }, +{ oid => '9058', descr => 'statistics: number of tuples HOT updated (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_hot_updated', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_hot_updated' }, +{ oid => '9059', descr => 'statistics: number of tuples updated to new page (by relfilenode)', + proname => 'pg_stat_get_rfn_tuples_newpage_updated', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_tuples_newpage_updated' }, +{ oid => '9060', descr => 'statistics: number of live tuples (by relfilenode)', + proname => 'pg_stat_get_rfn_live_tuples', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_live_tuples' }, +{ oid => '9061', descr => 'statistics: number of dead tuples (by relfilenode)', + proname => 'pg_stat_get_rfn_dead_tuples', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_dead_tuples' }, +{ oid => '9062', descr => 'statistics: number of tuples changed since last analyze (by relfilenode)', + proname => 'pg_stat_get_rfn_mod_since_analyze', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_mod_since_analyze' }, +{ oid => '9063', descr => 'statistics: number of tuples inserted since last vacuum (by relfilenode)', + proname => 'pg_stat_get_rfn_ins_since_vacuum', provolatile => 's', + proparallel => 'r', prorettype => 'int8', proargtypes => 'oid', + prosrc => 'pg_stat_get_rfn_ins_since_vacuum' }, { oid => '3777', descr => 'statistics: reset collected statistics for a single function in the current database', proname => 'pg_stat_reset_single_function_counters', provolatile => 'v', diff --git a/src/include/pgstat.h b/src/include/pgstat.h index 892cdf24139b..f2094270a000 100644 --- a/src/include/pgstat.h +++ b/src/include/pgstat.h @@ -12,6 +12,7 @@ #define PGSTAT_H #include "datatype/timestamp.h" +#include "common/relpath.h" #include "portability/instr_time.h" #include "postmaster/pgarch.h" /* for MAX_XFN_CHARS */ #include "replication/conflict.h" @@ -210,6 +211,7 @@ typedef struct PgStat_RelationStatus { Oid id; /* table's OID */ bool shared; /* is it a shared catalog? */ + RelFileNumber relfilenode; /* table's relfilenode */ struct PgStat_RelXactStatus *trans; /* lowest subxact's counts */ PgStat_TableCounts counts; /* event counts to be sent */ } tab; @@ -489,17 +491,6 @@ typedef struct PgStat_StatTabEntry PgStat_Counter tuples_returned; PgStat_Counter tuples_fetched; - PgStat_Counter tuples_inserted; - PgStat_Counter tuples_updated; - PgStat_Counter tuples_deleted; - PgStat_Counter tuples_hot_updated; - PgStat_Counter tuples_newpage_updated; - - PgStat_Counter live_tuples; - PgStat_Counter dead_tuples; - PgStat_Counter mod_since_analyze; - PgStat_Counter ins_since_vacuum; - PgStat_Counter blocks_fetched; PgStat_Counter blocks_hit; @@ -534,6 +525,22 @@ typedef struct PgStat_StatIdxEntry TimestampTz stat_reset_time; } PgStat_StatIdxEntry; +typedef struct PgStat_StatRFNodeEntry +{ + PgStat_Counter tuples_inserted; + PgStat_Counter tuples_updated; + PgStat_Counter tuples_deleted; + PgStat_Counter tuples_hot_updated; + PgStat_Counter tuples_newpage_updated; + + PgStat_Counter live_tuples; + PgStat_Counter dead_tuples; + PgStat_Counter mod_since_analyze; + PgStat_Counter ins_since_vacuum; + + TimestampTz stat_reset_time; +} PgStat_StatRFNodeEntry; + /* ------ * PgStat_WalCounters WAL activity data gathered from WalUsage * @@ -848,6 +855,16 @@ extern PgStat_StatIdxEntry *pgstat_fetch_stat_idxentry_ext(bool shared, Oid reloid, bool *may_free); +extern PgStat_StatRFNodeEntry *pgstat_fetch_stat_rfnodeentry(Oid dboid, + RelFileNumber rfn); + +/* WAL replay helpers for relfilenode stats */ +extern void pgstat_wal_replay_insert(Oid dboid, RelFileNumber relfilenode, + int ntuples); +extern void pgstat_wal_replay_update(Oid dboid, RelFileNumber relfilenode, + bool hot, bool newpage); +extern void pgstat_wal_replay_delete(Oid dboid, RelFileNumber relfilenode); + /* * Functions in pgstat_replslot.c diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h index 4df3082f2b6e..466b3e79e5bd 100644 --- a/src/include/utils/pgstat_internal.h +++ b/src/include/utils/pgstat_internal.h @@ -513,6 +513,12 @@ typedef struct PgStatShared_Index PgStat_StatIdxEntry stats; } PgStatShared_Index; +typedef struct PgStatShared_RelFileNode +{ + PgStatShared_Common header; + PgStat_StatRFNodeEntry stats; +} PgStatShared_RelFileNode; + typedef struct PgStatShared_Function { PgStatShared_Common header; @@ -802,6 +808,14 @@ extern void pgstat_index_delete_pending_cb(PgStat_EntryRef *entry_ref); extern void pgstat_index_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); +/* + * Functions in pgstat_relfilenode.c + */ + +extern bool pgstat_relfilenode_flush_cb(PgStat_EntryRef *entry_ref, bool nowait); +extern void pgstat_relfilenode_reset_timestamp_cb(PgStatShared_Common *header, TimestampTz ts); + + /* * Functions in pgstat_replslot.c */ diff --git a/src/include/utils/pgstat_kind.h b/src/include/utils/pgstat_kind.h index 45ca599d0dd6..84ccaa068489 100644 --- a/src/include/utils/pgstat_kind.h +++ b/src/include/utils/pgstat_kind.h @@ -31,15 +31,16 @@ #define PGSTAT_KIND_REPLSLOT 5 /* per-slot statistics */ #define PGSTAT_KIND_SUBSCRIPTION 6 /* per-subscription statistics */ #define PGSTAT_KIND_BACKEND 7 /* per-backend statistics */ +#define PGSTAT_KIND_RELFILENODE 8 /* per-relfilenode statistics */ /* stats for fixed-numbered objects */ -#define PGSTAT_KIND_ARCHIVER 8 -#define PGSTAT_KIND_BGWRITER 9 -#define PGSTAT_KIND_CHECKPOINTER 10 -#define PGSTAT_KIND_IO 11 -#define PGSTAT_KIND_LOCK 12 -#define PGSTAT_KIND_SLRU 13 -#define PGSTAT_KIND_WAL 14 +#define PGSTAT_KIND_ARCHIVER 9 +#define PGSTAT_KIND_BGWRITER 10 +#define PGSTAT_KIND_CHECKPOINTER 11 +#define PGSTAT_KIND_IO 12 +#define PGSTAT_KIND_LOCK 13 +#define PGSTAT_KIND_SLRU 14 +#define PGSTAT_KIND_WAL 15 #define PGSTAT_KIND_BUILTIN_MIN PGSTAT_KIND_DATABASE #define PGSTAT_KIND_BUILTIN_MAX PGSTAT_KIND_WAL diff --git a/src/backend/access/heap/heapam_xlog.c b/src/backend/access/heap/heapam_xlog.c index 9ed7024e8147..f1379c699900 100644 --- a/src/backend/access/heap/heapam_xlog.c +++ b/src/backend/access/heap/heapam_xlog.c @@ -19,6 +19,7 @@ #include "access/visibilitymap.h" #include "access/xlog.h" #include "access/xlogutils.h" +#include "pgstat.h" #include "storage/freespace.h" #include "storage/standby.h" @@ -357,6 +358,9 @@ heap_xlog_delete(XLogReaderState *record) } if (BufferIsValid(buffer)) UnlockReleaseBuffer(buffer); + + pgstat_wal_replay_delete(target_locator.dbOid, + target_locator.relNumber); } /* @@ -483,6 +487,9 @@ heap_xlog_insert(XLogReaderState *record) */ if (action == BLK_NEEDS_REDO && freespace < BLCKSZ / 5) XLogRecordPageWithFreeSpace(target_locator, blkno, freespace); + + pgstat_wal_replay_insert(target_locator.dbOid, + target_locator.relNumber, 1); } /* @@ -688,6 +695,9 @@ heap_xlog_multi_insert(XLogReaderState *record) */ if (action == BLK_NEEDS_REDO && freespace < BLCKSZ / 5) XLogRecordPageWithFreeSpace(rlocator, blkno, freespace); + + pgstat_wal_replay_insert(rlocator.dbOid, rlocator.relNumber, + xlrec->ntuples); } /* @@ -967,6 +977,9 @@ heap_xlog_update(XLogReaderState *record, bool hot_update) */ if (newaction == BLK_NEEDS_REDO && !hot_update && freespace < BLCKSZ / 5) XLogRecordPageWithFreeSpace(rlocator, newblk, freespace); + + pgstat_wal_replay_update(rlocator.dbOid, rlocator.relNumber, + hot_update, (oldblk != newblk && !hot_update)); } /* diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql index ce991e4269e7..2577fe608f23 100644 --- a/src/backend/catalog/system_views.sql +++ b/src/backend/catalog/system_views.sql @@ -726,15 +726,15 @@ CREATE VIEW pg_stat_all_tables AS max(pg_stat_get_idx_lastscan(I.indexrelid)) AS last_idx_scan, sum(pg_stat_get_idx_tuples_fetched(I.indexrelid))::bigint + pg_stat_get_tuples_fetched(C.oid) AS idx_tup_fetch, - pg_stat_get_tuples_inserted(C.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(C.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(C.oid) AS n_tup_del, - pg_stat_get_tuples_hot_updated(C.oid) AS n_tup_hot_upd, - pg_stat_get_tuples_newpage_updated(C.oid) AS n_tup_newpage_upd, - pg_stat_get_live_tuples(C.oid) AS n_live_tup, - pg_stat_get_dead_tuples(C.oid) AS n_dead_tup, - pg_stat_get_mod_since_analyze(C.oid) AS n_mod_since_analyze, - pg_stat_get_ins_since_vacuum(C.oid) AS n_ins_since_vacuum, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(C.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(C.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(C.oid)) AS n_tup_del, + pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode(C.oid)) AS n_tup_hot_upd, + pg_stat_get_rfn_tuples_newpage_updated(pg_relation_filenode(C.oid)) AS n_tup_newpage_upd, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(C.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(C.oid)) AS n_dead_tup, + pg_stat_get_rfn_mod_since_analyze(pg_relation_filenode(C.oid)) AS n_mod_since_analyze, + pg_stat_get_rfn_ins_since_vacuum(pg_relation_filenode(C.oid)) AS n_ins_since_vacuum, pg_stat_get_last_vacuum_time(C.oid) as last_vacuum, pg_stat_get_last_autovacuum_time(C.oid) as last_autovacuum, pg_stat_get_last_analyze_time(C.oid) as last_analyze, diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index 45abf48768af..23e4637968f8 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -109,6 +109,7 @@ #include "utils/memutils.h" #include "utils/ps_status.h" #include "utils/rel.h" +#include "utils/relmapper.h" #include "utils/snapmgr.h" #include "utils/syscache.h" #include "utils/timeout.h" @@ -3082,10 +3083,10 @@ relation_needs_vacanalyze(Oid relid, bool *wraparound, AutoVacuumScores *scores) { - PgStat_StatTabEntry *tabentry; + PgStat_StatRFNodeEntry *rfnentry; + RelFileNumber rfn; bool force_vacuum; bool av_enabled; - bool may_free = false; /* constants from reloptions or GUC variables */ int vac_base_thresh, @@ -3247,20 +3248,27 @@ relation_needs_vacanalyze(Oid relid, *dovacuum = true; /* - * If we found stats for the table, and autovacuum is currently enabled, - * make a threshold-based decision whether to vacuum and/or analyze. If - * autovacuum is currently disabled, we must be here for anti-wraparound - * vacuuming only, so don't vacuum (or analyze) anything that's not being - * forced. + * Fetch tuple counters from the relfilenode entry. For shared catalogs, + * use the relation map. */ - tabentry = pgstat_fetch_stat_tabentry_ext(classForm->relisshared, - relid, &may_free); - if (!tabentry) - return; + rfn = classForm->relfilenode; + if (!RelFileNumberIsValid(rfn)) + rfn = RelationMapOidToFilenumber(relid, classForm->relisshared); - vactuples = tabentry->dead_tuples; - instuples = tabentry->ins_since_vacuum; - anltuples = tabentry->mod_since_analyze; + rfnentry = pgstat_fetch_stat_rfnodeentry( + classForm->relisshared ? InvalidOid : MyDatabaseId, rfn); + if (rfnentry) + { + vactuples = rfnentry->dead_tuples; + instuples = rfnentry->ins_since_vacuum; + anltuples = rfnentry->mod_since_analyze; + } + else + { + vactuples = 0; + instuples = 0; + anltuples = 0; + } /* If the table hasn't yet been vacuumed, take reltuples as zero */ if (reltuples < 0) @@ -3334,10 +3342,6 @@ relation_needs_vacanalyze(Oid relid, vactuples, vacthresh, scores->vac, anltuples, anlthresh, scores->anl, scores->xid, scores->mxid); - - /* Avoid leaking pgstat entries until the end of autovacuum. */ - if (may_free) - pfree(tabentry); } /* diff --git a/src/backend/utils/activity/Makefile b/src/backend/utils/activity/Makefile index 2e32d1485d64..7ee4c82e1b65 100644 --- a/src/backend/utils/activity/Makefile +++ b/src/backend/utils/activity/Makefile @@ -30,6 +30,7 @@ OBJS = \ pgstat_kind.o \ pgstat_lock.o \ pgstat_relation.o \ + pgstat_relfilenode.o \ pgstat_replslot.o \ pgstat_shmem.o \ pgstat_slru.o \ diff --git a/src/backend/utils/activity/meson.build b/src/backend/utils/activity/meson.build index e6dcb2e26fc5..754e120dd612 100644 --- a/src/backend/utils/activity/meson.build +++ b/src/backend/utils/activity/meson.build @@ -15,6 +15,7 @@ backend_sources += files( 'pgstat_kind.c', 'pgstat_lock.c', 'pgstat_relation.c', + 'pgstat_relfilenode.c', 'pgstat_replslot.c', 'pgstat_shmem.c', 'pgstat_slru.c', diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c index 907d1f53a7ec..e545b1c7aaac 100644 --- a/src/backend/utils/activity/pgstat.c +++ b/src/backend/utils/activity/pgstat.c @@ -333,6 +333,21 @@ static const PgStat_KindInfo pgstat_kind_builtin_infos[PGSTAT_KIND_BUILTIN_SIZE] .reset_timestamp_cb = pgstat_index_reset_timestamp_cb, }, + [PGSTAT_KIND_RELFILENODE] = { + .name = "relfilenode", + + .fixed_amount = false, + .write_to_file = true, + + .shared_size = sizeof(PgStatShared_RelFileNode), + .shared_data_off = offsetof(PgStatShared_RelFileNode, stats), + .shared_data_len = sizeof(((PgStatShared_RelFileNode *) 0)->stats), + .pending_size = sizeof(PgStat_RelationStatus), + + .flush_pending_cb = pgstat_relfilenode_flush_cb, + .reset_timestamp_cb = pgstat_relfilenode_reset_timestamp_cb, + }, + [PGSTAT_KIND_FUNCTION] = { .name = "function", diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c index 0cfa5be49cde..7dbcc22fa78f 100644 --- a/src/backend/utils/activity/pgstat_relation.c +++ b/src/backend/utils/activity/pgstat_relation.c @@ -37,6 +37,7 @@ typedef struct TwoPhasePgStatRecord PgStat_Counter updated_pre_truncdrop; PgStat_Counter deleted_pre_truncdrop; Oid id; /* table's OID */ + RelFileNumber relfilenode; /* table's relfilenode at prepare time */ bool shared; /* is it a shared catalog? */ bool truncdropped; /* was the relation truncated/dropped? */ } TwoPhasePgStatRecord; @@ -182,6 +183,13 @@ pgstat_assoc_relation(Relation rel) RelationGetRelid(rel), rel->rd_rel->relisshared); + /* Store relfilenode for PGSTAT_KIND_RELFILENODE flush */ + if (rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE && + rel->rd_rel->relkind != RELKIND_INDEX) + rel->pgstat_info->tab.relfilenode = rel->rd_locator.relNumber; + else + rel->pgstat_info->tab.relfilenode = InvalidRelFileNumber; + /* don't allow link a stats to multiple relcache entries */ Assert(rel->pgstat_info->relation == NULL); @@ -264,8 +272,11 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, PgStat_Counter deadtuples, TimestampTz starttime) { PgStat_EntryRef *entry_ref; + PgStat_EntryRef *rfn_ref; PgStatShared_Relation *shtabentry; + PgStatShared_RelFileNode *shrfnstats; PgStat_StatTabEntry *tabentry; + PgStat_StatRFNodeEntry *rfnentry; Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId); TimestampTz ts; PgStat_Counter elapsedtime; @@ -284,21 +295,6 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats; tabentry = &shtabentry->stats; - tabentry->live_tuples = livetuples; - tabentry->dead_tuples = deadtuples; - - /* - * It is quite possible that a non-aggressive VACUUM ended up skipping - * various pages, however, we'll zero the insert counter here regardless. - * It's currently used only to track when we need to perform an "insert" - * autovacuum, which are mainly intended to freeze newly inserted tuples. - * Zeroing this may just mean we'll not try to vacuum the table again - * until enough tuples have been inserted to trigger another insert - * autovacuum. An anti-wraparound autovacuum will catch any persistent - * stragglers. - */ - tabentry->ins_since_vacuum = 0; - if (AmAutoVacuumWorkerProcess()) { tabentry->last_autovacuum_time = ts; @@ -314,6 +310,34 @@ pgstat_report_vacuum(Relation rel, PgStat_Counter livetuples, pgstat_unlock_entry(entry_ref); + /* Update the relfilenode entry */ + rfn_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, + (uint64) rel->rd_locator.relNumber, + false); + if (rfn_ref) + { + shrfnstats = (PgStatShared_RelFileNode *) rfn_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->live_tuples = livetuples; + rfnentry->dead_tuples = deadtuples; + + /* + * It is quite possible that a non-aggressive VACUUM ended up + * skipping various pages, however, we'll zero the insert counter + * here regardless. It's currently used only to track when we need + * to perform an "insert" autovacuum, which are mainly intended to + * freeze newly inserted tuples. Zeroing this may just mean we'll + * not try to vacuum the table again until enough tuples have been + * inserted to trigger another insert autovacuum. An + * anti-wraparound autovacuum will catch any persistent stragglers. + */ + rfnentry->ins_since_vacuum = 0; + + pgstat_unlock_entry(rfn_ref); + } + /* * Flush IO statistics now. pgstat_report_stat() will flush IO stats, * however this will not be called until after an entire autovacuum cycle @@ -336,8 +360,11 @@ pgstat_report_analyze(Relation rel, bool resetcounter, TimestampTz starttime) { PgStat_EntryRef *entry_ref; + PgStat_EntryRef *rfn_ref; PgStatShared_Relation *shtabentry; + PgStatShared_RelFileNode *shrfnstats; PgStat_StatTabEntry *tabentry; + PgStat_StatRFNodeEntry *rfnentry; Oid dboid = (rel->rd_rel->relisshared ? InvalidOid : MyDatabaseId); TimestampTz ts; PgStat_Counter elapsedtime; @@ -388,17 +415,6 @@ pgstat_report_analyze(Relation rel, shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats; tabentry = &shtabentry->stats; - tabentry->live_tuples = livetuples; - tabentry->dead_tuples = deadtuples; - - /* - * If commanded, reset mod_since_analyze to zero. This forgets any - * changes that were committed while the ANALYZE was in progress, but we - * have no good way to estimate how many of those there were. - */ - if (resetcounter) - tabentry->mod_since_analyze = 0; - if (AmAutoVacuumWorkerProcess()) { tabentry->last_autoanalyze_time = ts; @@ -414,6 +430,30 @@ pgstat_report_analyze(Relation rel, pgstat_unlock_entry(entry_ref); + /* Update the relfilenode entry */ + rfn_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, + (uint64) rel->rd_locator.relNumber, + false); + if (rfn_ref) + { + shrfnstats = (PgStatShared_RelFileNode *) rfn_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->live_tuples = livetuples; + rfnentry->dead_tuples = deadtuples; + + /* + * If commanded, reset mod_since_analyze to zero. This forgets any + * changes that were committed while the ANALYZE was in progress, but + * we have no good way to estimate how many of those there were. + */ + if (resetcounter) + rfnentry->mod_since_analyze = 0; + + pgstat_unlock_entry(rfn_ref); + } + /* see pgstat_report_vacuum() */ pgstat_flush_io(false); (void) pgstat_flush_backend(false, PGSTAT_BACKEND_FLUSH_IO); @@ -767,6 +807,7 @@ AtPrepare_PgStat_Relations(PgStat_SubXactStatus *xact_state) record.updated_pre_truncdrop = trans->updated_pre_truncdrop; record.deleted_pre_truncdrop = trans->deleted_pre_truncdrop; record.id = tabstat->tab.id; + record.relfilenode = tabstat->tab.relfilenode; record.shared = tabstat->tab.shared; record.truncdropped = trans->truncdropped; @@ -811,6 +852,7 @@ pgstat_twophase_postcommit(FullTransactionId fxid, uint16 info, /* Find or create a tabstat entry for the rel */ pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); + pgstat_info->tab.relfilenode = rec->relfilenode; /* Same math as in AtEOXact_PgStat, commit case */ pgstat_info->tab.counts.tuples_inserted += rec->tuples_inserted; @@ -847,6 +889,7 @@ pgstat_twophase_postabort(FullTransactionId fxid, uint16 info, /* Find or create a tabstat entry for the rel */ pgstat_info = pgstat_prep_relation_pending(PGSTAT_KIND_RELATION, rec->id, rec->shared); + pgstat_info->tab.relfilenode = rec->relfilenode; /* Same math as in AtEOXact_PgStat, abort case */ if (rec->truncdropped) @@ -875,10 +918,14 @@ bool pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) { Oid dboid; + RelFileNumber rfn; PgStat_RelationStatus *lstats; /* pending stats entry */ PgStatShared_Relation *shtabstats; PgStat_StatTabEntry *tabentry; /* table entry of shared stats */ PgStat_StatDBEntry *dbentry; /* pending database entry */ + PgStat_EntryRef *rfn_ref; + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; dboid = entry_ref->shared_entry->key.dboid; lstats = (PgStat_RelationStatus *) entry_ref->pending; @@ -908,45 +955,70 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) } tabentry->tuples_returned += lstats->tab.counts.tuples_returned; tabentry->tuples_fetched += lstats->tab.counts.tuples_fetched; - tabentry->tuples_inserted += lstats->tab.counts.tuples_inserted; - tabentry->tuples_updated += lstats->tab.counts.tuples_updated; - tabentry->tuples_deleted += lstats->tab.counts.tuples_deleted; - tabentry->tuples_hot_updated += lstats->tab.counts.tuples_hot_updated; - tabentry->tuples_newpage_updated += lstats->tab.counts.tuples_newpage_updated; - - /* - * If table was truncated/dropped, first reset the live/dead counters. - */ - if (lstats->tab.counts.truncdropped) - { - tabentry->live_tuples = 0; - tabentry->dead_tuples = 0; - tabentry->ins_since_vacuum = 0; - } - - tabentry->live_tuples += lstats->tab.counts.delta_live_tuples; - tabentry->dead_tuples += lstats->tab.counts.delta_dead_tuples; - tabentry->mod_since_analyze += lstats->tab.counts.changed_tuples; - - /* - * Using tuples_inserted to update ins_since_vacuum does mean that we'll - * track aborted inserts too. This isn't ideal, but otherwise probably - * not worth adding an extra field for. It may just amount to autovacuums - * triggering for inserts more often than they maybe should, which is - * probably not going to be common enough to be too concerned about here. - */ - tabentry->ins_since_vacuum += lstats->tab.counts.tuples_inserted; tabentry->blocks_fetched += lstats->tab.counts.blocks_fetched; tabentry->blocks_hit += lstats->tab.counts.blocks_hit; - /* Clamp live_tuples in case of negative delta_live_tuples */ - tabentry->live_tuples = Max(tabentry->live_tuples, 0); - /* Likewise for dead_tuples */ - tabentry->dead_tuples = Max(tabentry->dead_tuples, 0); - pgstat_unlock_entry(entry_ref); + /* + * Flush tuple counters to the relfilenode entry. + * + * Use the current relfilenode from the relation (if available) since it + * may have changed due to TRUNCATE. Fall back to the cached value. + */ + rfn = InvalidRelFileNumber; + + if (lstats->relation) + rfn = lstats->relation->rd_locator.relNumber; + else if (lstats->tab.relfilenode != InvalidRelFileNumber) + rfn = lstats->tab.relfilenode; + + if (rfn != InvalidRelFileNumber) + { + rfn_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, + (uint64) rfn, + nowait); + if (rfn_ref) + { + shrfnstats = (PgStatShared_RelFileNode *) rfn_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->tuples_inserted += lstats->tab.counts.tuples_inserted; + rfnentry->tuples_updated += lstats->tab.counts.tuples_updated; + rfnentry->tuples_deleted += lstats->tab.counts.tuples_deleted; + rfnentry->tuples_hot_updated += lstats->tab.counts.tuples_hot_updated; + rfnentry->tuples_newpage_updated += lstats->tab.counts.tuples_newpage_updated; + + if (lstats->tab.counts.truncdropped) + { + rfnentry->live_tuples = 0; + rfnentry->dead_tuples = 0; + rfnentry->ins_since_vacuum = 0; + } + + rfnentry->live_tuples += lstats->tab.counts.delta_live_tuples; + rfnentry->dead_tuples += lstats->tab.counts.delta_dead_tuples; + rfnentry->mod_since_analyze += lstats->tab.counts.changed_tuples; + + /* + * Using tuples_inserted to update ins_since_vacuum means we + * track aborted inserts too. This isn't ideal, but not worth + * adding an extra field for — it may just trigger autovacuums + * slightly more often than necessary. + */ + rfnentry->ins_since_vacuum += lstats->tab.counts.tuples_inserted; + + /* Clamp live_tuples in case of negative delta_live_tuples */ + rfnentry->live_tuples = Max(rfnentry->live_tuples, 0); + /* Likewise for dead_tuples */ + rfnentry->dead_tuples = Max(rfnentry->dead_tuples, 0); + + pgstat_unlock_entry(rfn_ref); + } + } + /* The entry was successfully flushed, add the same to database stats */ dbentry = pgstat_prep_database_pending(dboid); dbentry->tuples_returned += lstats->tab.counts.tuples_returned; diff --git a/src/backend/utils/activity/pgstat_relfilenode.c b/src/backend/utils/activity/pgstat_relfilenode.c new file mode 100644 index 000000000000..82613fb1a4ee --- /dev/null +++ b/src/backend/utils/activity/pgstat_relfilenode.c @@ -0,0 +1,179 @@ +/* ------------------------------------------------------------------------- + * + * pgstat_relfilenode.c + * Implementation of per-relfilenode statistics. + * + * This file contains the implementation of PGSTAT_KIND_RELFILENODE statistics, + * for tuple counters (inserts, updates, deletes, live/dead tuples, etc.) keyed + * by relfilenode. These counters are rebuilt during crash recovery, from WAL + * replayed. + * + * Copyright (c) 2001-2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/backend/utils/activity/pgstat_relfilenode.c + * ------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include "utils/memutils.h" +#include "utils/pgstat_internal.h" + + +/* + * Flush out pending stats for a relfilenode entry. + * + * If nowait is true and the lock could not be immediately acquired, returns + * false without flushing the entry. Otherwise returns true. + */ +bool +pgstat_relfilenode_flush_cb(PgStat_EntryRef *entry_ref, bool nowait) +{ + PgStat_RelationStatus *lstats; /* pending stats entry */ + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; + + lstats = (PgStat_RelationStatus *) entry_ref->pending; + shrfnstats = (PgStatShared_RelFileNode *) entry_ref->shared_stats; + + /* + * Ignore entries that didn't accumulate any actual counts, such as + * tables that were opened but not modified. + */ + if (pg_memory_is_all_zeros(&lstats->tab.counts, + sizeof(struct PgStat_TableCounts))) + return true; + + if (!pgstat_lock_entry(entry_ref, nowait)) + return false; + + /* Add the values to the shared entry. */ + rfnentry = &shrfnstats->stats; + + /* + * If table was truncated/dropped, first reset the live/dead counters. + */ + if (lstats->tab.counts.truncdropped) + { + rfnentry->live_tuples = 0; + rfnentry->dead_tuples = 0; + rfnentry->ins_since_vacuum = 0; + } + + rfnentry->live_tuples += lstats->tab.counts.delta_live_tuples; + rfnentry->dead_tuples += lstats->tab.counts.delta_dead_tuples; + rfnentry->mod_since_analyze += lstats->tab.counts.changed_tuples; + + /* + * Using tuples_inserted to update ins_since_vacuum means we track + * aborted inserts too. This isn't ideal, but not worth adding an extra + * field for — it may just trigger autovacuums slightly more often than + * necessary. + */ + rfnentry->ins_since_vacuum += lstats->tab.counts.tuples_inserted; + + /* Clamp live_tuples in case of negative delta_live_tuples */ + rfnentry->live_tuples = Max(rfnentry->live_tuples, 0); + /* Likewise for dead_tuples */ + rfnentry->dead_tuples = Max(rfnentry->dead_tuples, 0); + + pgstat_unlock_entry(entry_ref); + + return true; +} + +/* + * Callback to reset the timestamp on a relfilenode stats entry. + */ +void +pgstat_relfilenode_reset_timestamp_cb(PgStatShared_Common *header, + TimestampTz ts) +{ + ((PgStatShared_RelFileNode *) header)->stats.stat_reset_time = ts; +} + +/* + * Fetch the relfilenode stats entry for the given database and relfilenode. + */ +PgStat_StatRFNodeEntry * +pgstat_fetch_stat_rfnodeentry(Oid dboid, RelFileNumber rfn) +{ + return (PgStat_StatRFNodeEntry *) + pgstat_fetch_entry(PGSTAT_KIND_RELFILENODE, dboid, (uint64) rfn, NULL); +} + +/* + * WAL replay: increment tuple counters in the relfilenode entry. + * + * This is called during WAL replay to rebuild tuple counters from + * heap WAL records. It directly updates the shared stats entry without + * going through the pending/flush machinery. + */ +void +pgstat_wal_replay_insert(Oid dboid, RelFileNumber relfilenode, int ntuples) +{ + PgStat_EntryRef *entry_ref; + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; + + entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, (uint64) relfilenode, + false); + shrfnstats = (PgStatShared_RelFileNode *) entry_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->tuples_inserted += ntuples; + rfnentry->mod_since_analyze += ntuples; + rfnentry->ins_since_vacuum += ntuples; + rfnentry->live_tuples += ntuples; + + pgstat_unlock_entry(entry_ref); +} + +void +pgstat_wal_replay_update(Oid dboid, RelFileNumber relfilenode, + bool hot, bool newpage) +{ + PgStat_EntryRef *entry_ref; + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; + + entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, (uint64) relfilenode, + false); + shrfnstats = (PgStatShared_RelFileNode *) entry_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->tuples_updated++; + rfnentry->mod_since_analyze++; + rfnentry->dead_tuples++; + if (hot) + rfnentry->tuples_hot_updated++; + if (newpage) + rfnentry->tuples_newpage_updated++; + + pgstat_unlock_entry(entry_ref); +} + +void +pgstat_wal_replay_delete(Oid dboid, RelFileNumber relfilenode) +{ + PgStat_EntryRef *entry_ref; + PgStatShared_RelFileNode *shrfnstats; + PgStat_StatRFNodeEntry *rfnentry; + + entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELFILENODE, + dboid, (uint64) relfilenode, + false); + shrfnstats = (PgStatShared_RelFileNode *) entry_ref->shared_stats; + rfnentry = &shrfnstats->stats; + + rfnentry->tuples_deleted++; + rfnentry->mod_since_analyze++; + rfnentry->dead_tuples++; + rfnentry->live_tuples--; + rfnentry->live_tuples = Max(rfnentry->live_tuples, 0); + + pgstat_unlock_entry(entry_ref); +} diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c index 0d47d745c18f..8be2cb4d1293 100644 --- a/src/backend/utils/adt/pgstatfuncs.c +++ b/src/backend/utils/adt/pgstatfuncs.c @@ -69,45 +69,65 @@ PG_STAT_GET_RELENTRY_INT64(blocks_fetched) /* pg_stat_get_blocks_hit */ PG_STAT_GET_RELENTRY_INT64(blocks_hit) -/* pg_stat_get_dead_tuples */ -PG_STAT_GET_RELENTRY_INT64(dead_tuples) - -/* pg_stat_get_ins_since_vacuum */ -PG_STAT_GET_RELENTRY_INT64(ins_since_vacuum) - -/* pg_stat_get_live_tuples */ -PG_STAT_GET_RELENTRY_INT64(live_tuples) - -/* pg_stat_get_mod_since_analyze */ -PG_STAT_GET_RELENTRY_INT64(mod_since_analyze) - /* pg_stat_get_numscans */ PG_STAT_GET_RELENTRY_INT64(numscans) -/* pg_stat_get_tuples_deleted */ -PG_STAT_GET_RELENTRY_INT64(tuples_deleted) - /* pg_stat_get_tuples_fetched */ PG_STAT_GET_RELENTRY_INT64(tuples_fetched) -/* pg_stat_get_tuples_hot_updated */ -PG_STAT_GET_RELENTRY_INT64(tuples_hot_updated) - -/* pg_stat_get_tuples_newpage_updated */ -PG_STAT_GET_RELENTRY_INT64(tuples_newpage_updated) - -/* pg_stat_get_tuples_inserted */ -PG_STAT_GET_RELENTRY_INT64(tuples_inserted) - /* pg_stat_get_tuples_returned */ PG_STAT_GET_RELENTRY_INT64(tuples_returned) -/* pg_stat_get_tuples_updated */ -PG_STAT_GET_RELENTRY_INT64(tuples_updated) - /* pg_stat_get_vacuum_count */ PG_STAT_GET_RELENTRY_INT64(vacuum_count) +/* + * Accessor macro for relfilenode stats entries (PgStat_StatRFNodeEntry). + * Takes a relfilenode number as input (from pg_class.relfilenode). + */ +#define PG_STAT_GET_RFNENTRY_INT64(stat) \ +Datum \ +CppConcat(pg_stat_get_rfn_,stat)(PG_FUNCTION_ARGS) \ +{ \ + Oid rfn = PG_GETARG_OID(0); \ + int64 result; \ + PgStat_StatRFNodeEntry *rfnentry; \ + \ + if ((rfnentry = pgstat_fetch_stat_rfnodeentry(MyDatabaseId, rfn)) == NULL) \ + result = 0; \ + else \ + result = (int64) (rfnentry->stat); \ + \ + PG_RETURN_INT64(result); \ +} + +/* pg_stat_get_rfn_tuples_inserted */ +PG_STAT_GET_RFNENTRY_INT64(tuples_inserted) + +/* pg_stat_get_rfn_tuples_updated */ +PG_STAT_GET_RFNENTRY_INT64(tuples_updated) + +/* pg_stat_get_rfn_tuples_deleted */ +PG_STAT_GET_RFNENTRY_INT64(tuples_deleted) + +/* pg_stat_get_rfn_tuples_hot_updated */ +PG_STAT_GET_RFNENTRY_INT64(tuples_hot_updated) + +/* pg_stat_get_rfn_tuples_newpage_updated */ +PG_STAT_GET_RFNENTRY_INT64(tuples_newpage_updated) + +/* pg_stat_get_rfn_live_tuples */ +PG_STAT_GET_RFNENTRY_INT64(live_tuples) + +/* pg_stat_get_rfn_dead_tuples */ +PG_STAT_GET_RFNENTRY_INT64(dead_tuples) + +/* pg_stat_get_rfn_mod_since_analyze */ +PG_STAT_GET_RFNENTRY_INT64(mod_since_analyze) + +/* pg_stat_get_rfn_ins_since_vacuum */ +PG_STAT_GET_RFNENTRY_INT64(ins_since_vacuum) + /* * Accessor macro for index stats entries (PgStat_StatIdxEntry). */ diff --git a/src/test/isolation/expected/stats.out b/src/test/isolation/expected/stats.out index cfad309ccf34..1ede5af1cf81 100644 --- a/src/test/isolation/expected/stats.out +++ b/src/test/isolation/expected/stats.out @@ -2186,18 +2186,18 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count --------+------------+---------+---------+---------+----------+----------+------------ - 0| 0| 0| 0| 0| 0| 0| 0 + 0| 0| | | | | | 0 (1 row) @@ -2230,18 +2230,18 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count --------+------------+---------+---------+---------+----------+----------+------------ - 0| 0| 0| 0| 0| 0| 0| 0 + 0| 0| | | | | | 0 (1 row) @@ -2256,11 +2256,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2295,11 +2295,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2334,11 +2334,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2389,11 +2389,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2421,11 +2421,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2483,11 +2483,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2515,11 +2515,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2578,11 +2578,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2647,11 +2647,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2708,11 +2708,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2775,11 +2775,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2815,18 +2815,18 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count --------+------------+---------+---------+---------+----------+----------+------------ - 3| 9| 5| 1| 0| 1| 1| 0 + 3| 9| 0| 0| 0| 0| 0| 0 (1 row) @@ -2861,18 +2861,18 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' seq_scan|seq_tup_read|n_tup_ins|n_tup_upd|n_tup_del|n_live_tup|n_dead_tup|vacuum_count --------+------------+---------+---------+---------+----------+----------+------------ - 3| 9| 5| 1| 0| 1| 1| 0 + 3| 9| 0| 0| 0| 0| 0| 0 (1 row) @@ -2901,11 +2901,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2947,11 +2947,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -2988,11 +2988,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' @@ -3035,11 +3035,11 @@ step s1_table_stats: SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' diff --git a/src/test/isolation/specs/stats.spec b/src/test/isolation/specs/stats.spec index da16710da0fb..6eadc90463f2 100644 --- a/src/test/isolation/specs/stats.spec +++ b/src/test/isolation/specs/stats.spec @@ -102,11 +102,11 @@ step s1_table_stats { SELECT pg_stat_get_numscans(tso.oid) AS seq_scan, pg_stat_get_tuples_returned(tso.oid) AS seq_tup_read, - pg_stat_get_tuples_inserted(tso.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(tso.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(tso.oid) AS n_tup_del, - pg_stat_get_live_tuples(tso.oid) AS n_live_tup, - pg_stat_get_dead_tuples(tso.oid) AS n_dead_tup, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode(tso.oid)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode(tso.oid)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode(tso.oid)) AS n_tup_del, + pg_stat_get_rfn_live_tuples(pg_relation_filenode(tso.oid)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode(tso.oid)) AS n_dead_tup, pg_stat_get_vacuum_count(tso.oid) AS vacuum_count FROM test_stat_oid AS tso WHERE tso.name = 'test_stat_tab' diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out index ffec0903f038..3e3ef4f189b2 100644 --- a/src/test/regress/expected/rules.out +++ b/src/test/regress/expected/rules.out @@ -1825,15 +1825,15 @@ pg_stat_all_tables| SELECT c.oid AS relid, (sum(pg_stat_get_idx_numscans(i.indexrelid)))::bigint AS idx_scan, max(pg_stat_get_idx_lastscan(i.indexrelid)) AS last_idx_scan, ((sum(pg_stat_get_idx_tuples_fetched(i.indexrelid)))::bigint + pg_stat_get_tuples_fetched(c.oid)) AS idx_tup_fetch, - pg_stat_get_tuples_inserted(c.oid) AS n_tup_ins, - pg_stat_get_tuples_updated(c.oid) AS n_tup_upd, - pg_stat_get_tuples_deleted(c.oid) AS n_tup_del, - pg_stat_get_tuples_hot_updated(c.oid) AS n_tup_hot_upd, - pg_stat_get_tuples_newpage_updated(c.oid) AS n_tup_newpage_upd, - pg_stat_get_live_tuples(c.oid) AS n_live_tup, - pg_stat_get_dead_tuples(c.oid) AS n_dead_tup, - pg_stat_get_mod_since_analyze(c.oid) AS n_mod_since_analyze, - pg_stat_get_ins_since_vacuum(c.oid) AS n_ins_since_vacuum, + pg_stat_get_rfn_tuples_inserted(pg_relation_filenode((c.oid)::regclass)) AS n_tup_ins, + pg_stat_get_rfn_tuples_updated(pg_relation_filenode((c.oid)::regclass)) AS n_tup_upd, + pg_stat_get_rfn_tuples_deleted(pg_relation_filenode((c.oid)::regclass)) AS n_tup_del, + pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode((c.oid)::regclass)) AS n_tup_hot_upd, + pg_stat_get_rfn_tuples_newpage_updated(pg_relation_filenode((c.oid)::regclass)) AS n_tup_newpage_upd, + pg_stat_get_rfn_live_tuples(pg_relation_filenode((c.oid)::regclass)) AS n_live_tup, + pg_stat_get_rfn_dead_tuples(pg_relation_filenode((c.oid)::regclass)) AS n_dead_tup, + pg_stat_get_rfn_mod_since_analyze(pg_relation_filenode((c.oid)::regclass)) AS n_mod_since_analyze, + pg_stat_get_rfn_ins_since_vacuum(pg_relation_filenode((c.oid)::regclass)) AS n_ins_since_vacuum, pg_stat_get_last_vacuum_time(c.oid) AS last_vacuum, pg_stat_get_last_autovacuum_time(c.oid) AS last_autovacuum, pg_stat_get_last_analyze_time(c.oid) AS last_analyze, diff --git a/src/test/regress/expected/stats.out b/src/test/regress/expected/stats.out index b44e30f557c9..f7d573b890aa 100644 --- a/src/test/regress/expected/stats.out +++ b/src/test/regress/expected/stats.out @@ -128,14 +128,15 @@ SELECT id, name, fixed_amount, 5 | replslot | f | t | t 6 | subscription | f | t | t 7 | backend | f | t | f - 8 | archiver | t | f | t - 9 | bgwriter | t | f | t - 10 | checkpointer | t | f | t - 11 | io | t | f | t - 12 | lock | t | f | t - 13 | slru | t | f | t - 14 | wal | t | f | t -(14 rows) + 8 | relfilenode | f | f | t + 9 | archiver | t | f | t + 10 | bgwriter | t | f | t + 11 | checkpointer | t | f | t + 12 | io | t | f | t + 13 | lock | t | f | t + 14 | slru | t | f | t + 15 | wal | t | f | t +(15 rows) -- ensure that both seqscan and indexscan plans are allowed SET enable_seqscan TO on; @@ -241,9 +242,9 @@ SELECT relname, n_tup_ins, n_tup_upd, n_tup_del, n_live_tup, n_dead_tup WHERE relname like 'trunc_stats_test%' order by relname; relname | n_tup_ins | n_tup_upd | n_tup_del | n_live_tup | n_dead_tup -------------------+-----------+-----------+-----------+------------+------------ - trunc_stats_test | 3 | 0 | 0 | 0 | 0 - trunc_stats_test1 | 4 | 2 | 1 | 1 | 0 - trunc_stats_test2 | 1 | 0 | 0 | 1 | 0 + trunc_stats_test | 0 | 0 | 0 | 0 | 0 + trunc_stats_test1 | 0 | 0 | 0 | 0 | 0 + trunc_stats_test2 | 0 | 0 | 0 | 0 | 0 trunc_stats_test3 | 4 | 0 | 0 | 2 | 2 trunc_stats_test4 | 2 | 0 | 0 | 0 | 2 (5 rows) @@ -470,30 +471,30 @@ SELECT pg_stat_get_function_calls(:stats_test_func2_oid); -- by oid after the DROP TABLE. Save oids. CREATE TABLE drop_stats_test(); INSERT INTO drop_stats_test DEFAULT VALUES; -SELECT 'drop_stats_test'::regclass::oid AS drop_stats_test_oid \gset +SELECT pg_relation_filenode('drop_stats_test'::regclass) AS drop_stats_test_oid \gset CREATE TABLE drop_stats_test_xact(); INSERT INTO drop_stats_test_xact DEFAULT VALUES; -SELECT 'drop_stats_test_xact'::regclass::oid AS drop_stats_test_xact_oid \gset +SELECT pg_relation_filenode('drop_stats_test_xact'::regclass) AS drop_stats_test_xact_oid \gset CREATE TABLE drop_stats_test_subxact(); INSERT INTO drop_stats_test_subxact DEFAULT VALUES; -SELECT 'drop_stats_test_subxact'::regclass::oid AS drop_stats_test_subxact_oid \gset +SELECT pg_relation_filenode('drop_stats_test_subxact'::regclass) AS drop_stats_test_subxact_oid \gset SELECT pg_stat_force_next_flush(); pg_stat_force_next_flush -------------------------- (1 row) -SELECT pg_stat_get_live_tuples(:drop_stats_test_oid); - pg_stat_get_live_tuples -------------------------- - 1 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) DROP TABLE drop_stats_test; -SELECT pg_stat_get_live_tuples(:drop_stats_test_oid); - pg_stat_get_live_tuples -------------------------- - 0 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_oid); @@ -504,18 +505,18 @@ SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_oid); -- check that rollback protects against having stats dropped and that local -- modifications don't pose a problem -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); - pg_stat_get_live_tuples -------------------------- - 1 -(1 row) - -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); - pg_stat_get_tuples_inserted +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); + pg_stat_get_rfn_live_tuples ----------------------------- 1 (1 row) +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); + pg_stat_get_rfn_tuples_inserted +--------------------------------- + 1 +(1 row) + SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); pg_stat_get_xact_tuples_inserted ---------------------------------- @@ -544,29 +545,29 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); - pg_stat_get_live_tuples -------------------------- - 1 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); - pg_stat_get_tuples_inserted ------------------------------ - 2 +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); + pg_stat_get_rfn_tuples_inserted +--------------------------------- + 2 (1 row) -- transactional drop -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); - pg_stat_get_live_tuples -------------------------- - 1 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); - pg_stat_get_tuples_inserted ------------------------------ - 2 +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); + pg_stat_get_rfn_tuples_inserted +--------------------------------- + 2 (1 row) BEGIN; @@ -591,23 +592,23 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); - pg_stat_get_live_tuples -------------------------- - 0 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); - pg_stat_get_tuples_inserted ------------------------------ - 0 +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); + pg_stat_get_rfn_tuples_inserted +--------------------------------- + 2 (1 row) -- savepoint rollback (2 levels) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 1 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 1 (1 row) BEGIN; @@ -636,17 +637,17 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 3 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) -- savepoint rolback (1 level) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 3 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) BEGIN; @@ -655,17 +656,17 @@ DROP TABLE drop_stats_test_subxact; SAVEPOINT sp2; ROLLBACK TO SAVEPOINT sp1; COMMIT; -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 3 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) -- and now actually drop -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 3 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) BEGIN; @@ -674,10 +675,10 @@ DROP TABLE drop_stats_test_subxact; SAVEPOINT sp2; RELEASE SAVEPOINT sp1; COMMIT; -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); - pg_stat_get_live_tuples -------------------------- - 0 +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); + pg_stat_get_rfn_live_tuples +----------------------------- + 3 (1 row) DROP TABLE trunc_stats_test, trunc_stats_test1, trunc_stats_test2, trunc_stats_test3, trunc_stats_test4; @@ -945,7 +946,7 @@ SELECT (n_tup_ins + n_tup_upd) > 0 AS has_data FROM pg_stat_all_tables WHERE relid = 'pg_shdescription'::regclass; has_data ---------- - t + f (1 row) -- stats_reset may not be set for datid=0 and shared objects in @@ -1902,7 +1903,7 @@ DECLARE BEGIN -- we don't want to wait forever; loop will exit after 30 seconds FOR i IN 1 .. 300 LOOP - SELECT (pg_stat_get_tuples_hot_updated('brin_hot'::regclass::oid) > 0) INTO updated; + SELECT (pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode('brin_hot'::regclass::oid)) > 0) INTO updated; EXIT WHEN updated; -- wait a little @@ -1929,10 +1930,10 @@ SELECT wait_for_hot_stats(); (1 row) -SELECT pg_stat_get_tuples_hot_updated('brin_hot'::regclass::oid); - pg_stat_get_tuples_hot_updated --------------------------------- - 1 +SELECT pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode('brin_hot'::regclass::oid)); + pg_stat_get_rfn_tuples_hot_updated +------------------------------------ + 1 (1 row) DROP TABLE brin_hot; diff --git a/src/test/regress/expected/stats_rewrite.out b/src/test/regress/expected/stats_rewrite.out index 93752bab9cb3..4f341672e5f1 100644 --- a/src/test/regress/expected/stats_rewrite.out +++ b/src/test/regress/expected/stats_rewrite.out @@ -42,7 +42,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_rewrite_alone'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 1 | 1 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_2pc_rewrite_alone; @@ -66,7 +66,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 4 | 4 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_2pc; @@ -93,7 +93,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_multi'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 6 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_2pc_multi; @@ -114,7 +114,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_rewrite_alone_abort'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 1 | 1 | 0 + 2 | 1 | 1 (1 row) DROP TABLE test_2pc_rewrite_alone_abort; @@ -138,7 +138,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_abort'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 4 | 1 | 3 + 7 | 1 | 6 (1 row) DROP TABLE test_2pc_abort; @@ -169,7 +169,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_2pc_savepoint'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 3 | 3 + 0 | 0 | 0 (1 row) DROP TABLE test_2pc_savepoint; @@ -209,7 +209,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_alone'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 1 | 1 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_alone; @@ -232,7 +232,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 4 | 4 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test; @@ -258,7 +258,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_multi'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 6 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_multi; @@ -331,7 +331,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_savepoint'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 3 | 3 + 0 | 0 | 0 (1 row) DROP TABLE test_savepoint; diff --git a/src/test/regress/expected/stats_rewrite_1.out b/src/test/regress/expected/stats_rewrite_1.out index 909188b18fd7..32c8cf803ebe 100644 --- a/src/test/regress/expected/stats_rewrite_1.out +++ b/src/test/regress/expected/stats_rewrite_1.out @@ -230,7 +230,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_alone'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 1 | 1 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_alone; @@ -253,7 +253,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 4 | 4 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test; @@ -279,7 +279,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_multi'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 6 | 0 + 0 | 0 | 0 (1 row) DROP TABLE test_multi; @@ -352,7 +352,7 @@ SELECT n_tup_ins, n_live_tup, n_dead_tup FROM pg_stat_all_tables WHERE relname = 'test_savepoint'; n_tup_ins | n_live_tup | n_dead_tup -----------+------------+------------ - 6 | 3 | 3 + 0 | 0 | 0 (1 row) DROP TABLE test_savepoint; diff --git a/src/test/regress/sql/stats.sql b/src/test/regress/sql/stats.sql index d4623c32cd32..9188429e03ca 100644 --- a/src/test/regress/sql/stats.sql +++ b/src/test/regress/sql/stats.sql @@ -219,27 +219,27 @@ SELECT pg_stat_get_function_calls(:stats_test_func2_oid); -- by oid after the DROP TABLE. Save oids. CREATE TABLE drop_stats_test(); INSERT INTO drop_stats_test DEFAULT VALUES; -SELECT 'drop_stats_test'::regclass::oid AS drop_stats_test_oid \gset +SELECT pg_relation_filenode('drop_stats_test'::regclass) AS drop_stats_test_oid \gset CREATE TABLE drop_stats_test_xact(); INSERT INTO drop_stats_test_xact DEFAULT VALUES; -SELECT 'drop_stats_test_xact'::regclass::oid AS drop_stats_test_xact_oid \gset +SELECT pg_relation_filenode('drop_stats_test_xact'::regclass) AS drop_stats_test_xact_oid \gset CREATE TABLE drop_stats_test_subxact(); INSERT INTO drop_stats_test_subxact DEFAULT VALUES; -SELECT 'drop_stats_test_subxact'::regclass::oid AS drop_stats_test_subxact_oid \gset +SELECT pg_relation_filenode('drop_stats_test_subxact'::regclass) AS drop_stats_test_subxact_oid \gset SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples(:drop_stats_test_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_oid); DROP TABLE drop_stats_test; -SELECT pg_stat_get_live_tuples(:drop_stats_test_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_oid); SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_oid); -- check that rollback protects against having stats dropped and that local -- modifications don't pose a problem -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); BEGIN; INSERT INTO drop_stats_test_xact DEFAULT VALUES; @@ -248,12 +248,12 @@ DROP TABLE drop_stats_test_xact; SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); ROLLBACK; SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); -- transactional drop -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); BEGIN; INSERT INTO drop_stats_test_xact DEFAULT VALUES; SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); @@ -261,11 +261,11 @@ DROP TABLE drop_stats_test_xact; SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_xact_oid); COMMIT; SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples(:drop_stats_test_xact_oid); -SELECT pg_stat_get_tuples_inserted(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_xact_oid); +SELECT pg_stat_get_rfn_tuples_inserted(:drop_stats_test_xact_oid); -- savepoint rollback (2 levels) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); BEGIN; INSERT INTO drop_stats_test_subxact DEFAULT VALUES; SAVEPOINT sp1; @@ -277,27 +277,27 @@ ROLLBACK TO SAVEPOINT sp2; SELECT pg_stat_get_xact_tuples_inserted(:drop_stats_test_subxact_oid); COMMIT; SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); -- savepoint rolback (1 level) -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); BEGIN; SAVEPOINT sp1; DROP TABLE drop_stats_test_subxact; SAVEPOINT sp2; ROLLBACK TO SAVEPOINT sp1; COMMIT; -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); -- and now actually drop -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); BEGIN; SAVEPOINT sp1; DROP TABLE drop_stats_test_subxact; SAVEPOINT sp2; RELEASE SAVEPOINT sp1; COMMIT; -SELECT pg_stat_get_live_tuples(:drop_stats_test_subxact_oid); +SELECT pg_stat_get_rfn_live_tuples(:drop_stats_test_subxact_oid); DROP TABLE trunc_stats_test, trunc_stats_test1, trunc_stats_test2, trunc_stats_test3, trunc_stats_test4; DROP TABLE prevstats; @@ -898,7 +898,7 @@ DECLARE BEGIN -- we don't want to wait forever; loop will exit after 30 seconds FOR i IN 1 .. 300 LOOP - SELECT (pg_stat_get_tuples_hot_updated('brin_hot'::regclass::oid) > 0) INTO updated; + SELECT (pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode('brin_hot'::regclass::oid)) > 0) INTO updated; EXIT WHEN updated; -- wait a little @@ -923,7 +923,7 @@ UPDATE brin_hot SET val = -3 WHERE id = 42; \c - SELECT wait_for_hot_stats(); -SELECT pg_stat_get_tuples_hot_updated('brin_hot'::regclass::oid); +SELECT pg_stat_get_rfn_tuples_hot_updated(pg_relation_filenode('brin_hot'::regclass::oid)); DROP TABLE brin_hot; DROP FUNCTION wait_for_hot_stats(); diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out index 5ebae1cedc2f..eed714f7d588 100644 --- a/contrib/postgres_fdw/expected/postgres_fdw.out +++ b/contrib/postgres_fdw/expected/postgres_fdw.out @@ -12988,12 +12988,12 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples('public.simport_ftable'::regclass), - pg_stat_get_dead_tuples('public.simport_ftable'::regclass), +SELECT pg_stat_get_rfn_live_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), + pg_stat_get_rfn_dead_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), pg_stat_get_analyze_count('public.simport_ftable'::regclass); - pg_stat_get_live_tuples | pg_stat_get_dead_tuples | pg_stat_get_analyze_count --------------------------+-------------------------+--------------------------- - 0 | 0 | 1 + pg_stat_get_rfn_live_tuples | pg_stat_get_rfn_dead_tuples | pg_stat_get_analyze_count +-----------------------------+-----------------------------+--------------------------- + | | 1 (1 row) ALTER TABLE simport_table ALTER COLUMN c1 SET STATISTICS 0; @@ -13023,12 +13023,12 @@ SELECT pg_stat_force_next_flush(); (1 row) -SELECT pg_stat_get_live_tuples('public.simport_ftable'::regclass), - pg_stat_get_dead_tuples('public.simport_ftable'::regclass), +SELECT pg_stat_get_rfn_live_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), + pg_stat_get_rfn_dead_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), pg_stat_get_analyze_count('public.simport_ftable'::regclass); - pg_stat_get_live_tuples | pg_stat_get_dead_tuples | pg_stat_get_analyze_count --------------------------+-------------------------+--------------------------- - 4 | 0 | 1 + pg_stat_get_rfn_live_tuples | pg_stat_get_rfn_dead_tuples | pg_stat_get_analyze_count +-----------------------------+-----------------------------+--------------------------- + | | 1 (1 row) ANALYZE VERBOSE simport_ftable (c1); -- should work diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql index e868da00ace7..a3be2f260280 100644 --- a/contrib/postgres_fdw/sql/postgres_fdw.sql +++ b/contrib/postgres_fdw/sql/postgres_fdw.sql @@ -4590,8 +4590,8 @@ ANALYZE simport_table; SELECT pg_stat_reset_single_table_counters('public.simport_ftable'::regclass); ANALYZE VERBOSE simport_ftable; -- should work SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples('public.simport_ftable'::regclass), - pg_stat_get_dead_tuples('public.simport_ftable'::regclass), +SELECT pg_stat_get_rfn_live_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), + pg_stat_get_rfn_dead_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), pg_stat_get_analyze_count('public.simport_ftable'::regclass); ALTER TABLE simport_table ALTER COLUMN c1 SET STATISTICS 0; @@ -4612,8 +4612,8 @@ ANALYZE simport_table; SELECT pg_stat_reset_single_table_counters('public.simport_ftable'::regclass); ANALYZE VERBOSE simport_ftable; -- should work SELECT pg_stat_force_next_flush(); -SELECT pg_stat_get_live_tuples('public.simport_ftable'::regclass), - pg_stat_get_dead_tuples('public.simport_ftable'::regclass), +SELECT pg_stat_get_rfn_live_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), + pg_stat_get_rfn_dead_tuples(pg_relation_filenode('public.simport_ftable'::regclass)), pg_stat_get_analyze_count('public.simport_ftable'::regclass); ANALYZE VERBOSE simport_ftable (c1); -- should work -- 2.55.0 [application/pgp-signature] signature.asc (833B, ../../[email protected]/5-signature.asc) download ^ permalink raw reply [nested|flat] 30+ messages in thread
* Re: relfilenode statistics @ 2026-07-10 13:09 Andres Freund <[email protected]> parent: Michael Paquier <[email protected]> 0 siblings, 0 replies; 30+ messages in thread From: Andres Freund @ 2026-07-10 13:09 UTC (permalink / raw) To: Michael Paquier <[email protected]>; +Cc: Bertrand Drouvot <[email protected]>; Kirill Reshke <[email protected]>; Robert Haas <[email protected]>; [email protected] Hi, On 2026-07-10 15:05:35 +0900, Michael Paquier wrote: > On Mon, Mar 09, 2026 at 07:43:43AM +0000, Bertrand Drouvot wrote: > > PFA, a new mandatory rebase. > > So, we have discussed that a couple of days ago offline, and I still > felt uncomfortable to not do a cleanup of the relation stats data for > indexes. A lot of the fields are stored in memory and written in the > file, and remain unused for indexes. Why solve that as part of this thread / redesign, rather than as an orthogonal issue? Greetings, Andres Freund ^ permalink raw reply [nested|flat] 30+ messages in thread
end of thread, other threads:[~2026-07-10 13:09 UTC | newest] Thread overview: 30+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-04-25 23:23 Re: Direct SSL connection with ALPN and HBA rules Jacob Champion <[email protected]> 2024-04-29 08:38 ` Heikki Linnakangas <[email protected]> 2024-04-29 12:38 ` Robert Haas <[email protected]> 2024-04-29 18:43 ` Jacob Champion <[email protected]> 2024-04-29 19:06 ` Heikki Linnakangas <[email protected]> 2024-04-29 19:32 ` Jacob Champion <[email protected]> 2024-04-29 19:34 ` Jacob Champion <[email protected]> 2024-05-10 13:50 ` Heikki Linnakangas <[email protected]> 2024-05-11 20:45 ` Jelte Fennema-Nio <[email protected]> 2024-05-12 21:39 ` Heikki Linnakangas <[email protected]> 2024-05-13 09:50 ` Jelte Fennema-Nio <[email protected]> 2024-05-13 11:07 ` Heikki Linnakangas <[email protected]> 2024-05-13 14:19 ` Jelte Fennema-Nio <[email protected]> 2024-05-13 18:09 ` On the use of channel binding without server certificates (was: Direct SSL connection with ALPN and HBA rules) Jacob Champion <[email protected]> 2024-05-14 00:05 ` Jacob Champion <[email protected]> 2024-05-13 13:37 ` Heikki Linnakangas <[email protected]> 2024-05-13 13:55 ` Jelte Fennema-Nio <[email protected]> 2024-05-13 14:54 ` Heikki Linnakangas <[email protected]> 2024-05-13 16:13 ` Robert Haas <[email protected]> 2024-05-13 16:45 ` Jacob Champion <[email protected]> 2024-05-13 19:12 ` Robert Haas <[email protected]> 2024-05-13 17:41 ` Jelte Fennema-Nio <[email protected]> 2024-05-13 19:24 ` Robert Haas <[email protected]> 2024-05-13 22:29 ` Jacob Champion <[email protected]> 2024-05-15 13:33 ` Heikki Linnakangas <[email protected]> 2024-05-15 18:24 ` Jacob Champion <[email protected]> 2024-05-16 13:53 ` Robert Haas <[email protected]> 2024-04-30 10:10 ` Daniel Gustafsson <[email protected]> 2026-07-10 06:05 Re: relfilenode statistics Michael Paquier <[email protected]> 2026-07-10 13:09 ` Re: relfilenode statistics Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox