public inbox for [email protected]help / color / mirror / Atom feed
[PATCH] Remove PROC_IN_ANALYZE 4+ messages / 4 participants [nested] [flat]
* [PATCH] Remove PROC_IN_ANALYZE @ 2020-08-05 22:57 Alvaro Herrera <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Alvaro Herrera @ 2020-08-05 22:57 UTC (permalink / raw) --- src/backend/commands/analyze.c | 13 +------------ src/include/storage/proc.h | 3 +-- src/include/storage/procarray.h | 7 ------- 3 files changed, 2 insertions(+), 21 deletions(-) diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 924ef37c81..e0fa73ba79 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -247,11 +247,8 @@ analyze_rel(Oid relid, RangeVar *relation, } /* - * OK, let's do it. First let other backends know I'm in ANALYZE. + * OK, let's do it. First, initialize progress reporting. */ - LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); - MyPgXact->vacuumFlags |= PROC_IN_ANALYZE; - LWLockRelease(ProcArrayLock); pgstat_progress_start_command(PROGRESS_COMMAND_ANALYZE, RelationGetRelid(onerel)); @@ -279,14 +276,6 @@ analyze_rel(Oid relid, RangeVar *relation, relation_close(onerel, NoLock); pgstat_progress_end_command(); - - /* - * Reset my PGXACT flag. Note: we need this here, and not in vacuum_rel, - * because the vacuum flag is cleared by the end-of-xact code. - */ - LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); - MyPgXact->vacuumFlags &= ~PROC_IN_ANALYZE; - LWLockRelease(ProcArrayLock); } /* diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h index b20e2ad4f6..5ceb2494ba 100644 --- a/src/include/storage/proc.h +++ b/src/include/storage/proc.h @@ -52,7 +52,6 @@ struct XidCache */ #define PROC_IS_AUTOVACUUM 0x01 /* is it an autovac worker? */ #define PROC_IN_VACUUM 0x02 /* currently running lazy vacuum */ -#define PROC_IN_ANALYZE 0x04 /* currently running analyze */ #define PROC_VACUUM_FOR_WRAPAROUND 0x08 /* set by autovac only */ #define PROC_IN_LOGICAL_DECODING 0x10 /* currently doing logical * decoding outside xact */ @@ -60,7 +59,7 @@ struct XidCache /* flags reset at EOXact */ #define PROC_VACUUM_STATE_MASK \ - (PROC_IN_VACUUM | PROC_IN_ANALYZE | PROC_VACUUM_FOR_WRAPAROUND) + (PROC_IN_VACUUM | PROC_VACUUM_FOR_WRAPAROUND) /* * We allow a small number of "weak" relation locks (AccessShareLock, diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h index a5c7d0c064..01040d76e1 100644 --- a/src/include/storage/procarray.h +++ b/src/include/storage/procarray.h @@ -29,8 +29,6 @@ */ #define PROCARRAY_VACUUM_FLAG 0x02 /* currently running lazy * vacuum */ -#define PROCARRAY_ANALYZE_FLAG 0x04 /* currently running - * analyze */ #define PROCARRAY_LOGICAL_DECODING_FLAG 0x10 /* currently doing logical * decoding outside xact */ @@ -42,7 +40,6 @@ * have no corresponding PROC flag equivalent. */ #define PROCARRAY_PROC_FLAGS_MASK (PROCARRAY_VACUUM_FLAG | \ - PROCARRAY_ANALYZE_FLAG | \ PROCARRAY_LOGICAL_DECODING_FLAG) /* Use the following flags as an input "flags" to GetOldestXmin function */ @@ -50,10 +47,6 @@ #define PROCARRAY_FLAGS_DEFAULT PROCARRAY_LOGICAL_DECODING_FLAG /* Ignore vacuum backends */ #define PROCARRAY_FLAGS_VACUUM PROCARRAY_FLAGS_DEFAULT | PROCARRAY_VACUUM_FLAG -/* Ignore analyze backends */ -#define PROCARRAY_FLAGS_ANALYZE PROCARRAY_FLAGS_DEFAULT | PROCARRAY_ANALYZE_FLAG -/* Ignore both vacuum and analyze backends */ -#define PROCARRAY_FLAGS_VACUUM_ANALYZE PROCARRAY_FLAGS_DEFAULT | PROCARRAY_VACUUM_FLAG | PROCARRAY_ANALYZE_FLAG extern Size ProcArrayShmemSize(void); extern void CreateSharedProcArray(void); -- 2.20.1 --Kj7319i9nmIyA2yE-- ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: do only critical work during single-user vacuum? @ 2022-02-03 18:05 Robert Haas <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Robert Haas @ 2022-02-03 18:05 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; John Naylor <[email protected]>; PostgreSQL Hackers <[email protected]> On Thu, Dec 9, 2021 at 8:56 PM Andres Freund <[email protected]> wrote: > I think we should move *away* from single user mode, rather than the > opposite. It's a substantial code burden and it's hard to use. Yes. This thread seems to be largely devoted to the topic of making single-user vacuum work better, but I don't see anyone asking the question "why do we have a message that tells people to vacuum in single user mode in the first place?". It's basically bad advice, with one small exception that I'll talk about in a minute. Suppose we had a message in the tree that said "HINT: Consider angering a live anaconda to fix this problem." If that were so, the correct thing to do wouldn't be to add a section to our documentation explaining how to deal with angry anacondas. The correct thing to do would be to remove the hint as bad advice that we never should have offered in the first place. And so here. We should not try to make vacuum in single user-mode work better or differently, or at least that shouldn't be our primary objective. We should just stop telling people to do it. We should probably add messages and documentation *discouraging* the use of single user mode for recovering from wraparound trouble, exactly the opposite of what we do now. There's nothing we can do in single-user mode that we can't do equally well in multi-user mode. If people try to fix wraparound problems in multi-user mode, they still have read-only access to their database, they can use parallelism, they can use command line utilities like vacuumdb, and they can use psql which has line editing and allows remote access and is a way nicer user experience than running postgres --single. We need a really compelling reason to tell people to give up all those advantages, and there is no such reason. It makes just as much sense as telling people to deal with wraparound problems by angering a live anaconda. I did say there was an exception, and it's this: the last time I studied this issue back in 2019,[1] vacuum insisted on trying to truncate tables even when the system is in wraparound danger. Then it would fail, because truncating the table required allocating an XID, which would fail if we were short on XIDs. By putting the system in single user mode, you could continue to allocate XIDs and thus VACUUM would work. However, if you think about this for even 10 seconds, you can see that it's terrible. If we're so short of XIDs that we are scared to allocate them for fear of causing an actual wraparound, putting the system into a mode where that protection is bypassed is a super-terrible idea. People will be able to run vacuum, yes, but if they have too many tables, they will actually experience wraparound and thus data loss before they process all the tables they have. What we ought to do to solve this problem is NOT TRUNCATE when the number of remaining XIDs is small, so that we don't consume any of the remaining XIDs until we get the system out of wraparound danger. I think the "failsafe" stuff Peter added in v14 fixes that, though. If not, we should adjust it so it does. And then we should KILL WITH FIRE the message telling people to use single user mode -- and once we do that, the question of what the behavior ought to be when someone does run VACUUM in single user mode becomes a lot less important. This problem is basically self-inflicted. We have given people bad advice (use single user mode) and then they suffer when they take it. Ameliorating the suffering isn't the worst idea ever, but it's basically fixing the wrong problem. -- Robert Haas EDB: http://www.enterprisedb.com [1] http://postgr.es/m/CA+Tgmob1QCMJrHwRBK8HZtGsr+6cJANRQw2mEgJ9e=D+z7cOsw@mail.gmail.com ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: do only critical work during single-user vacuum? @ 2022-06-27 19:36 Justin Pryzby <[email protected]> parent: Robert Haas <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Justin Pryzby @ 2022-06-27 19:36 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Peter Geoghegan <[email protected]>; John Naylor <[email protected]>; [email protected]; Hannu Krosing <[email protected]> On Thu, Feb 03, 2022 at 01:05:50PM -0500, Robert Haas wrote: > On Thu, Dec 9, 2021 at 8:56 PM Andres Freund <[email protected]> wrote: > > I think we should move *away* from single user mode, rather than the > > opposite. It's a substantial code burden and it's hard to use. > > Yes. This thread seems to be largely devoted to the topic of making > single-user vacuum work better, but I don't see anyone asking the > question "why do we have a message that tells people to vacuum in > single user mode in the first place?". It's basically bad advice, > The correct thing to do would be to remove > the hint as bad advice that we never should have offered in the first > place. And so here. We should not try to make vacuum in single > user-mode work better or differently, or at least that shouldn't be > our primary objective. We should just stop telling people to do it. We > should probably add messages and documentation *discouraging* the use > of single user mode for recovering from wraparound trouble, exactly > the opposite of what we do now. There's nothing we can do in > single-user mode that we can't do equally well in multi-user mode. If > people try to fix wraparound problems in multi-user mode, they still > have read-only access to their database, they can use parallelism, > they can use command line utilities like vacuumdb, and they can use > psql which has line editing and allows remote access and is a way > nicer user experience than running postgres --single. We need a really > compelling reason to tell people to give up all those advantages, and > there is no such reason. It makes just as much sense as telling people > to deal with wraparound problems by angering a live anaconda. By chance, I came across this prior thread which advocated the same thing in a initially (rather than indirectly as in this year's thread). https://www.postgresql.org/message-id/flat/CAMT0RQTmRj_Egtmre6fbiMA9E2hM3BsLULiV8W00stwa3URvzA%40mai... |We should stop telling users to "vacuum that database in single-user mode" ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: do only critical work during single-user vacuum? @ 2022-06-27 20:36 Peter Geoghegan <[email protected]> parent: Justin Pryzby <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Peter Geoghegan @ 2022-06-27 20:36 UTC (permalink / raw) To: Justin Pryzby <[email protected]>; +Cc: Robert Haas <[email protected]>; Andres Freund <[email protected]>; John Naylor <[email protected]>; PostgreSQL Hackers <[email protected]>; Hannu Krosing <[email protected]> On Mon, Jun 27, 2022 at 12:36 PM Justin Pryzby <[email protected]> wrote: > By chance, I came across this prior thread which advocated the same thing in a > initially (rather than indirectly as in this year's thread). Revisiting this topic reminded me that PostgreSQL 14 (the first version that had the wraparound failsafe mechanism controlled by vacuum_failsafe_age) has been a stable release for 9 months now. As of today I am still not aware of even one user that ran into the failsafe mechanism in production. It might well have happened by now, of course, but I am not aware of any specific case. Perhaps this will change soon enough -- maybe somebody else will read this and enlighten me. To me the fact that the failsafe seems to seldom kick-in in practice suggests something about workload characteristics in general: that it isn't all that common for users to try to get away with putting off freezing until a table attains an age that is significantly above 1 billion XIDs. When people talk about things like 64-bit XIDs, I tend to wonder: if 2 billion XIDs wasn't enough, why should 4 billion or 8 billion be enough? *Maybe* the system can do better by getting even further into debt than it can today, but you can't expect to avoid freezing altogether (without significant work elsewhere). My general sense is that freezing isn't a particularly good thing to try to do lazily -- even if we ignore the risk of an eventual wraparound failure. -- Peter Geoghegan ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2022-06-27 20:36 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-08-05 22:57 [PATCH] Remove PROC_IN_ANALYZE Alvaro Herrera <[email protected]> 2022-02-03 18:05 Re: do only critical work during single-user vacuum? Robert Haas <[email protected]> 2022-06-27 19:36 ` Re: do only critical work during single-user vacuum? Justin Pryzby <[email protected]> 2022-06-27 20:36 ` Re: do only critical work during single-user vacuum? Peter Geoghegan <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox