public inbox for [email protected]
help / color / mirror / Atom feed[bug fix] Cascaded standby cannot start after a clean shutdown
91+ messages / 7 participants
[nested] [flat]
* [bug fix] Cascaded standby cannot start after a clean shutdown
@ 2018-02-14 04:37 Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Tsunakawa, Takayuki @ 2018-02-14 04:37 UTC (permalink / raw)
To: pgsql-hackers
Hello,
Our customer encountered a rare bug of PostgreSQL which prevents a cascaded standby from starting up. The attached patch is a fix for it. I hope this will be back-patched. I'll add this to the next CF.
PROBLEM
==============================
The PostgreSQL version is 9.5. The cluster consists of a master, a cascading standby (SB1), and a cascaded standby (SB2). The WAL flows like this: master -> SB1 -> SB2.
The user shut down SB2 and tried to restart it, but failed with the following message:
FATAL: invalid memory alloc request size 3075129344
The master and SB1 continued to run.
CAUSE
==============================
total_len in the code below was about 3GB, so palloc() rejected the memory allocation request.
[xlogreader.c]
record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ);
total_len = record->xl_tot_len;
...
/*
* Enlarge readRecordBuf as needed.
*/
if (total_len > state->readRecordBufSize &&
!allocate_recordbuf(state, total_len))
{
Why was XLogRecord->xl_tot_len so big? That's because the XLOG reader read the garbage portion in a WAL file, which is immediately after the end of valid WAL records.
Why was there the garbage? Because the cascading standby sends just the valid WAL records, not the whole WAL blocks, to the cascaded standby. When the cascaded standby receives those WAL records and write them in a recycled WAL file, the last WAL block in the file contains the mix of valid WAL records and the garbage left over.
Why did the XLOG reader try to allocate memory for a garbage? Doesn't it stop reading the WAL? As the following code shows, when the WAL record header crosses a WAL block boundary, the XLOG reader first allocates memory for the whole record, and then checks the validity of the record header as soon as it reads the entire header.
[xlogreader.c]
/*
* If the whole record header is on this page, validate it immediately.
* Otherwise do just a basic sanity check on xl_tot_len, and validate the
* rest of the header after reading it from the next page. The xl_tot_len
* check is necessary here to ensure that we enter the "Need to reassemble
* record" code path below; otherwise we might fail to apply
* ValidXLogRecordHeader at all.
*/
if (targetRecOff <= XLOG_BLCKSZ - SizeOfXLogRecord)
{
if (!ValidXLogRecordHeader(state, RecPtr, state->ReadRecPtr, record,
randAccess))
goto err;
gotheader = true;
}
else
{
/* XXX: more validation should be done here */
if (total_len < SizeOfXLogRecord)
{
report_invalid_record(state,
"invalid record length at %X/%X: wanted %u, got %u",
(uint32) (RecPtr >> 32), (uint32) RecPtr,
(uint32) SizeOfXLogRecord, total_len);
goto err;
}
gotheader = false;
}
FIX
==============================
One idea is to defer the memory allocation until the entire record header is read and ValidXLogRecordHeader() is called. However, ValidXLogRecordHeader() might misjudge the validity if the garbage contains xl_prev seeming correct.
So, I modified walreceiver to zero-fill the last partial WAL block. This is the guard that the master does in AdvanceXLInsertBuffer() as below:
/*
* Be sure to re-zero the buffer so that bytes beyond what we've
* written will look like zeroes and not valid XLOG records...
*/
MemSet((char *) NewPage, 0, XLOG_BLCKSZ);
FYI, the following unsolved problem may be solved, too.
https://www.postgresql.org/message-id/CAE2gYzzVZNsGn%3D-E6grO4sVQs04J02zNKQofQEO8gu8%3DqCFR%2BQ%40ma...
Regards
Takayuki Tsunakawa
Attachments:
[application/octet-stream] zerofill_walblock_on_standby.patch (2.4K, ../../0A3221C70F24FB45833433255569204D1F8B57AD@G01JPEXMBYT05/2-zerofill_walblock_on_standby.patch)
download | inline diff:
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index a39a98f..8f5fe4d 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -33,7 +33,7 @@
* specific parts are in the libpqwalreceiver module. It's loaded
* dynamically to avoid linking the server with libpq.
*
- * Portions Copyright (c) 2010-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 2010-2017, PostgreSQL Global Development Group
*
*
* IDENTIFICATION
@@ -944,6 +944,12 @@ XLogWalRcvWrite(char *buf, Size nbytes, XLogRecPtr recptr)
{
int startoff;
int byteswritten;
+ static char *zbuffer = NULL;
+ static int lastfile = -1;
+ static int lastblock = -1;
+
+ if (zbuffer == NULL)
+ zbuffer = palloc0(XLOG_BLCKSZ);
while (nbytes > 0)
{
@@ -1040,6 +1046,36 @@ XLogWalRcvWrite(char *buf, Size nbytes, XLogRecPtr recptr)
LogstreamResult.Write = recptr;
}
+
+ /*
+ * Zero-fill the remaining portion of the last block so that the XLOG reader won't get confused with the garbage left over from a recycled WAL segment.
+ * Remember the last zero-filled block to avoid filling the same block repeatedly.
+ */
+ if (recvOff % XLOG_BLCKSZ != 0 &&
+ !(recvFile == lastfile && recvOff / XLOG_BLCKSZ == lastblock))
+ {
+ int bytestowrite;
+
+ bytestowrite = XLOG_BLCKSZ - recvOff % XLOG_BLCKSZ;
+ errno = 0;
+ byteswritten = write(recvFile, zbuffer, bytestowrite);
+ if (byteswritten != bytestowrite)
+ {
+ /* if write didn't set errno, assume no disk space */
+ if (errno == 0)
+ errno = ENOSPC;
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not write to log segment %s "
+ "at offset %u, length %lu: %m",
+ XLogFileNameP(recvFileTLI, recvSegNo),
+ recvOff, (unsigned long) bytestowrite)));
+ }
+
+ lastfile = recvFile;
+ lastblock = recvOff / XLOG_BLCKSZ;
+ recvOff += byteswritten;
+ }
}
/*
@@ -1442,8 +1478,7 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
if (!is_member_of_role(GetUserId(), DEFAULT_ROLE_READ_ALL_STATS))
{
/*
- * Only superusers and members of pg_read_all_stats can see details.
- * Other users only get the pid value
+ * Only superusers can see details. Other users only get the pid value
* to know whether it is a WAL receiver, but no details.
*/
MemSet(&nulls[1], true, sizeof(bool) * (tupdesc->natts - 1));
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
@ 2018-02-16 07:19 ` Michael Paquier <[email protected]>
2018-02-16 08:06 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
0 siblings, 2 replies; 91+ messages in thread
From: Michael Paquier @ 2018-02-16 07:19 UTC (permalink / raw)
To: Tsunakawa, Takayuki <[email protected]>; +Cc: pgsql-hackers
On Wed, Feb 14, 2018 at 04:37:05AM +0000, Tsunakawa, Takayuki wrote:
> The PostgreSQL version is 9.5. The cluster consists of a master, a
> cascading standby (SB1), and a cascaded standby (SB2). The WAL flows
> like this: master -> SB1 -> SB2.
>
> The user shut down SB2 and tried to restart it, but failed with the
> following message:
>
> FATAL: invalid memory alloc request size 3075129344
>
> The master and SB1 continued to run.
This matches a couple of recent bug reports we have seen around like
this one:
https://www.postgresql.org/message-id/CAE2gYzzVZNsGn%3D-E6grO4sVQs04J02zNKQofQEO8gu8%3DqCFR%2BQ%40ma...
I recalled as well this one but in this case the user shot his own foot
with the failover scenario he designed:
https://www.postgresql.org/message-id/CAAc9rOyKAyzipiq7ee1%3DVbcRy2fRqV_hRujLbC0mrBkL07%3D7wQ%40mail...
> CAUSE
> ==============================
>
> total_len in the code below was about 3GB, so palloc() rejected the
> memory allocation request.
Yes, it is limited to 1GB.
> [xlogreader.c]
> record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ);
> total_len = record->xl_tot_len;
> ...
> /*
> * Enlarge readRecordBuf as needed.
> */
> if (total_len > state->readRecordBufSize &&
> !allocate_recordbuf(state, total_len))
> {
>
> Why was XLogRecord->xl_tot_len so big? That's because the XLOG reader
> read the garbage portion in a WAL file, which is immediately after the
> end of valid WAL records.
>
> Why was there the garbage? Because the cascading standby sends just
> the valid WAL records, not the whole WAL blocks, to the cascaded
> standby. When the cascaded standby receives those WAL records and
> write them in a recycled WAL file, the last WAL block in the file
> contains the mix of valid WAL records and the garbage left over.
Wait a minute here, when recycled past WAL segments would be filled with
zeros before being used.
> Why did the XLOG reader try to allocate memory for a garbage? Doesn't
> it stop reading the WAL? As the following code shows, when the WAL
> record header crosses a WAL block boundary, the XLOG reader first
> allocates memory for the whole record, and then checks the validity of
> the record header as soon as it reads the entire header.
>
> [...]
>
> FIX
> ==============================
>
> One idea is to defer the memory allocation until the entire record
> header is read and ValidXLogRecordHeader() is called. However,
> ValidXLogRecordHeader() might misjudge the validity if the garbage
> contains xl_prev seeming correct.
It seems to me that the consolidation of the page read should happen
directly in xlogreader.c and not even in one of its callbacks so as no
garbage data is presented back to the caller using its own XLogReader.
I think that you need to initialize XLogReaderState->readBuf directly in
ReadPageInternal() before reading a page and you should be good. With
your patch you get visibly only one portion of things addressed, what of
other code paths using xlogreader.c's APIs like pg_rewind, 2PC code and
such?
> FYI, the following unsolved problem may be solved, too.
>
> https://www.postgresql.org/message-id/CAE2gYzzVZNsGn%3D-E6grO4sVQs04J02zNKQofQEO8gu8%3DqCFR%2BQ%40ma...
Yeah, I noticed this one too before going through your message in
details ;)
Please note that your patch has some unnecessary garbage in two places:
- * Portions Copyright (c) 2010-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 2010-2017, PostgreSQL Global Development Group
[...]
- * Only superusers and members of pg_read_all_stats can see details.
- * Other users only get the pid value
+ * Only superusers can see details. Other users only get the pid valu
You may want to indent your patch as well before sending it.
+ if (zbuffer == NULL)
+ zbuffer = palloc0(XLOG_BLCKSZ);
You could just use a static buffer which is MemSet'd with 0 only if
necessary.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-02-16 08:06 ` Michael Paquier <[email protected]>
1 sibling, 0 replies; 91+ messages in thread
From: Michael Paquier @ 2018-02-16 08:06 UTC (permalink / raw)
To: Tsunakawa, Takayuki <[email protected]>; +Cc: pgsql-hackers
On Fri, Feb 16, 2018 at 04:19:00PM +0900, Michael Paquier wrote:
> Wait a minute here, when recycled past WAL segments would be filled with
> zeros before being used.
Please feel free to ignore this part. I pushed the "Send" button
without seeing it, and I was thinking uner which circumstances segments
get zero-filled, which is moot for this thread.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* RE: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-02-19 03:01 ` Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
1 sibling, 1 reply; 91+ messages in thread
From: Tsunakawa, Takayuki @ 2018-02-19 03:01 UTC (permalink / raw)
To: 'Michael Paquier' <[email protected]>; +Cc: pgsql-hackers
Thank you for reviewing.
From: Michael Paquier [mailto:[email protected]]
> It seems to me that the consolidation of the page read should happen directly
> in xlogreader.c and not even in one of its callbacks so as no garbage data
> is presented back to the caller using its own XLogReader.
> I think that you need to initialize XLogReaderState->readBuf directly in
> ReadPageInternal() before reading a page and you should be good. With your
> patch you get visibly only one portion of things addressed, what of other
> code paths using xlogreader.c's APIs like pg_rewind, 2PC code and such?
ReadPageInternal() doesn't know where the end of valid WAL is, so it cannot determine where to do memset(0). For example, in non-streaming cases, it reads the entire WAL block into readbuf, including the garbage.
/*
* If the current segment is being streamed from master, calculate how
* much of the current page we have received already. We know the
* requested record has been received, but this is for the benefit of
* future calls, to allow quick exit at the top of this function.
*/
if (readSource == XLOG_FROM_STREAM)
{
if (((targetPagePtr) / XLOG_BLCKSZ) != (receivedUpto / XLOG_BLCKSZ))
readLen = XLOG_BLCKSZ;
else
readLen = receivedUpto % XLogSegSize - targetPageOff;
}
else
readLen = XLOG_BLCKSZ;
So we have to zero-fill the empty space of a WAL block before writing it. Currently, the master does that in AdvanceXLInsertBuffer(), StartupXLOG(), XLogFileCopy(). The cascading standby receives WAL from the master block by block, so it doesn't suffer from the garbage. pg_receivexlog zero-fills a new WAL file.
> Please note that your patch has some unnecessary garbage in two places:
Ouch, cleaned up.
> + if (zbuffer == NULL)
> + zbuffer = palloc0(XLOG_BLCKSZ);
> You could just use a static buffer which is MemSet'd with 0 only if necessary.
I wanted to allocate memory only when necessary. Currently, only the cascaded standby needs this. To that end, I moved the memory allocation code to a right place. Thanks for letting me notice this.
Regards
Takayuki Tsunakawa
Attachments:
[application/octet-stream] zerofill_walblock_on_standby_v2.patch (1.6K, ../../0A3221C70F24FB45833433255569204D1F8C977C@G01JPEXMBYT05/2-zerofill_walblock_on_standby_v2.patch)
download | inline diff:
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index a39a98f..2393cb9 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -944,6 +944,9 @@ XLogWalRcvWrite(char *buf, Size nbytes, XLogRecPtr recptr)
{
int startoff;
int byteswritten;
+ static char *zbuffer = NULL;
+ static int lastfile = -1;
+ static int lastblock = -1;
while (nbytes > 0)
{
@@ -1040,6 +1043,39 @@ XLogWalRcvWrite(char *buf, Size nbytes, XLogRecPtr recptr)
LogstreamResult.Write = recptr;
}
+
+ /*
+ * Zero-fill the remaining portion of the last block so that the XLOG reader won't get confused with the garbage left over from a recycled WAL segment.
+ * Remember the last zero-filled block to avoid filling the same block repeatedly.
+ */
+ if (recvOff % XLOG_BLCKSZ != 0 &&
+ !(recvFile == lastfile && recvOff / XLOG_BLCKSZ == lastblock))
+ {
+ int bytestowrite;
+
+ if (zbuffer == NULL)
+ zbuffer = palloc0(XLOG_BLCKSZ);
+
+ bytestowrite = XLOG_BLCKSZ - recvOff % XLOG_BLCKSZ;
+ errno = 0;
+ byteswritten = write(recvFile, zbuffer, bytestowrite);
+ if (byteswritten != bytestowrite)
+ {
+ /* if write didn't set errno, assume no disk space */
+ if (errno == 0)
+ errno = ENOSPC;
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not write to log segment %s "
+ "at offset %u, length %lu: %m",
+ XLogFileNameP(recvFileTLI, recvSegNo),
+ recvOff, (unsigned long) bytestowrite)));
+ }
+
+ lastfile = recvFile;
+ lastblock = recvOff / XLOG_BLCKSZ;
+ recvOff += byteswritten;
+ }
}
/*
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
@ 2018-02-22 07:55 ` Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Michael Paquier @ 2018-02-22 07:55 UTC (permalink / raw)
To: Tsunakawa, Takayuki <[email protected]>; +Cc: pgsql-hackers
On Mon, Feb 19, 2018 at 03:01:15AM +0000, Tsunakawa, Takayuki wrote:
> From: Michael Paquier [mailto:[email protected]]
Sorry for my late reply. I was looking at this problem for the last
couple of days here and there, still thinking about it.
>> It seems to me that the consolidation of the page read should happen directly
>> in xlogreader.c and not even in one of its callbacks so as no garbage data
>> is presented back to the caller using its own XLogReader.
>> I think that you need to initialize XLogReaderState->readBuf directly in
>> ReadPageInternal() before reading a page and you should be good. With your
>> patch you get visibly only one portion of things addressed, what of other
>> code paths using xlogreader.c's APIs like pg_rewind, 2PC code and such?
>
> ReadPageInternal() doesn't know where the end of valid WAL is, so it
> cannot determine where to do memset(0). For example, in non-streaming
> cases, it reads the entire WAL block into readbuf, including the
> garbage.
Why couldn't it know about it? It would be perfectly fine to feed to it
the end LSN position as well and it is an internal API to xlogreader.c.
Note that both its callers, XLogFindNextRecord or XLogReadRecord know
about that as well.
> /*
> * If the current segment is being streamed from master, calculate how
> * much of the current page we have received already. We know the
> * requested record has been received, but this is for the benefit of
> * future calls, to allow quick exit at the top of this function.
> */
> if (readSource == XLOG_FROM_STREAM)
> {
> if (((targetPagePtr) / XLOG_BLCKSZ) != (receivedUpto / XLOG_BLCKSZ))
> readLen = XLOG_BLCKSZ;
> else
> readLen = receivedUpto % XLogSegSize - targetPageOff;
> }
> else
> readLen = XLOG_BLCKSZ;
>
> So we have to zero-fill the empty space of a WAL block before writing
> it. Currently, the master does that in AdvanceXLInsertBuffer(),
> StartupXLOG(), XLogFileCopy(). The cascading standby receives WAL
> from the master block by block, so it doesn't suffer from the garbage.
> pg_receivexlog zero-fills a new WAL file.
I cannot completely parse this statement. Physical replication sends up
to 16 WAL pages per message, rounded down to the last page completed.
Even a cascading standby sends WAL following this protocol, using the
last flush position as a base for the maximum amount of data sent.
>> + if (zbuffer == NULL)
>> + zbuffer = palloc0(XLOG_BLCKSZ);
>> You could just use a static buffer which is MemSet'd with 0 only if necessary.
>
> I wanted to allocate memory only when necessary. Currently, only the
> cascaded standby needs this. To that end, I moved the memory
> allocation code to a right place. Thanks for letting me notice this.
Still, it seems to me that any problems are not related to the fact that
we are using cascading standbys. As [1] says as well, similar problems
have been reported on a standby after restarting its primary.
I am definitely ready to buy that it can be possible to have garbage
being read the length field which can cause allocate_recordbuf to fail
as that's the only code path in xlogreader.c which does such an
allocation. Still, it seems to me that we should first try to see if
there are strange allocation patterns that happen and see if it is
possible to have a reproduceable test case or a pattern which gives us
confidence that we are on the right track. One idea I have to
monitor those allocations like the following:
--- a/src/backend/access/transam/xlogreader.c
+++ b/src/backend/access/transam/xlogreader.c
@@ -162,6 +162,10 @@ allocate_recordbuf(XLogReaderState *state, uint32 reclength)
newSize += XLOG_BLCKSZ - (newSize % XLOG_BLCKSZ);
newSize = Max(newSize, 5 * Max(BLCKSZ, XLOG_BLCKSZ));
+#ifndef FRONTEND
+ elog(LOG, "Allocation for xlogreader increased to %u", newSize);
+#endif
This could be upgraded to a PANIC or such if it sees a larger
allocation, as pgbench generates records with known lengths, that would
be a good fit to see if there is some garbage being read. No need to go
up to 1GB before seeing a failure.
[1]:
https://www.postgresql.org/message-id/CAE2gYzzVZNsGn%3D-E6grO4sVQs04J02zNKQofQEO8gu8%3DqCFR%2BQ%40ma...
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-02-23 02:26 ` Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Michael Paquier @ 2018-02-23 02:26 UTC (permalink / raw)
To: Tsunakawa, Takayuki <[email protected]>; +Cc: pgsql-hackers
On Thu, Feb 22, 2018 at 04:55:38PM +0900, Michael Paquier wrote:
> I am definitely ready to buy that it can be possible to have garbage
> being read the length field which can cause allocate_recordbuf to fail
> as that's the only code path in xlogreader.c which does such an
> allocation. Still, it seems to me that we should first try to see if
> there are strange allocation patterns that happen and see if it is
> possible to have a reproduceable test case or a pattern which gives us
> confidence that we are on the right track. One idea I have to
> monitor those allocations like the following:
> --- a/src/backend/access/transam/xlogreader.c
> +++ b/src/backend/access/transam/xlogreader.c
> @@ -162,6 +162,10 @@ allocate_recordbuf(XLogReaderState *state, uint32 reclength)
> newSize += XLOG_BLCKSZ - (newSize % XLOG_BLCKSZ);
> newSize = Max(newSize, 5 * Max(BLCKSZ, XLOG_BLCKSZ));
>
> +#ifndef FRONTEND
> + elog(LOG, "Allocation for xlogreader increased to %u", newSize);
> +#endif
So, I have been playing a bit more with that and defined the following
strategy to see if it is possible to create inconsistencies:
- Use a primary and a standby.
- Set up max_wal_size and min_wal_size to a minimum of 80MB so as the
segment recycling takes effect more quickly.
- Create a single table with a UUID column to increase the likelihood of
random data in INSERT records and FPWs, and insert enough data to
trigger a full WAL recycling.
- Every 5 seconds, insert a set of tuples into the table, using 110 to
120 tuples generates enough data for a bit more than a full WAL page.
And then restart the primary. This causes the standby to catch up with
normally a page streamed which is not completely initialized as it
fetches the page in the middle.
With the monitoring mentioned in the upper comment block, I have let the
whole thing run for a couple of hours, but I have not been able to catch
up problems, except the usual "invalid record length at 0/XXX: wanted
24, got 0". The allocation for recordbuf did not get higher than 40960
bytes as well, which matches with 5 WAL pages.
An other, evil, idea that I have on top of all those things is to
directly hexedit the WAL segment of the standby just at the limit where
it would receive a record from the primary and insert in it garbage
data which would make the validation tests to blow up in xlogreader.c
for the record allocation.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-02-23 14:02 ` Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Michael Paquier @ 2018-02-23 14:02 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Tsunakawa, Takayuki <[email protected]>
On Fri, Feb 23, 2018 at 11:26:31AM +0900, Michael Paquier wrote:
> An other, evil, idea that I have on top of all those things is to
> directly hexedit the WAL segment of the standby just at the limit where
> it would receive a record from the primary and insert in it garbage
> data which would make the validation tests to blow up in xlogreader.c
> for the record allocation.
OK, I have been playing with such methods and finally I have been able
to check the theory of Tsunakawa-san here. At first I played with
hexedit to pollute the data after the last record received by a standby,
record which is not at the end of a WAL page. This can easily happen by
stopping the primary which will most likely only send data up to the
middle of a page. Using that I have easily faced errors like that:
LOG: invalid resource manager ID 255 at 0/75353B8
And I have been able to see that garbage could be read as the length of
a record before the validation of the header is done. With the patch
attached you can easily see a collection of garbage:
LOG: length of 77913214 fetched for record 0/30009D8
And this happens if a standby is reading a page with the last record in
the middle of it.
At this state, the XLOG reader is dealing with random data, so this
would most likely fail in ValidXLogRecordHeader(), which is what
happened with the invalid rmgr for example. However, if one is really
unlucky, and the probability of facing that are really low, the random
data being read *can* pass the validation checks of the header, which
would cause a set of problems:
- In the backend, this means a failure on palloc_extended if the garbage
read is higher than 1GB. This causes a standby to stop immediately
while it should normally continue to poll the primary by spawning a new
WAL receiver and begin streaming from the beginning of the segment where
it needs its data (assuming that there is no archive).
- In the frontend, this can cause problems for pg_waldump and
pg_rewind. For pg_waldump, this is not actually a big deal because a
failure means the end of the data that can be decoded. However the
problem is more serious for pg_rewind. At the initial phase of the
tool, pg_rewind scans the WAL segments of the target server to look for
the modified blocks since the last checkpoint before promotion up to the
end of the stream. If at the end of the stream it finds garbage data,
then there is a risk that it allocates a couple of GBs of data, likely
finishing by causing the process to be killed on OOM by the automatic
OOM killer, preventing the rewind to complete :(
After some thoughts, I have also come up with a more simple way to test
the stabilility of the XLOG reader:
1) Create a primary/standby cluster.
2) Stop the standby.
3) Add in the standby's pg_wal a set of junk WAL segments generated with
if=/dev/urandom of=junk_walseg bs=16MB count=1. Note that recycled WAL
segments are simple copies of past ones when fetching an existing one,
so on a standby when a new segment writes based on a past existing
segment it writes data on top of some garbage. So it is possible to
face this problem, this just makes it show up faster.
4) Start the standby again, and let it alive.
5) Generate on the primary enough records worth of more or less 8kB to
fill in a complete WAL page.
6) Restart the primary cleanly, sleep like 5s to let the standby the
time to stream the new partial page and return to 5).
When 5) and 6) loop, you will see the monitoring log of the attached
patch complain after a couple of restarts.
Tsunakawa-san has proposed upthread to fix the problem by zero-ing the
page read in the WAL receiver. While I agree that zeroing the page is
the way to go, doing so in the WAL receiver does not take care of
problems with the frontends. I don't have the time to test that yet as
it is late here, but based on my code lookups tweaking
ReadPageInternal() so as the page is zero'ed correctly should do it for
all the cases. This way, the checks happening after a page read would
fail because of the zero'ed fields consistently instead of the garbage
accumulated.
--
Michael
Attachments:
[text/x-diff] monitor-xlogreader-garbage.patch (875B, ../../[email protected]/2-monitor-xlogreader-garbage.patch)
download | inline diff:
diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c
index 3a86f3497e..13b25e5236 100644
--- a/src/backend/access/transam/xlogreader.c
+++ b/src/backend/access/transam/xlogreader.c
@@ -303,6 +303,16 @@ XLogReadRecord(XLogReaderState *state, XLogRecPtr RecPtr, char **errormsg)
record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ);
total_len = record->xl_tot_len;
+#ifndef FRONTEND
+ /*
+ * XXX: remove from final patch , just here to check that garbage
+ * data can be fetched from blocks read.
+ */
+ if (total_len > 5 * XLOG_BLCKSZ)
+ elog(LOG, "length of %u fetched for record %X/%X", total_len,
+ (uint32) (RecPtr >> 32), (uint32) RecPtr);
+#endif
+
/*
* If the whole record header is on this page, validate it immediately.
* Otherwise do just a basic sanity check on xl_tot_len, and validate the
[application/pgp-signature] signature.asc (833B, ../../[email protected]/3-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-02-26 02:57 ` Michael Paquier <[email protected]>
2018-02-26 07:25 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Michael Paquier @ 2018-02-26 02:57 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Tsunakawa, Takayuki <[email protected]>
On Fri, Feb 23, 2018 at 11:02:19PM +0900, Michael Paquier wrote:
> Tsunakawa-san has proposed upthread to fix the problem by zero-ing the
> page read in the WAL receiver. While I agree that zeroing the page is
> the way to go, doing so in the WAL receiver does not take care of
> problems with the frontends. I don't have the time to test that yet as
> it is late here, but based on my code lookups tweaking
> ReadPageInternal() so as the page is zero'ed correctly should do it for
> all the cases. This way, the checks happening after a page read would
> fail because of the zero'ed fields consistently instead of the garbage
> accumulated.
I have been playing more with that this morning, and trying to tweak the
XLOG reader so as the fetched page is zeroed where necessary does not
help much. XLogReaderState->EndRecPtr is updated once the last record
is set so it is possible to use it and zero the rest of the page which
would prevent a lookup. But this is actually not wise for performance:
- The data after EndRecPtr could be perfectly valid, so you could zero
data which could be reused later.
- It is necessary to drop the quick-exit checks in PageReadInternal.
The WAL receiver approach also has a drawback. If WAL is streamed at
full speed, then the primary sends data with a maximum of 6 WAL pages.
When beginning streaming with a new segment, then the WAL sent stops at
page boundary. But if you stop once in the middle of a page then you
need to zero-fill the page until the current segment is finished
streaming. So if the workload generates spiky WAL then the WAL receiver
can would a lot of extra lseek() calls with the patch applied, while all
the writes would be sequential on HEAD, so that's not performant-wise
IMO.
Another idea I am thinking about would be to zero-fill the segments
when recycled instead of being just renamed when doing
InstallXLogFileSegment(). This would also have the advantage of making
the segments ahead more compressible, which is a gain for custom
backups, and the WAL receiver does not need any tweaks as it would write
the data on a clean file. Zero-filling the segments is done only when a
new segment is created (see XLogFileInit).
Each approach has its own cost. Thoughts?
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* RE: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-02-26 07:25 ` Tsunakawa, Takayuki <[email protected]>
2018-02-26 08:08 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Tsunakawa, Takayuki @ 2018-02-26 07:25 UTC (permalink / raw)
To: 'Michael Paquier' <[email protected]>; pgsql-hackers
From: Michael Paquier [mailto:[email protected]]
> I have been playing more with that this morning, and trying to tweak the
> XLOG reader so as the fetched page is zeroed where necessary does not help
> much. XLogReaderState->EndRecPtr is updated once the last record is set
> so it is possible to use it and zero the rest of the page which would prevent
> a lookup. But this is actually not wise for performance:
> - The data after EndRecPtr could be perfectly valid, so you could zero data
> which could be reused later.
> - It is necessary to drop the quick-exit checks in PageReadInternal.
First of all, thanks for your many experiments and ideas.
Yes, the above method doesn't work. The reason is not only performance but also correctness. After you zero-fill the remaining portion of state->readbuf based on state->EndRecPtr, you need to read the same page upon the request of the next WAL record. And that page read brings garbage again into state->readbuf. After all, the callers of ReadPageInternal() has to face the garbage.
> The WAL receiver approach also has a drawback. If WAL is streamed at full
> speed, then the primary sends data with a maximum of 6 WAL pages.
> When beginning streaming with a new segment, then the WAL sent stops at
> page boundary. But if you stop once in the middle of a page then you need
> to zero-fill the page until the current segment is finished streaming. So
> if the workload generates spiky WAL then the WAL receiver can would a lot
> of extra lseek() calls with the patch applied, while all the writes would
> be sequential on HEAD, so that's not performant-wise IMO.
Does even the non-cascading standby stop in the middle of a page? I thought the master always the whole WAL blocks without stopping in the middle of a page.
> Another idea I am thinking about would be to zero-fill the segments when
> recycled instead of being just renamed when doing InstallXLogFileSegment().
> This would also have the advantage of making the segments ahead more
> compressible, which is a gain for custom backups, and the WAL receiver does
> not need any tweaks as it would write the data on a clean file. Zero-filling
> the segments is done only when a new segment is created (see XLogFileInit).
Yes, I was (and am) inclined to take this approach; this is easy and clean, but not good for performance... I hope there's something which justifies this approach.
Regards
Takayuki Tsunakawa
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 07:25 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
@ 2018-02-26 08:08 ` Michael Paquier <[email protected]>
2018-02-26 09:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Michael Paquier @ 2018-02-26 08:08 UTC (permalink / raw)
To: Tsunakawa, Takayuki <[email protected]>; +Cc: pgsql-hackers
On Mon, Feb 26, 2018 at 07:25:46AM +0000, Tsunakawa, Takayuki wrote:
> From: Michael Paquier [mailto:[email protected]]
>> The WAL receiver approach also has a drawback. If WAL is streamed at full
>> speed, then the primary sends data with a maximum of 6 WAL pages.
>> When beginning streaming with a new segment, then the WAL sent stops at
>> page boundary. But if you stop once in the middle of a page then you need
>> to zero-fill the page until the current segment is finished streaming. So
>> if the workload generates spiky WAL then the WAL receiver can would a lot
>> of extra lseek() calls with the patch applied, while all the writes would
>> be sequential on HEAD, so that's not performant-wise IMO.
>
> Does even the non-cascading standby stop in the middle of a page? I
> thought the master always the whole WAL blocks without stopping in the
> middle of a page.
You even have problems on normal standbys. I have a small script which
is able to reproduce that if you want (need a small rewrite as it is
adapted to my test framework) which introduces a garbage set of WAL
segments on a stopped standby. With the small monitoring patch I
mentioned upthread then you can see the XLOG reader finding garbage data
as well before validating the record header. With any fixes on the WAL
receiver, your first patch included, then the garbage read goes away,
and XLOG reader complains about a record with an incorrect length
(invalid record length at XX/YYY: wanted 24, got 0) instead of complains
from header validation part. One key point is to cleanly stop the
primary to as it forces the standby's WAL receiver to write to its WAL
segment in the middle of a page.
>> Another idea I am thinking about would be to zero-fill the segments when
>> recycled instead of being just renamed when doing InstallXLogFileSegment().
>> This would also have the advantage of making the segments ahead more
>> compressible, which is a gain for custom backups, and the WAL receiver does
>> not need any tweaks as it would write the data on a clean file. Zero-filling
>> the segments is done only when a new segment is created (see XLogFileInit).
>
> Yes, I was (and am) inclined to take this approach; this is easy and
> clean, but not good for performance... I hope there's something which
> justifies this approach.
InstallXLogFileSegment uses a plain durable_link_or_rename() to recycle
the past segment which syncs the old segment before the rename anyway,
so the I/O effort will be there, no?
This was mentioned back in 2001 by the way, but this did not count much
for the case discussed here:
https://www.postgresql.org/message-id/24901.995381770%40sss.pgh.pa.us
The issue here is that the streaming case makes it easier to hit the
problem as it opens more easily access to not-completely written WAL
pages depending on the message frequency during replication. At the
same time, we are discussing about a very low-probability issue. Note
that if the XLOG reader is bumping into this problem, then at the next
WAL receiver wake up, recovery would begin from the beginning of the
last segment, and if the primary has produced some more WAL then the
standby would be able to actually avoid the random junk. It is also
possible to bypass the problem by zeroing manually the areas in
question, or to actually wait for the standby to generate more WAL so as
the garbage is overwritten automatically. And you really need to be
very, very unlucky to have random garbage able to bypass the header
validation checks.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 07:25 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-26 08:08 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-02-26 09:19 ` Michael Paquier <[email protected]>
2018-02-27 05:15 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Michael Paquier @ 2018-02-26 09:19 UTC (permalink / raw)
To: Tsunakawa, Takayuki <[email protected]>; +Cc: pgsql-hackers
On Mon, Feb 26, 2018 at 05:08:49PM +0900, Michael Paquier wrote:
> This was mentioned back in 2001 by the way, but this did not count much
> for the case discussed here:
> https://www.postgresql.org/message-id/24901.995381770%40sss.pgh.pa.us
> The issue here is that the streaming case makes it easier to hit the
> problem as it opens more easily access to not-completely written WAL
> pages depending on the message frequency during replication. At the
> same time, we are discussing about a very low-probability issue. Note
> that if the XLOG reader is bumping into this problem, then at the next
> WAL receiver wake up, recovery would begin from the beginning of the
> last segment, and if the primary has produced some more WAL then the
> standby would be able to actually avoid the random junk. It is also
> possible to bypass the problem by zeroing manually the areas in
> question, or to actually wait for the standby to generate more WAL so as
> the garbage is overwritten automatically. And you really need to be
> very, very unlucky to have random garbage able to bypass the header
> validation checks.
By the way, as long as I have my mind of it. Another strategy would be
to just make the checks in XLogReadRecord() a bit smarter if the whole
record header is not on the page. If we check at least for
AllocSizeIsValid(total_len) then there this code would not fail on an
allocation as you user reported. Still this misses the case where a
record size is lower than 1GB but invalid so you would allocate
allocate_recordbuf for nothing :(
At least this extra check is costless, and avoids all kind of hard
failures.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* RE: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 07:25 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-26 08:08 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 09:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-02-27 05:15 ` Tsunakawa, Takayuki <[email protected]>
2018-03-14 05:27 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Tsunakawa, Takayuki @ 2018-02-27 05:15 UTC (permalink / raw)
To: 'Michael Paquier' <[email protected]>; +Cc: pgsql-hackers
From: Michael Paquier [mailto:[email protected]]
> By the way, as long as I have my mind of it. Another strategy would be
> to just make the checks in XLogReadRecord() a bit smarter if the whole record
> header is not on the page. If we check at least for
> AllocSizeIsValid(total_len) then there this code would not fail on an
> allocation as you user reported. Still this misses the case where a record
> size is lower than 1GB but invalid so you would allocate allocate_recordbuf
> for nothing :(
That was my first thought, and I gave it up. As you say, XLogReadRecord() could allocate up to 1 GB of memory for a garbage. That allocation can fail due to memory shortage, which prevents the recovery from proceeding.
Regards
Takayuki Tsunakawa
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 07:25 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-26 08:08 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 09:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-27 05:15 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
@ 2018-03-14 05:27 ` Michael Paquier <[email protected]>
2018-03-16 05:27 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Michael Paquier @ 2018-03-14 05:27 UTC (permalink / raw)
To: Tsunakawa, Takayuki <[email protected]>; +Cc: pgsql-hackers
On Tue, Feb 27, 2018 at 05:15:29AM +0000, Tsunakawa, Takayuki wrote:
> That was my first thought, and I gave it up. As you say,
> XLogReadRecord() could allocate up to 1 GB of memory for a garbage.
> That allocation can fail due to memory shortage, which prevents the
> recovery from proceeding.
Even with that, the resulting patch is sort of ugly... So it seems to
me that the conclusion to this thread is that there is no clear winner,
and that the problem is so unlikely to happen that it is not worth the
performance impact to zero all the WAL pages fetched. Still, the
attached has the advantage to not cause the startup process to fail
suddendly because of the allocation failure but to fail afterwards when
validating the record's contents, which has the disadvantage to allocate
temporarily up to 1GB of memory for some of the garbage, but that
allocation is short-lived. So that would switch the failure from a
FATAL allocation failure taking down Postgres to something which will
fetching of new WAL records to be tried again.
All in all, that would be a win for the case reported on this thread.
Thoughts from anybody?
--
Michael
Attachments:
[text/x-diff] xlogreader-record-len-check.patch (1.8K, ../../[email protected]/2-xlogreader-record-len-check.patch)
download | inline diff:
diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c
index 3a86f3497e..5bf50a2656 100644
--- a/src/backend/access/transam/xlogreader.c
+++ b/src/backend/access/transam/xlogreader.c
@@ -25,6 +25,10 @@
#include "common/pg_lzcompress.h"
#include "replication/origin.h"
+#ifndef FRONTEND
+#include "utils/memutils.h"
+#endif
+
static bool allocate_recordbuf(XLogReaderState *state, uint32 reclength);
static bool ValidXLogPageHeader(XLogReaderState *state, XLogRecPtr recptr,
@@ -309,7 +313,14 @@ XLogReadRecord(XLogReaderState *state, XLogRecPtr RecPtr, char **errormsg)
* rest of the header after reading it from the next page. The xl_tot_len
* check is necessary here to ensure that we enter the "Need to reassemble
* record" code path below; otherwise we might fail to apply
- * ValidXLogRecordHeader at all.
+ * ValidXLogRecordHeader at all. Note that in much unlucky circumstances,
+ * the random data read from a recycled segment could allow the set of
+ * sanity checks to pass. If the garbage data read in xl_tot_len is
+ * higher than MaxAllocSize on the backend, then the startup process
+ * would retry to fetch fresh WAL data. If this is lower than
+ * MaxAllocSize, then a more-than-necessary memory allocation will
+ * happen for a short amount of time, until the WAL reader fails at
+ * validating the next record's data.
*/
if (targetRecOff <= XLOG_BLCKSZ - SizeOfXLogRecord)
{
@@ -321,7 +332,11 @@ XLogReadRecord(XLogReaderState *state, XLogRecPtr RecPtr, char **errormsg)
else
{
/* XXX: more validation should be done here */
- if (total_len < SizeOfXLogRecord)
+ if (total_len < SizeOfXLogRecord
+#ifndef FRONTEND
+ || !AllocSizeIsValid(total_len)
+#endif
+ )
{
report_invalid_record(state,
"invalid record length at %X/%X: wanted %u, got %u",
[application/pgp-signature] signature.asc (833B, ../../[email protected]/3-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* RE: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 07:25 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-26 08:08 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 09:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-27 05:15 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-03-14 05:27 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-03-16 05:27 ` Tsunakawa, Takayuki <[email protected]>
2018-03-16 05:47 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Tsunakawa, Takayuki @ 2018-03-16 05:27 UTC (permalink / raw)
To: 'Michael Paquier' <[email protected]>; +Cc: pgsql-hackers
From: Michael Paquier [mailto:[email protected]]
> Even with that, the resulting patch is sort of ugly... So it seems to me
> that the conclusion to this thread is that there is no clear winner, and
> that the problem is so unlikely to happen that it is not worth the performance
> impact to zero all the WAL pages fetched. Still, the attached has the
> advantage to not cause the startup process to fail suddendly because of
> the allocation failure but to fail afterwards when validating the record's
> contents, which has the disadvantage to allocate temporarily up to 1GB of
> memory for some of the garbage, but that allocation is short-lived. So
> that would switch the failure from a FATAL allocation failure taking down
> Postgres to something which will fetching of new WAL records to be tried
> again.
>
> All in all, that would be a win for the case reported on this thread.
I'm sorry to be late to reply, and thanks for another patch.
Honestly, I'm fine with either patch. I like your simpler and cleaner one that has no performance impact. But please note that the allocation attempt could amount to nearly 1 GB. That can fail due to memory shortage, which leads to FATAL error (ereport(ERROR) results in FATAL in startup process), and cause standby to shut down.
Regards
Takayuki Tsunakawa
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 07:25 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-26 08:08 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 09:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-27 05:15 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-03-14 05:27 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-03-16 05:27 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
@ 2018-03-16 05:47 ` Michael Paquier <[email protected]>
2018-03-16 06:02 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Michael Paquier @ 2018-03-16 05:47 UTC (permalink / raw)
To: Tsunakawa, Takayuki <[email protected]>; +Cc: pgsql-hackers
On Fri, Mar 16, 2018 at 05:27:58AM +0000, Tsunakawa, Takayuki wrote:
> Honestly, I'm fine with either patch. I like your simpler and cleaner
> one that has no performance impact. But please note that the
> allocation attempt could amount to nearly 1 GB. That can fail due to
> memory shortage, which leads to FATAL error (ereport(ERROR) results in
> FATAL in startup process), and cause standby to shut down.
We use palloc_extended with MCXT_ALLOC_NO_OOM in 9.5~, and malloc()
further down, so once you remove the FATAL error caused by a record
whose length is higher than 1GB, you discard all the hard failures, no?
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* RE: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 07:25 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-26 08:08 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 09:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-27 05:15 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-03-14 05:27 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-03-16 05:27 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-03-16 05:47 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
@ 2018-03-16 06:02 ` Tsunakawa, Takayuki <[email protected]>
2018-03-17 23:49 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Tsunakawa, Takayuki @ 2018-03-16 06:02 UTC (permalink / raw)
To: 'Michael Paquier' <[email protected]>; +Cc: pgsql-hackers
From: Michael Paquier [mailto:[email protected]]
> We use palloc_extended with MCXT_ALLOC_NO_OOM in 9.5~, and malloc() further
> down, so once you remove the FATAL error caused by a record whose length
> is higher than 1GB, you discard all the hard failures, no?
Ouch, you're right. If memory allocation fails, the startup process would emit a LOG message and continue to fetch new WAL records. Then, I'm completely happy with your patch.
I added you as the author, and marked this as ready for committer.
Regards
Takayuki Tsunakawa
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: [bug fix] Cascaded standby cannot start after a clean shutdown
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-19 03:01 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 02:26 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-23 14:02 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 02:57 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 07:25 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-26 08:08 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-26 09:19 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-02-27 05:15 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-03-14 05:27 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-03-16 05:27 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-03-16 05:47 ` Re: [bug fix] Cascaded standby cannot start after a clean shutdown Michael Paquier <[email protected]>
2018-03-16 06:02 ` RE: [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
@ 2018-03-17 23:49 ` Michael Paquier <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Michael Paquier @ 2018-03-17 23:49 UTC (permalink / raw)
To: Tsunakawa, Takayuki <[email protected]>; +Cc: pgsql-hackers
On Fri, Mar 16, 2018 at 06:02:25AM +0000, Tsunakawa, Takayuki wrote:
> Ouch, you're right. If memory allocation fails, the startup process
> would emit a LOG message and continue to fetch new WAL records. Then,
> I'm completely happy with your patch.
Thanks for double-checking, Tsunakawa-san.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v2 2/2] Propagate replica identity to partitions
@ 2019-02-04 16:43 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Alvaro Herrera @ 2019-02-04 16:43 UTC (permalink / raw)
---
src/backend/commands/tablecmds.c | 108 +++++++++++++++--
src/bin/psql/describe.c | 3 +-
src/test/regress/expected/replica_identity.out | 160 +++++++++++++++++++++++++
src/test/regress/sql/replica_identity.sql | 43 +++++++
4 files changed, 304 insertions(+), 10 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 877bac506f..22cec85ab0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -300,7 +300,7 @@ static void truncate_check_activity(Relation rel);
static void RangeVarCallbackForTruncate(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
static List *MergeAttributes(List *schema, List *supers, char relpersistence,
- bool is_partition, List **supconstr);
+ bool is_partition, List **supconstr, char *ri_type);
static bool MergeCheckConstraint(List *constraints, char *name, Node *expr);
static void MergeAttributesIntoExisting(Relation child_rel, Relation parent_rel);
static void MergeConstraintsIntoExisting(Relation child_rel, Relation parent_rel);
@@ -485,6 +485,7 @@ static void AttachPartitionEnsureIndexes(Relation rel, Relation attachrel);
static void QueuePartitionConstraintValidation(List **wqueue, Relation scanrel,
List *partConstraint,
bool validate_default);
+static void MatchReplicaIdentity(Relation tgtrel, Relation srcrel);
static void CloneRowTriggersToPartition(Relation parent, Relation partition);
static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name);
static ObjectAddress ATExecAttachPartitionIdx(List **wqueue, Relation rel,
@@ -527,6 +528,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
TupleDesc descriptor;
List *inheritOids;
List *old_constraints;
+ char ri_type;
List *rawDefaults;
List *cookedDefaults;
Datum reloptions;
@@ -708,7 +710,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
MergeAttributes(stmt->tableElts, inheritOids,
stmt->relation->relpersistence,
stmt->partbound != NULL,
- &old_constraints);
+ &old_constraints, &ri_type);
/*
* Create a tuple descriptor from the relation schema. Note that this
@@ -1014,6 +1016,10 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
CloneForeignKeyConstraints(parentId, relationId, NULL);
+ /* Propagate REPLICA IDENTITY information too */
+ if (ri_type != REPLICA_IDENTITY_DEFAULT)
+ MatchReplicaIdentity(rel, parent);
+
table_close(parent, NoLock);
}
@@ -1873,6 +1879,8 @@ storage_name(char c)
* Output arguments:
* 'supconstr' receives a list of constraints belonging to the parents,
* updated as necessary to be valid for the child.
+ * 'ri_type' receives the replica identity type of the last parent seen,
+ * or default if none.
*
* Return value:
* Completed schema list.
@@ -1914,11 +1922,15 @@ storage_name(char c)
* (4) Otherwise the inherited default is used.
* Rule (3) is new in Postgres 7.1; in earlier releases you got a
* rather arbitrary choice of which parent default to use.
+ *
+ * It only makes sense to use the returned 'ri_type' when there's a single
+ * parent, such as in declarative partitioning.
*----------
*/
static List *
MergeAttributes(List *schema, List *supers, char relpersistence,
- bool is_partition, List **supconstr)
+ bool is_partition, List **supconstr,
+ char *ri_type)
{
ListCell *entry;
List *inhSchema = NIL;
@@ -2015,6 +2027,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
}
/*
+ * Initialize replica identity to default; parents may change it later
+ */
+ *ri_type = REPLICA_IDENTITY_DEFAULT;
+
+ /*
* Scan the parents left-to-right, and merge their attributes to form a
* list of inherited attributes (inhSchema). Also check to see if we need
* to inherit an OID column.
@@ -2095,6 +2112,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
? "cannot inherit from temporary relation of another session"
: "cannot create as partition of temporary relation of another session")));
+ /* Indicate replica identity back to caller */
+ *ri_type = relation->rd_rel->relreplident;
+
/*
* We should have an UNDER permission flag for this, but for now,
* demand that creator of a child table own the parent.
@@ -3935,7 +3955,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_ReplicaIdentity: /* REPLICA IDENTITY ... */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
pass = AT_PASS_MISC;
- /* This command never recurses */
+ /* Recursion occurs during execution phase */
/* No command-specific prep needed */
break;
case AT_EnableTrig: /* ENABLE TRIGGER variants */
@@ -12756,7 +12776,7 @@ ATExecDropOf(Relation rel, LOCKMODE lockmode)
*/
static void
relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
- bool is_internal)
+ bool is_internal, LOCKMODE lockmode)
{
Relation pg_index;
Relation pg_class;
@@ -12847,6 +12867,42 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
}
table_close(pg_index, RowExclusiveLock);
+
+ /*
+ * If there are any partitions, handle them too.
+ */
+ if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ PartitionDesc pd = RelationGetPartitionDesc(rel);
+
+ for (int i = 0; i < pd->nparts; i++)
+ {
+ Relation part = table_open(pd->oids[i], lockmode);
+ Oid idxOid;
+
+ if (ri_type == REPLICA_IDENTITY_INDEX)
+ {
+ idxOid = index_get_partition(part, indexOid);
+ if (!OidIsValid(idxOid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("replica index does not exist in partition \"%s\"",
+ RelationGetRelationName(part))));
+
+ LockRelationOid(idxOid, ShareLock);
+ }
+ else
+ idxOid = InvalidOid;
+
+ idxOid = ri_type == REPLICA_IDENTITY_INDEX ?
+ index_get_partition(part, indexOid) : InvalidOid;
+
+ relation_mark_replica_identity(part, ri_type, idxOid, true,
+ lockmode);
+
+ table_close(part, NoLock);
+ }
+ }
}
/*
@@ -12861,17 +12917,20 @@ ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKMODE lockmode
if (stmt->identity_type == REPLICA_IDENTITY_DEFAULT)
{
- relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true);
+ relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true,
+ lockmode);
return;
}
else if (stmt->identity_type == REPLICA_IDENTITY_FULL)
{
- relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true);
+ relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true,
+ lockmode);
return;
}
else if (stmt->identity_type == REPLICA_IDENTITY_NOTHING)
{
- relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true);
+ relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true,
+ lockmode);
return;
}
else if (stmt->identity_type == REPLICA_IDENTITY_INDEX)
@@ -12959,7 +13018,8 @@ ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKMODE lockmode
}
/* This index is suitable for use as a replica identity. Mark it. */
- relation_mark_replica_identity(rel, stmt->identity_type, indexOid, true);
+ relation_mark_replica_identity(rel, stmt->identity_type, indexOid, true,
+ lockmode);
index_close(indexRel, NoLock);
}
@@ -14664,6 +14724,10 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
/* and triggers */
CloneRowTriggersToPartition(rel, attachrel);
+ /* Propagate REPLICA IDENTITY information */
+ if (rel->rd_rel->relreplident != REPLICA_IDENTITY_DEFAULT)
+ MatchReplicaIdentity(attachrel, rel);
+
/*
* Clone foreign key constraints, and setup for Phase 3 to verify them.
*/
@@ -14915,6 +14979,32 @@ AttachPartitionEnsureIndexes(Relation rel, Relation attachrel)
}
/*
+ * Set up partRel's (a partition) replica identity to match parentRel's (its
+ * parent).
+ */
+static void
+MatchReplicaIdentity(Relation partRel, Relation srcrel)
+{
+ Oid ri_index;
+
+ if (srcrel->rd_rel->relreplident == REPLICA_IDENTITY_INDEX)
+ {
+ ri_index = index_get_partition(partRel,
+ RelationGetReplicaIndex(srcrel));
+ if (!OidIsValid(ri_index))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("replica index does not exist in partition \"%s\"",
+ RelationGetRelationName(partRel))));
+ }
+ else
+ ri_index = InvalidOid;
+
+ relation_mark_replica_identity(partRel, srcrel->rd_rel->relreplident,
+ ri_index, true, AccessExclusiveLock);
+}
+
+/*
* CloneRowTriggersToPartition
* subroutine for ATExecAttachPartition/DefineRelation to create row
* triggers on partitions
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 4da6719ce7..6145a000cb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -3113,7 +3113,8 @@ describeOneTableDetails(const char *schemaname,
if (verbose &&
(tableinfo.relkind == RELKIND_RELATION ||
- tableinfo.relkind == RELKIND_MATVIEW) &&
+ tableinfo.relkind == RELKIND_MATVIEW ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE) &&
/*
* No need to display default values; we already display a REPLICA
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 175ecd2879..d6014df840 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -181,3 +181,163 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity_othertable;
+----
+-- Make sure it propagates to partitions
+----
+CREATE TABLE test_replica_identity_part (a int, b int) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part1 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (0) TO (1000) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part2 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (1000) TO (2000);
+CREATE TABLE test_replica_identity_part11 PARTITION OF test_replica_identity_part1
+ FOR VALUES FROM (1000) TO (1500);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_part3 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (2000) TO (3000);
+CREATE TABLE test_replica_identity_part4 (LIKE test_replica_identity_part);
+ALTER TABLE test_replica_identity_part ATTACH PARTITION test_replica_identity_part4
+ FOR VALUES FROM (3000) TO (4000);
+\d+ test_replica_identity_part2
+ Table "public.test_replica_identity_part2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (1000) TO (2000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 1000) AND (a < 2000))
+Replica Identity: FULL
+
+\d+ test_replica_identity_part11
+ Table "public.test_replica_identity_part11"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part1 FOR VALUES FROM (1000) TO (1500)
+Partition constraint: ((a IS NOT NULL) AND (a >= 0) AND (a < 1000) AND (a IS NOT NULL) AND (a >= 1000) AND (a < 1500))
+Replica Identity: FULL
+
+\d+ test_replica_identity_part
+ Partitioned table "public.test_replica_identity_part"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition key: RANGE (a)
+Partitions: test_replica_identity_part1 FOR VALUES FROM (0) TO (1000), PARTITIONED,
+ test_replica_identity_part2 FOR VALUES FROM (1000) TO (2000),
+ test_replica_identity_part3 FOR VALUES FROM (2000) TO (3000),
+ test_replica_identity_part4 FOR VALUES FROM (3000) TO (4000)
+Replica Identity: FULL
+
+\d+ test_replica_identity_part3
+ Table "public.test_replica_identity_part3"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (2000) TO (3000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 2000) AND (a < 3000))
+Replica Identity: FULL
+
+\d+ test_replica_identity_part4
+ Table "public.test_replica_identity_part4"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (3000) TO (4000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 3000) AND (a < 4000))
+Replica Identity: FULL
+
+ALTER TABLE test_replica_identity_part ALTER a SET NOT NULL;
+CREATE UNIQUE INDEX trip_b_idx ON test_replica_identity_part (a);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY USING INDEX trip_b_idx;
+\d+ test_replica_identity_part2
+ Table "public.test_replica_identity_part2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (1000) TO (2000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 1000) AND (a < 2000))
+Indexes:
+ "test_replica_identity_part2_a_idx" UNIQUE, btree (a) REPLICA IDENTITY
+
+\d+ test_replica_identity_part11
+ Table "public.test_replica_identity_part11"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part1 FOR VALUES FROM (1000) TO (1500)
+Partition constraint: ((a IS NOT NULL) AND (a >= 0) AND (a < 1000) AND (a IS NOT NULL) AND (a >= 1000) AND (a < 1500))
+Indexes:
+ "test_replica_identity_part11_a_idx" UNIQUE, btree (a) REPLICA IDENTITY
+
+\d+ test_replica_identity_part
+ Partitioned table "public.test_replica_identity_part"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition key: RANGE (a)
+Indexes:
+ "trip_b_idx" UNIQUE, btree (a) REPLICA IDENTITY
+Partitions: test_replica_identity_part1 FOR VALUES FROM (0) TO (1000), PARTITIONED,
+ test_replica_identity_part2 FOR VALUES FROM (1000) TO (2000),
+ test_replica_identity_part3 FOR VALUES FROM (2000) TO (3000),
+ test_replica_identity_part4 FOR VALUES FROM (3000) TO (4000)
+
+\d+ test_replica_identity_part3
+ Table "public.test_replica_identity_part3"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (2000) TO (3000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 2000) AND (a < 3000))
+Indexes:
+ "test_replica_identity_part3_a_idx" UNIQUE, btree (a) REPLICA IDENTITY
+
+\d+ test_replica_identity_part4
+ Table "public.test_replica_identity_part4"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (3000) TO (4000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 3000) AND (a < 4000))
+Indexes:
+ "test_replica_identity_part4_a_idx" UNIQUE, btree (a) REPLICA IDENTITY
+
+----
+-- Check behavior with inherited tables
+----
+CREATE TABLE test_replica_identity_inh (a int);
+CREATE TABLE test_replica_identity_cld () INHERITS (test_replica_identity_inh);
+ALTER TABLE test_replica_identity_inh REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_cld2 () INHERITS (test_replica_identity_inh);
+\d+ test_replica_identity_inh
+ Table "public.test_replica_identity_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+Child tables: test_replica_identity_cld,
+ test_replica_identity_cld2
+Replica Identity: FULL
+
+\d+ test_replica_identity_cld
+ Table "public.test_replica_identity_cld"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+Inherits: test_replica_identity_inh
+
+\d+ test_replica_identity_cld2
+ Table "public.test_replica_identity_cld2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+Inherits: test_replica_identity_inh
+
diff --git a/src/test/regress/sql/replica_identity.sql b/src/test/regress/sql/replica_identity.sql
index b08a3623b8..9e309796f2 100644
--- a/src/test/regress/sql/replica_identity.sql
+++ b/src/test/regress/sql/replica_identity.sql
@@ -77,3 +77,46 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity_othertable;
+
+----
+-- Make sure it propagates to partitions
+----
+CREATE TABLE test_replica_identity_part (a int, b int) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part1 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (0) TO (1000) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part2 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (1000) TO (2000);
+CREATE TABLE test_replica_identity_part11 PARTITION OF test_replica_identity_part1
+ FOR VALUES FROM (1000) TO (1500);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_part3 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (2000) TO (3000);
+CREATE TABLE test_replica_identity_part4 (LIKE test_replica_identity_part);
+ALTER TABLE test_replica_identity_part ATTACH PARTITION test_replica_identity_part4
+ FOR VALUES FROM (3000) TO (4000);
+\d+ test_replica_identity_part2
+\d+ test_replica_identity_part11
+\d+ test_replica_identity_part
+\d+ test_replica_identity_part3
+\d+ test_replica_identity_part4
+
+ALTER TABLE test_replica_identity_part ALTER a SET NOT NULL;
+CREATE UNIQUE INDEX trip_b_idx ON test_replica_identity_part (a);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY USING INDEX trip_b_idx;
+\d+ test_replica_identity_part2
+\d+ test_replica_identity_part11
+\d+ test_replica_identity_part
+\d+ test_replica_identity_part3
+\d+ test_replica_identity_part4
+
+
+----
+-- Check behavior with inherited tables
+----
+CREATE TABLE test_replica_identity_inh (a int);
+CREATE TABLE test_replica_identity_cld () INHERITS (test_replica_identity_inh);
+ALTER TABLE test_replica_identity_inh REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_cld2 () INHERITS (test_replica_identity_inh);
+\d+ test_replica_identity_inh
+\d+ test_replica_identity_cld
+\d+ test_replica_identity_cld2
--
2.11.0
--v53z3yvcsi6z6wpg--
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH 2/2] Propagate replica identity to partitions
@ 2019-02-04 16:43 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Alvaro Herrera @ 2019-02-04 16:43 UTC (permalink / raw)
---
src/backend/commands/tablecmds.c | 59 +++++++++++++--
src/bin/psql/describe.c | 3 +-
src/test/regress/expected/replica_identity.out | 99 ++++++++++++++++++++++++++
src/test/regress/sql/replica_identity.sql | 33 +++++++++
4 files changed, 189 insertions(+), 5 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 877bac506f..4b2ae01f15 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -300,7 +300,7 @@ static void truncate_check_activity(Relation rel);
static void RangeVarCallbackForTruncate(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
static List *MergeAttributes(List *schema, List *supers, char relpersistence,
- bool is_partition, List **supconstr);
+ bool is_partition, List **supconstr, char *ri_type);
static bool MergeCheckConstraint(List *constraints, char *name, Node *expr);
static void MergeAttributesIntoExisting(Relation child_rel, Relation parent_rel);
static void MergeConstraintsIntoExisting(Relation child_rel, Relation parent_rel);
@@ -485,6 +485,7 @@ static void AttachPartitionEnsureIndexes(Relation rel, Relation attachrel);
static void QueuePartitionConstraintValidation(List **wqueue, Relation scanrel,
List *partConstraint,
bool validate_default);
+static void MatchReplicaIdentity(Relation tgtrel, Relation srcrel);
static void CloneRowTriggersToPartition(Relation parent, Relation partition);
static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name);
static ObjectAddress ATExecAttachPartitionIdx(List **wqueue, Relation rel,
@@ -527,6 +528,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
TupleDesc descriptor;
List *inheritOids;
List *old_constraints;
+ char ri_type;
List *rawDefaults;
List *cookedDefaults;
Datum reloptions;
@@ -708,7 +710,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
MergeAttributes(stmt->tableElts, inheritOids,
stmt->relation->relpersistence,
stmt->partbound != NULL,
- &old_constraints);
+ &old_constraints, &ri_type);
/*
* Create a tuple descriptor from the relation schema. Note that this
@@ -1014,6 +1016,10 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
CloneForeignKeyConstraints(parentId, relationId, NULL);
+ /* Propagate REPLICA IDENTITY information too */
+ if (ri_type != REPLICA_IDENTITY_DEFAULT)
+ MatchReplicaIdentity(rel, parent);
+
table_close(parent, NoLock);
}
@@ -1873,6 +1879,8 @@ storage_name(char c)
* Output arguments:
* 'supconstr' receives a list of constraints belonging to the parents,
* updated as necessary to be valid for the child.
+ * 'ri_type' receives the replica identity type of the last parent seen,
+ * or default if none.
*
* Return value:
* Completed schema list.
@@ -1914,11 +1922,15 @@ storage_name(char c)
* (4) Otherwise the inherited default is used.
* Rule (3) is new in Postgres 7.1; in earlier releases you got a
* rather arbitrary choice of which parent default to use.
+ *
+ * It only makes sense to use the returned 'ri_type' when there's a single
+ * parent, such as in declarative partitioning.
*----------
*/
static List *
MergeAttributes(List *schema, List *supers, char relpersistence,
- bool is_partition, List **supconstr)
+ bool is_partition, List **supconstr,
+ char *ri_type)
{
ListCell *entry;
List *inhSchema = NIL;
@@ -2015,6 +2027,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
}
/*
+ * Initialize replica identity to default; parents may change it later
+ */
+ *ri_type = REPLICA_IDENTITY_DEFAULT;
+
+ /*
* Scan the parents left-to-right, and merge their attributes to form a
* list of inherited attributes (inhSchema). Also check to see if we need
* to inherit an OID column.
@@ -2095,6 +2112,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
? "cannot inherit from temporary relation of another session"
: "cannot create as partition of temporary relation of another session")));
+ /* Indicate replica identity back to caller */
+ *ri_type = relation->rd_rel->relreplident;
+
/*
* We should have an UNDER permission flag for this, but for now,
* demand that creator of a child table own the parent.
@@ -3935,7 +3955,9 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_ReplicaIdentity: /* REPLICA IDENTITY ... */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
pass = AT_PASS_MISC;
- /* This command never recurses */
+ /* Recurse only on partitioned tables */
+ if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode);
/* No command-specific prep needed */
break;
case AT_EnableTrig: /* ENABLE TRIGGER variants */
@@ -14664,6 +14686,10 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
/* and triggers */
CloneRowTriggersToPartition(rel, attachrel);
+ /* Propagate REPLICA IDENTITY information */
+ if (rel->rd_rel->relreplident != REPLICA_IDENTITY_DEFAULT)
+ MatchReplicaIdentity(attachrel, rel);
+
/*
* Clone foreign key constraints, and setup for Phase 3 to verify them.
*/
@@ -14915,6 +14941,31 @@ AttachPartitionEnsureIndexes(Relation rel, Relation attachrel)
}
/*
+ * Set up partRel's (a partition) replica identity to match parentRel's (its
+ * parent).
+ */
+static void
+MatchReplicaIdentity(Relation partRel, Relation srcrel)
+{
+ Oid ri_index;
+
+ if (srcrel->rd_rel->relreplident == REPLICA_IDENTITY_INDEX)
+ {
+ ri_index = index_get_partition(partRel,
+ RelationGetReplicaIndex(srcrel));
+ if (!OidIsValid(ri_index))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("replica index does not exist in partition")));
+ }
+ else
+ ri_index = InvalidOid;
+
+ relation_mark_replica_identity(partRel, srcrel->rd_rel->relreplident,
+ ri_index, true);
+}
+
+/*
* CloneRowTriggersToPartition
* subroutine for ATExecAttachPartition/DefineRelation to create row
* triggers on partitions
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 4da6719ce7..6145a000cb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -3113,7 +3113,8 @@ describeOneTableDetails(const char *schemaname,
if (verbose &&
(tableinfo.relkind == RELKIND_RELATION ||
- tableinfo.relkind == RELKIND_MATVIEW) &&
+ tableinfo.relkind == RELKIND_MATVIEW ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE) &&
/*
* No need to display default values; we already display a REPLICA
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 175ecd2879..3051cf1551 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -181,3 +181,102 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity_othertable;
+----
+-- Make sure it propagates to partitions
+----
+CREATE TABLE test_replica_identity_part (a int, b int) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part1 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (0) TO (1000) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part2 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (1000) TO (2000);
+CREATE TABLE test_replica_identity_part11 PARTITION OF test_replica_identity_part1
+ FOR VALUES FROM (1000) TO (1500);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_part3 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (2000) TO (3000);
+CREATE TABLE test_replica_identity_part4 (LIKE test_replica_identity_part);
+ALTER TABLE test_replica_identity_part ATTACH PARTITION test_replica_identity_part4
+ FOR VALUES FROM (3000) TO (4000);
+\d+ test_replica_identity_part2
+ Table "public.test_replica_identity_part2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (1000) TO (2000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 1000) AND (a < 2000))
+Replica Identity: FULL
+
+\d+ test_replica_identity_part11
+ Table "public.test_replica_identity_part11"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part1 FOR VALUES FROM (1000) TO (1500)
+Partition constraint: ((a IS NOT NULL) AND (a >= 0) AND (a < 1000) AND (a IS NOT NULL) AND (a >= 1000) AND (a < 1500))
+Replica Identity: FULL
+
+\d+ test_replica_identity_part
+ Partitioned table "public.test_replica_identity_part"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition key: RANGE (a)
+Partitions: test_replica_identity_part1 FOR VALUES FROM (0) TO (1000), PARTITIONED,
+ test_replica_identity_part2 FOR VALUES FROM (1000) TO (2000),
+ test_replica_identity_part3 FOR VALUES FROM (2000) TO (3000),
+ test_replica_identity_part4 FOR VALUES FROM (3000) TO (4000)
+Replica Identity: FULL
+
+\d+ test_replica_identity_part3
+ Table "public.test_replica_identity_part3"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (2000) TO (3000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 2000) AND (a < 3000))
+Replica Identity: FULL
+
+\d+ test_replica_identity_part4
+ Table "public.test_replica_identity_part4"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (3000) TO (4000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 3000) AND (a < 4000))
+Replica Identity: FULL
+
+----
+-- Check behavior with inherited tables
+----
+CREATE TABLE test_replica_identity_inh (a int);
+CREATE TABLE test_replica_identity_cld () INHERITS (test_replica_identity_inh);
+ALTER TABLE test_replica_identity_inh REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_cld2 () INHERITS (test_replica_identity_inh);
+\d+ test_replica_identity_inh
+ Table "public.test_replica_identity_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+Child tables: test_replica_identity_cld,
+ test_replica_identity_cld2
+Replica Identity: FULL
+
+\d+ test_replica_identity_cld
+ Table "public.test_replica_identity_cld"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+Inherits: test_replica_identity_inh
+
+\d+ test_replica_identity_cld2
+ Table "public.test_replica_identity_cld2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+Inherits: test_replica_identity_inh
+
diff --git a/src/test/regress/sql/replica_identity.sql b/src/test/regress/sql/replica_identity.sql
index b08a3623b8..a567f2b52d 100644
--- a/src/test/regress/sql/replica_identity.sql
+++ b/src/test/regress/sql/replica_identity.sql
@@ -77,3 +77,36 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity_othertable;
+
+----
+-- Make sure it propagates to partitions
+----
+CREATE TABLE test_replica_identity_part (a int, b int) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part1 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (0) TO (1000) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part2 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (1000) TO (2000);
+CREATE TABLE test_replica_identity_part11 PARTITION OF test_replica_identity_part1
+ FOR VALUES FROM (1000) TO (1500);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_part3 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (2000) TO (3000);
+CREATE TABLE test_replica_identity_part4 (LIKE test_replica_identity_part);
+ALTER TABLE test_replica_identity_part ATTACH PARTITION test_replica_identity_part4
+ FOR VALUES FROM (3000) TO (4000);
+\d+ test_replica_identity_part2
+\d+ test_replica_identity_part11
+\d+ test_replica_identity_part
+\d+ test_replica_identity_part3
+\d+ test_replica_identity_part4
+
+----
+-- Check behavior with inherited tables
+----
+CREATE TABLE test_replica_identity_inh (a int);
+CREATE TABLE test_replica_identity_cld () INHERITS (test_replica_identity_inh);
+ALTER TABLE test_replica_identity_inh REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_cld2 () INHERITS (test_replica_identity_inh);
+\d+ test_replica_identity_inh
+\d+ test_replica_identity_cld
+\d+ test_replica_identity_cld2
--
2.11.0
--gy7na3sjffpqghel--
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v3 2/2] Propagate replica identity to partitions
@ 2019-02-04 16:43 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Alvaro Herrera @ 2019-02-04 16:43 UTC (permalink / raw)
---
src/backend/commands/tablecmds.c | 132 ++++++++++++++++++--
src/bin/psql/describe.c | 3 +-
src/test/regress/expected/replica_identity.out | 160 +++++++++++++++++++++++++
src/test/regress/sql/replica_identity.sql | 43 +++++++
4 files changed, 326 insertions(+), 12 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 877bac506f..b26872e78f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -300,7 +300,7 @@ static void truncate_check_activity(Relation rel);
static void RangeVarCallbackForTruncate(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
static List *MergeAttributes(List *schema, List *supers, char relpersistence,
- bool is_partition, List **supconstr);
+ bool is_partition, List **supconstr, char *ri_type);
static bool MergeCheckConstraint(List *constraints, char *name, Node *expr);
static void MergeAttributesIntoExisting(Relation child_rel, Relation parent_rel);
static void MergeConstraintsIntoExisting(Relation child_rel, Relation parent_rel);
@@ -485,6 +485,7 @@ static void AttachPartitionEnsureIndexes(Relation rel, Relation attachrel);
static void QueuePartitionConstraintValidation(List **wqueue, Relation scanrel,
List *partConstraint,
bool validate_default);
+static void MatchReplicaIdentity(Relation tgtrel, Relation srcrel);
static void CloneRowTriggersToPartition(Relation parent, Relation partition);
static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name);
static ObjectAddress ATExecAttachPartitionIdx(List **wqueue, Relation rel,
@@ -527,6 +528,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
TupleDesc descriptor;
List *inheritOids;
List *old_constraints;
+ char ri_type;
List *rawDefaults;
List *cookedDefaults;
Datum reloptions;
@@ -708,7 +710,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
MergeAttributes(stmt->tableElts, inheritOids,
stmt->relation->relpersistence,
stmt->partbound != NULL,
- &old_constraints);
+ &old_constraints, &ri_type);
/*
* Create a tuple descriptor from the relation schema. Note that this
@@ -1014,6 +1016,10 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
CloneForeignKeyConstraints(parentId, relationId, NULL);
+ /* Propagate REPLICA IDENTITY information too */
+ if (ri_type != REPLICA_IDENTITY_DEFAULT)
+ MatchReplicaIdentity(rel, parent);
+
table_close(parent, NoLock);
}
@@ -1873,6 +1879,8 @@ storage_name(char c)
* Output arguments:
* 'supconstr' receives a list of constraints belonging to the parents,
* updated as necessary to be valid for the child.
+ * 'ri_type' receives the replica identity type of the last parent seen,
+ * or default if none.
*
* Return value:
* Completed schema list.
@@ -1914,11 +1922,15 @@ storage_name(char c)
* (4) Otherwise the inherited default is used.
* Rule (3) is new in Postgres 7.1; in earlier releases you got a
* rather arbitrary choice of which parent default to use.
+ *
+ * It only makes sense to use the returned 'ri_type' when there's a single
+ * parent, such as in declarative partitioning.
*----------
*/
static List *
MergeAttributes(List *schema, List *supers, char relpersistence,
- bool is_partition, List **supconstr)
+ bool is_partition, List **supconstr,
+ char *ri_type)
{
ListCell *entry;
List *inhSchema = NIL;
@@ -2015,6 +2027,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
}
/*
+ * Initialize replica identity to default; parents may change it later
+ */
+ *ri_type = REPLICA_IDENTITY_DEFAULT;
+
+ /*
* Scan the parents left-to-right, and merge their attributes to form a
* list of inherited attributes (inhSchema). Also check to see if we need
* to inherit an OID column.
@@ -2095,6 +2112,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
? "cannot inherit from temporary relation of another session"
: "cannot create as partition of temporary relation of another session")));
+ /* Indicate replica identity back to caller */
+ *ri_type = relation->rd_rel->relreplident;
+
/*
* We should have an UNDER permission flag for this, but for now,
* demand that creator of a child table own the parent.
@@ -3935,7 +3955,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_ReplicaIdentity: /* REPLICA IDENTITY ... */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
pass = AT_PASS_MISC;
- /* This command never recurses */
+ /* Recursion occurs during execution phase */
/* No command-specific prep needed */
break;
case AT_EnableTrig: /* ENABLE TRIGGER variants */
@@ -12753,10 +12773,13 @@ ATExecDropOf(Relation rel, LOCKMODE lockmode)
*
* Iff ri_type = REPLICA_IDENTITY_INDEX, indexOid must be the Oid of a suitable
* index. Otherwise, it should be InvalidOid.
+ *
+ * 'permissive' means to ignore the case where a partitioned table contains
+ * a partition without the index. If false, raise an error.
*/
static void
relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
- bool is_internal)
+ bool is_internal, bool permissive, LOCKMODE lockmode)
{
Relation pg_index;
Relation pg_class;
@@ -12847,6 +12870,58 @@ relation_mark_replica_identity(Relation rel, char ri_type, Oid indexOid,
}
table_close(pg_index, RowExclusiveLock);
+
+ /*
+ * If there are any partitions, handle them too.
+ */
+ if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ PartitionDesc pd = RelationGetPartitionDesc(rel);
+
+ for (int i = 0; i < pd->nparts; i++)
+ {
+ Relation part = table_open(pd->oids[i], lockmode);
+ Oid idxOid;
+
+ if (ri_type == REPLICA_IDENTITY_INDEX)
+ {
+ /*
+ * When using an index as identity, and some partition does not
+ * have the index, we either raise an error (if not in
+ * "permissive" mode), or ignore the partition; this happens
+ * when the index hierarchy is being restored by pg_dump. The
+ * only thing we can do at this point in that case is hope that
+ * there will be another command creating the right index soon.
+ */
+ idxOid = index_get_partition(part, indexOid);
+ if (!OidIsValid(idxOid))
+ {
+ if (permissive)
+ {
+ table_close(part, NoLock);
+ continue;
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("replica index does not exist in partition \"%s\"",
+ RelationGetRelationName(part))));
+ }
+
+ LockRelationOid(idxOid, ShareLock);
+ }
+ else
+ idxOid = InvalidOid;
+
+ idxOid = ri_type == REPLICA_IDENTITY_INDEX ?
+ index_get_partition(part, indexOid) : InvalidOid;
+
+ relation_mark_replica_identity(part, ri_type, idxOid, true,
+ permissive, lockmode);
+
+ table_close(part, NoLock);
+ }
+ }
}
/*
@@ -12861,17 +12936,20 @@ ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKMODE lockmode
if (stmt->identity_type == REPLICA_IDENTITY_DEFAULT)
{
- relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true);
+ relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true,
+ false, lockmode);
return;
}
else if (stmt->identity_type == REPLICA_IDENTITY_FULL)
{
- relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true);
+ relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true,
+ false, lockmode);
return;
}
else if (stmt->identity_type == REPLICA_IDENTITY_NOTHING)
{
- relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true);
+ relation_mark_replica_identity(rel, stmt->identity_type, InvalidOid, true,
+ false, lockmode);
return;
}
else if (stmt->identity_type == REPLICA_IDENTITY_INDEX)
@@ -12925,8 +13003,9 @@ ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKMODE lockmode
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot use partial index \"%s\" as replica identity",
RelationGetRelationName(indexRel))));
- /* And neither are invalid indexes. */
- if (!indexRel->rd_index->indisvalid)
+ /* And neither are invalid indexes, except on partitioned tables. */
+ if (!indexRel->rd_index->indisvalid &&
+ rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot use invalid index \"%s\" as replica identity",
@@ -12959,7 +13038,8 @@ ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKMODE lockmode
}
/* This index is suitable for use as a replica identity. Mark it. */
- relation_mark_replica_identity(rel, stmt->identity_type, indexOid, true);
+ relation_mark_replica_identity(rel, stmt->identity_type, indexOid, true,
+ true, lockmode);
index_close(indexRel, NoLock);
}
@@ -14664,6 +14744,10 @@ ATExecAttachPartition(List **wqueue, Relation rel, PartitionCmd *cmd)
/* and triggers */
CloneRowTriggersToPartition(rel, attachrel);
+ /* Propagate REPLICA IDENTITY information */
+ if (rel->rd_rel->relreplident != REPLICA_IDENTITY_DEFAULT)
+ MatchReplicaIdentity(attachrel, rel);
+
/*
* Clone foreign key constraints, and setup for Phase 3 to verify them.
*/
@@ -14915,6 +14999,32 @@ AttachPartitionEnsureIndexes(Relation rel, Relation attachrel)
}
/*
+ * Set up partRel's (a partition) replica identity to match parentRel's (its
+ * parent).
+ */
+static void
+MatchReplicaIdentity(Relation partRel, Relation srcrel)
+{
+ Oid ri_index;
+
+ if (srcrel->rd_rel->relreplident == REPLICA_IDENTITY_INDEX)
+ {
+ ri_index = index_get_partition(partRel,
+ RelationGetReplicaIndex(srcrel));
+ if (!OidIsValid(ri_index))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("replica index does not exist in partition \"%s\"",
+ RelationGetRelationName(partRel))));
+ }
+ else
+ ri_index = InvalidOid;
+
+ relation_mark_replica_identity(partRel, srcrel->rd_rel->relreplident,
+ ri_index, true, false, AccessExclusiveLock);
+}
+
+/*
* CloneRowTriggersToPartition
* subroutine for ATExecAttachPartition/DefineRelation to create row
* triggers on partitions
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 4da6719ce7..6145a000cb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -3113,7 +3113,8 @@ describeOneTableDetails(const char *schemaname,
if (verbose &&
(tableinfo.relkind == RELKIND_RELATION ||
- tableinfo.relkind == RELKIND_MATVIEW) &&
+ tableinfo.relkind == RELKIND_MATVIEW ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE) &&
/*
* No need to display default values; we already display a REPLICA
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 175ecd2879..d6014df840 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -181,3 +181,163 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity_othertable;
+----
+-- Make sure it propagates to partitions
+----
+CREATE TABLE test_replica_identity_part (a int, b int) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part1 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (0) TO (1000) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part2 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (1000) TO (2000);
+CREATE TABLE test_replica_identity_part11 PARTITION OF test_replica_identity_part1
+ FOR VALUES FROM (1000) TO (1500);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_part3 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (2000) TO (3000);
+CREATE TABLE test_replica_identity_part4 (LIKE test_replica_identity_part);
+ALTER TABLE test_replica_identity_part ATTACH PARTITION test_replica_identity_part4
+ FOR VALUES FROM (3000) TO (4000);
+\d+ test_replica_identity_part2
+ Table "public.test_replica_identity_part2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (1000) TO (2000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 1000) AND (a < 2000))
+Replica Identity: FULL
+
+\d+ test_replica_identity_part11
+ Table "public.test_replica_identity_part11"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part1 FOR VALUES FROM (1000) TO (1500)
+Partition constraint: ((a IS NOT NULL) AND (a >= 0) AND (a < 1000) AND (a IS NOT NULL) AND (a >= 1000) AND (a < 1500))
+Replica Identity: FULL
+
+\d+ test_replica_identity_part
+ Partitioned table "public.test_replica_identity_part"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition key: RANGE (a)
+Partitions: test_replica_identity_part1 FOR VALUES FROM (0) TO (1000), PARTITIONED,
+ test_replica_identity_part2 FOR VALUES FROM (1000) TO (2000),
+ test_replica_identity_part3 FOR VALUES FROM (2000) TO (3000),
+ test_replica_identity_part4 FOR VALUES FROM (3000) TO (4000)
+Replica Identity: FULL
+
+\d+ test_replica_identity_part3
+ Table "public.test_replica_identity_part3"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (2000) TO (3000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 2000) AND (a < 3000))
+Replica Identity: FULL
+
+\d+ test_replica_identity_part4
+ Table "public.test_replica_identity_part4"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (3000) TO (4000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 3000) AND (a < 4000))
+Replica Identity: FULL
+
+ALTER TABLE test_replica_identity_part ALTER a SET NOT NULL;
+CREATE UNIQUE INDEX trip_b_idx ON test_replica_identity_part (a);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY USING INDEX trip_b_idx;
+\d+ test_replica_identity_part2
+ Table "public.test_replica_identity_part2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (1000) TO (2000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 1000) AND (a < 2000))
+Indexes:
+ "test_replica_identity_part2_a_idx" UNIQUE, btree (a) REPLICA IDENTITY
+
+\d+ test_replica_identity_part11
+ Table "public.test_replica_identity_part11"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part1 FOR VALUES FROM (1000) TO (1500)
+Partition constraint: ((a IS NOT NULL) AND (a >= 0) AND (a < 1000) AND (a IS NOT NULL) AND (a >= 1000) AND (a < 1500))
+Indexes:
+ "test_replica_identity_part11_a_idx" UNIQUE, btree (a) REPLICA IDENTITY
+
+\d+ test_replica_identity_part
+ Partitioned table "public.test_replica_identity_part"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition key: RANGE (a)
+Indexes:
+ "trip_b_idx" UNIQUE, btree (a) REPLICA IDENTITY
+Partitions: test_replica_identity_part1 FOR VALUES FROM (0) TO (1000), PARTITIONED,
+ test_replica_identity_part2 FOR VALUES FROM (1000) TO (2000),
+ test_replica_identity_part3 FOR VALUES FROM (2000) TO (3000),
+ test_replica_identity_part4 FOR VALUES FROM (3000) TO (4000)
+
+\d+ test_replica_identity_part3
+ Table "public.test_replica_identity_part3"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (2000) TO (3000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 2000) AND (a < 3000))
+Indexes:
+ "test_replica_identity_part3_a_idx" UNIQUE, btree (a) REPLICA IDENTITY
+
+\d+ test_replica_identity_part4
+ Table "public.test_replica_identity_part4"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | not null | | plain | |
+ b | integer | | | | plain | |
+Partition of: test_replica_identity_part FOR VALUES FROM (3000) TO (4000)
+Partition constraint: ((a IS NOT NULL) AND (a >= 3000) AND (a < 4000))
+Indexes:
+ "test_replica_identity_part4_a_idx" UNIQUE, btree (a) REPLICA IDENTITY
+
+----
+-- Check behavior with inherited tables
+----
+CREATE TABLE test_replica_identity_inh (a int);
+CREATE TABLE test_replica_identity_cld () INHERITS (test_replica_identity_inh);
+ALTER TABLE test_replica_identity_inh REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_cld2 () INHERITS (test_replica_identity_inh);
+\d+ test_replica_identity_inh
+ Table "public.test_replica_identity_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+Child tables: test_replica_identity_cld,
+ test_replica_identity_cld2
+Replica Identity: FULL
+
+\d+ test_replica_identity_cld
+ Table "public.test_replica_identity_cld"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+Inherits: test_replica_identity_inh
+
+\d+ test_replica_identity_cld2
+ Table "public.test_replica_identity_cld2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ a | integer | | | | plain | |
+Inherits: test_replica_identity_inh
+
diff --git a/src/test/regress/sql/replica_identity.sql b/src/test/regress/sql/replica_identity.sql
index b08a3623b8..9e309796f2 100644
--- a/src/test/regress/sql/replica_identity.sql
+++ b/src/test/regress/sql/replica_identity.sql
@@ -77,3 +77,46 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
DROP TABLE test_replica_identity;
DROP TABLE test_replica_identity_othertable;
+
+----
+-- Make sure it propagates to partitions
+----
+CREATE TABLE test_replica_identity_part (a int, b int) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part1 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (0) TO (1000) PARTITION BY RANGE (a);
+CREATE TABLE test_replica_identity_part2 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (1000) TO (2000);
+CREATE TABLE test_replica_identity_part11 PARTITION OF test_replica_identity_part1
+ FOR VALUES FROM (1000) TO (1500);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_part3 PARTITION OF test_replica_identity_part
+ FOR VALUES FROM (2000) TO (3000);
+CREATE TABLE test_replica_identity_part4 (LIKE test_replica_identity_part);
+ALTER TABLE test_replica_identity_part ATTACH PARTITION test_replica_identity_part4
+ FOR VALUES FROM (3000) TO (4000);
+\d+ test_replica_identity_part2
+\d+ test_replica_identity_part11
+\d+ test_replica_identity_part
+\d+ test_replica_identity_part3
+\d+ test_replica_identity_part4
+
+ALTER TABLE test_replica_identity_part ALTER a SET NOT NULL;
+CREATE UNIQUE INDEX trip_b_idx ON test_replica_identity_part (a);
+ALTER TABLE test_replica_identity_part REPLICA IDENTITY USING INDEX trip_b_idx;
+\d+ test_replica_identity_part2
+\d+ test_replica_identity_part11
+\d+ test_replica_identity_part
+\d+ test_replica_identity_part3
+\d+ test_replica_identity_part4
+
+
+----
+-- Check behavior with inherited tables
+----
+CREATE TABLE test_replica_identity_inh (a int);
+CREATE TABLE test_replica_identity_cld () INHERITS (test_replica_identity_inh);
+ALTER TABLE test_replica_identity_inh REPLICA IDENTITY FULL;
+CREATE TABLE test_replica_identity_cld2 () INHERITS (test_replica_identity_inh);
+\d+ test_replica_identity_inh
+\d+ test_replica_identity_cld
+\d+ test_replica_identity_cld2
--
2.11.0
--wqciur7sbsf3uhpb--
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v32 06/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/expected/tablespace.out | 7 +++++++
src/test/regress/sql/tablespace.sql | 4 ++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index eed71892bd1..c381a170ad6 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -680,7 +680,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index 7d8398b0ca1..549117256fc 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -24,6 +24,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/sql/tablespace.sql b/src/test/regress/sql/tablespace.sql
index 291ca029c14..cc5f2f9d0dc 100644
--- a/src/test/regress/sql/tablespace.sql
+++ b/src/test/regress/sql/tablespace.sql
@@ -21,6 +21,10 @@ CREATE TABLESPACE regress_tblspace LOCATION :'testtablespace';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--Bne5rrxQd65beI7a
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v32-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v18 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--5I6of5zJg18YgZEa
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v18-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v20 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index a4d4782c8c..e1fb160124 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -683,7 +683,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--Z1Z8UV8BNhgCynIS
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v20-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v28 06/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index a7d1a65f10..e25010d40c 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -680,7 +680,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index a32212be04..bb184d3fe0 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 4183a77b23..3571f14626 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--vk/v8fjDPiDepTtA
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v12 10/11] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index de63e77836..810c6b0f23 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -651,7 +651,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_MODERN);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_MODERN|LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--wIc/V6YLA2QdyfT4
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v12-0011-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v13 7/8] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 342d6f1205..21c265aab3 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -642,7 +642,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_MODERN);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_MODERN|LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--E0h0CbphJD8hN+Gf
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v13-0008-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v23 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index a07d1717f9..f7a0cb3f46 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -677,7 +677,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--mhjHhnbe5PrRcwjY
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v23-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v14 7/8] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 21611dfbc0..00de0c091d 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -653,7 +653,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--Kynn+LdAwU9N+JqL
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v14-0008-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v34 06/15] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/expected/tablespace.out | 7 +++++++
src/test/regress/sql/tablespace.sql | 4 ++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 1c84be69e91..80516833e33 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -639,7 +639,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index 20a7245d5e1..50a3a13fbea 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -32,6 +32,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/sql/tablespace.sql b/src/test/regress/sql/tablespace.sql
index 3aadac8b611..034c903b41c 100644
--- a/src/test/regress/sql/tablespace.sql
+++ b/src/test/regress/sql/tablespace.sql
@@ -28,6 +28,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.1
--smOfPzt+Qjm5bNGJ
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v34-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v35 6/7] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/expected/tablespace.out | 7 +++++++
src/test/regress/sql/tablespace.sql | 4 ++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index cfb7fc7e080..ca5223be2ee 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -641,7 +641,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index ce5d6f73e01..53662fb0efd 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -41,6 +41,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/sql/tablespace.sql b/src/test/regress/sql/tablespace.sql
index cf683c3bf3a..18b435c9abb 100644
--- a/src/test/regress/sql/tablespace.sql
+++ b/src/test/regress/sql/tablespace.sql
@@ -32,6 +32,10 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN'
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.1
--olLTNZSltDMg5Vbm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v35-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v10 8/9] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 3ea8c5683e..19e0b8a82c 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -612,7 +612,7 @@ Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
return pg_ls_dir_files(fcinfo, Log_directory,
- FLAG_SKIP_HIDDEN|FLAG_SKIP_SPECIAL|FLAG_METADATA);
+ FLAG_MISSING_OK|FLAG_SKIP_HIDDEN|FLAG_SKIP_SPECIAL|FLAG_METADATA);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..dacaafcae6 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification
+------+------+--------------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--rCwQ2Y43eQY6RBgR
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v10-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v31 06/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index eed71892bd..c381a170ad 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -680,7 +680,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index e69fa17004..da624020b2 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index c146a4c129..3689bfac6d 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--qZVVwWJgpX9Jzs7f
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v31-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v25 07/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index d21a95aebe..07fce2a9bc 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -677,7 +677,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--SBikYMzjhZGK9d4p
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0008-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v11 8/9] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 9c2feb0986..2d4f561ae0 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -632,7 +632,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_MODERN);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_MODERN|LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--oTHb8nViIGeoXxdp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v11-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v9 09/11] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 888560f78e..4ce39516d7 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -616,7 +616,7 @@ Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
return pg_ls_dir_files(fcinfo, Log_directory,
- FLAG_SKIP_DIRS|FLAG_SKIP_HIDDEN|FLAG_METADATA);
+ FLAG_MISSING_OK|FLAG_SKIP_DIRS|FLAG_SKIP_HIDDEN|FLAG_METADATA);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..dacaafcae6 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification
+------+------+--------------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--32u276st3Jlj2kUU
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v9-0010-pg_ls_-dir-to-show-directories-and-isdir-column.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v33 06/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/expected/tablespace.out | 7 +++++++
src/test/regress/sql/tablespace.sql | 4 ++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index e24c43e3a9b..e49a15aa279 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -681,7 +681,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index 20a7245d5e1..50a3a13fbea 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -32,6 +32,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/sql/tablespace.sql b/src/test/regress/sql/tablespace.sql
index 3aadac8b611..034c903b41c 100644
--- a/src/test/regress/sql/tablespace.sql
+++ b/src/test/regress/sql/tablespace.sql
@@ -28,6 +28,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.1
--9CzcV6dAFIr7O1Ie
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v33-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v30 06/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 12bb70c442..f3437d306a 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -679,7 +679,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index a32212be04..bb184d3fe0 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 4183a77b23..3571f14626 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--ZwgA9U+XZDXt4+m+
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v30-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v26 07/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index d21a95aebe..07fce2a9bc 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -677,7 +677,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--yKkOmjQZXRsvHRX8
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v26-0008-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v27 06/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index a7d1a65f10..e25010d40c 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -680,7 +680,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index a32212be04..bb184d3fe0 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 4183a77b23..3571f14626 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--19uQFt6ulqmgNgg1
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v22 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index a61ddfc451..35fe4d7cc1 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -677,7 +677,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--d6Gm4EdcadzBjdND
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v22-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v19 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--S0GG+JvAI2G0KxBG
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v19-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v21 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 92e045e613..16a929f5bb 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -683,7 +683,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--Tcb1KvpfnM4LxW2s
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v21-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v36 6/7] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/expected/tablespace.out | 7 +++++++
src/test/regress/sql/tablespace.sql | 4 ++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fd837e22987..a01c7109f4c 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -641,7 +641,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index ce5d6f73e01..53662fb0efd 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -41,6 +41,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/sql/tablespace.sql b/src/test/regress/sql/tablespace.sql
index cf683c3bf3a..18b435c9abb 100644
--- a/src/test/regress/sql/tablespace.sql
+++ b/src/test/regress/sql/tablespace.sql
@@ -32,6 +32,10 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN'
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.1
--4ybNbZnZ8tziJ7D6
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v36-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index fba63568b6..91bf8c69e9 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -647,7 +647,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0009-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v37 06/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/expected/tablespace.out | 7 +++++++
src/test/regress/sql/tablespace.sql | 4 ++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 5356136ff5a..5402ecb3d05 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -711,7 +711,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out
index ce5d6f73e01..53662fb0efd 100644
--- a/src/test/regress/expected/tablespace.out
+++ b/src/test/regress/expected/tablespace.out
@@ -41,6 +41,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/sql/tablespace.sql b/src/test/regress/sql/tablespace.sql
index cf683c3bf3a..18b435c9abb 100644
--- a/src/test/regress/sql/tablespace.sql
+++ b/src/test/regress/sql/tablespace.sql
@@ -32,6 +32,10 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN'
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.25.1
--Pk/CTwBz1VvfPIDp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v37-0007-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* [PATCH v24 07/11] pg_ls_logdir to ignore error if initial/top dir is missing..
@ 2020-03-06 23:23 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Justin Pryzby @ 2020-03-06 23:23 UTC (permalink / raw)
..since ./log is created dynamically and not by initdb
---
src/backend/utils/adt/genfile.c | 2 +-
src/test/regress/input/tablespace.source | 4 ++++
src/test/regress/output/tablespace.source | 7 +++++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index d21a95aebe..07fce2a9bc 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -677,7 +677,7 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags)
Datum
pg_ls_logdir(PG_FUNCTION_ARGS)
{
- return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON);
+ return pg_ls_dir_files(fcinfo, Log_directory, LS_DIR_COMMON | LS_DIR_MISSING_OK);
}
/* Function to return the list of files in the WAL directory */
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/input/tablespace.source
index 0b9cfe615e..2a1268e17c 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/input/tablespace.source
@@ -16,6 +16,10 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
-- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir()
SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist';
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1c88e914e3..ba9a3fe29a 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -21,6 +21,13 @@ SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspa
------+------+--------------+-------
(0 rows)
+-- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet.
+-- The name='' condition is never true, so the function runs to completion but returns zero rows.
+SELECT * FROM pg_ls_logdir() WHERE name='Does not exist';
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
--
2.17.0
--mPTHnM80CEnHQ2WJ
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0008-pg_ls_-dir-to-return-all-the-metadata-from-pg_st.patch"
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: Disabling Heap-Only Tuples
@ 2023-09-19 10:26 Alvaro Herrera <[email protected]>
2023-09-19 16:09 ` Re: Disabling Heap-Only Tuples Robert Haas <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Alvaro Herrera @ 2023-09-19 10:26 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Laurenz Albe <[email protected]>; Matthias van de Meent <[email protected]>; Thom Brown <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2023-Sep-18, Robert Haas wrote:
> On Tue, Sep 5, 2023 at 11:15 PM Laurenz Albe <[email protected]> wrote:
> > I don't think that is a good comparison. While most people probably
> > never need to touch "local_update_limit", "work_mem" is something everybody
> > has to consider.
> >
> > And it is not so hard to tune: the setting would be the desired table
> > size, and you could use pgstattuple to find a good value.
>
> What I suspect would happen, though, is that you'd end up tuning the
> value over and over. You'd set it to some value and after some number
> of vacuums maybe you'd realize that you could save even more disk
> space if you reduced it a bit further or maybe your data set would
> grow a bit and you'd have to increase it a little (or a lot). And if
> you didn't keep adjusting it then maybe something quite bad would
> happen to your database.
As I understand it, the setting being proposed is useful as an emergency
for removing excessive bloat -- a substitute for VACUUM FULL when you
don't want to lock the table for long. Trying to use it as a permanent
gadget is going to be misguided. So my first thought is that we should
tell people to use it that way: if you're not in the irrecoverable-space
situation, just do not use this. Then we don't have to worry about
people misusing it the way you imagine.
Second, I think we should make it auto-reset. That is, have the user
set some value; later, when some condition triggers (say, the table size
is 1.2x the limit value you configured), then the local_update_limit is
automatically removed from the table options. From that point onwards,
the table is operated normally.
This removes the other concern that makes the system behaves
suboptimally because some DBA in the past decade left this set for no
good reason: if you run into an emergency, then you activate the
emergency escape hatch, and it will close on its own as soon as the
emergency is over.
This also dissuades people from using it for these other things you
describe. It just won't work.
The point here is that third-party tools such as pg_repack or pg_squeeze
exist, which work in a way we don't like, yet we offer no alternative.
This proposal is a mechanism that essentially replaces those tools with
a simple in-core feature, without having to include the tool itself in
core.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Thou shalt check the array bounds of all strings (indeed, all arrays), for
surely where thou typest "foo" someone someday shall type
"supercalifragilisticexpialidocious" (5th Commandment for C programmers)
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: Disabling Heap-Only Tuples
2023-09-19 10:26 Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
@ 2023-09-19 16:09 ` Robert Haas <[email protected]>
2023-09-19 16:30 ` Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Robert Haas @ 2023-09-19 16:09 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Laurenz Albe <[email protected]>; Matthias van de Meent <[email protected]>; Thom Brown <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, Sep 19, 2023 at 6:26 AM Alvaro Herrera <[email protected]> wrote:
> Second, I think we should make it auto-reset. That is, have the user
> set some value; later, when some condition triggers (say, the table size
> is 1.2x the limit value you configured), then the local_update_limit is
> automatically removed from the table options. From that point onwards,
> the table is operated normally.
That's an interesting idea. It would require taking AEL on the table.
And also, what do you mean by 1.2x the limit value? Is that supposed
to be a >= condition or a <= condition? It can't really be a >=
condition, but you wouldn't set it in the first place unless the table
were significantly bigger than it could be. But if it's a <= condition
it doesn't really protect you from hosing yourself. You just have to
insert a bit more data before enough of the bloat gets removed, and
now the table just bloats infinitely and probably rather quickly. The
correct value of the setting depends on the amount of real data
(non-bloat) in the table, not the actual table size.
> The point here is that third-party tools such as pg_repack or pg_squeeze
> exist, which work in a way we don't like, yet we offer no alternative.
> This proposal is a mechanism that essentially replaces those tools with
> a simple in-core feature, without having to include the tool itself in
> core.
I agree that it would be nice to have something in core that can be
used to help with this problem, but this feature isn't the same thing
as pg_repack or pg_squeeze, either. In some ways, it's better, because
it can shrink the table without rewriting it, which is very desirable.
But in other ways, it's worse, and the fact that it seems like it can
backfire spectacularly if you set the wrong value seems like one big
way that it is a lot worse. If there is a way that we can make this a
mode that you activate for a table, and the system calculates and
updates the threshold, I think that would actually be a pretty good
feature. It would be tricky to use it to recover from acute
emergencies, because it doesn't actually do anything until updates
happen, but you could use it for that in a pinch. And even without
that it would be useful if you have a table that is sometimes very
large and sometimes very small and you want to get the space back from
the OS when it is in the small phase of its lifecycle.
But without any kind of auto-tuning, in my opinion, it's a fairly poor
feature. Sure, some people will get use out of it, if they're
sufficiently knowledgeable and sufficiently determined. But I think
for most people in most situations, it will be a struggle.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: Disabling Heap-Only Tuples
2023-09-19 10:26 Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
2023-09-19 16:09 ` Re: Disabling Heap-Only Tuples Robert Haas <[email protected]>
@ 2023-09-19 16:30 ` Alvaro Herrera <[email protected]>
2023-09-19 16:56 ` Re: Disabling Heap-Only Tuples Andres Freund <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Alvaro Herrera @ 2023-09-19 16:30 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Laurenz Albe <[email protected]>; Matthias van de Meent <[email protected]>; Thom Brown <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2023-Sep-19, Robert Haas wrote:
> On Tue, Sep 19, 2023 at 6:26 AM Alvaro Herrera <[email protected]> wrote:
> > Second, I think we should make it auto-reset. That is, have the user
> > set some value; later, when some condition triggers (say, the table size
> > is 1.2x the limit value you configured), then the local_update_limit is
> > automatically removed from the table options. From that point onwards,
> > the table is operated normally.
>
> That's an interesting idea. It would require taking AEL on the table.
> And also, what do you mean by 1.2x the limit value? Is that supposed
> to be a >= condition or a <= condition? It can't really be a >=
> condition, but you wouldn't set it in the first place unless the table
> were significantly bigger than it could be. But if it's a <= condition
> it doesn't really protect you from hosing yourself. You just have to
> insert a bit more data before enough of the bloat gets removed, and
> now the table just bloats infinitely and probably rather quickly. The
> correct value of the setting depends on the amount of real data
> (non-bloat) in the table, not the actual table size.
I was thinking something vaguely like "a table size that's roughly what
an optimal autovacuuming schedule would leave the table at" assuming 0.2
vacuum_scale_factor. You would determine the absolute minimum size for
the table given the current live tuples in the table, then add 20% to
account for a steady state of dead tuples and vacuumed space. So it's
not 1.2x of the "current" table size at the time the local_update_limit
feature is installed, but 1.2x of the optimal table size.
This makes me think that maybe the logic needs to be a little more
complex to avoid the problem you describe: if an UPDATE is prevented
from being HOT because of this setting, but then it goes and consults
FSM and it gives the update a higher block number than the tuple's
current block (or it fails to give a block number at all so it is forced
to extend the relation), then the update should give up on that strategy
and use a HOT update after all. (I have not read the actual patch;
maybe it already does this? It sounds kinda obvious.)
Having to set AEL is not nice for sure, but wouldn't
ShareUpdateExclusiveLock be sufficient? We have a bunch of reloptions
for which that is sufficient.
> But without any kind of auto-tuning, in my opinion, it's a fairly poor
> feature. Sure, some people will get use out of it, if they're
> sufficiently knowledgeable and sufficiently determined. But I think
> for most people in most situations, it will be a struggle.
>
> --
> Robert Haas
> EDB: http://www.enterprisedb.com
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Tiene valor aquel que admite que es un cobarde" (Fernandel)
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: Disabling Heap-Only Tuples
2023-09-19 10:26 Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
2023-09-19 16:09 ` Re: Disabling Heap-Only Tuples Robert Haas <[email protected]>
2023-09-19 16:30 ` Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
@ 2023-09-19 16:56 ` Andres Freund <[email protected]>
2023-09-19 17:33 ` Re: Disabling Heap-Only Tuples Matthias van de Meent <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Andres Freund @ 2023-09-19 16:56 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Robert Haas <[email protected]>; Laurenz Albe <[email protected]>; Matthias van de Meent <[email protected]>; Thom Brown <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On 2023-09-19 18:30:44 +0200, Alvaro Herrera wrote:
> This makes me think that maybe the logic needs to be a little more
> complex to avoid the problem you describe: if an UPDATE is prevented
> from being HOT because of this setting, but then it goes and consults
> FSM and it gives the update a higher block number than the tuple's
> current block (or it fails to give a block number at all so it is forced
> to extend the relation), then the update should give up on that strategy
> and use a HOT update after all. (I have not read the actual patch;
> maybe it already does this? It sounds kinda obvious.)
Yea, a setting like what's discussed here seems, uh, not particularly useful
for achieving the goal of compacting tables. I don't think guiding this
through SQL makes a lot of sense. For decent compaction you'd want to scan the
table backwards, and move rows from the end to earlier, but stop once
everything is filled up. You can somewhat do that from SQL, but it's going to
be awkward and slow. I doubt you even want to use the normal UPDATE WAL
logging.
I think having explicit compaction support in VACUUM or somewhere similar
would make sense, but I don't think the proposed GUC is a useful stepping
stone.
> > But without any kind of auto-tuning, in my opinion, it's a fairly poor
> > feature. Sure, some people will get use out of it, if they're
> > sufficiently knowledgeable and sufficiently determined. But I think
> > for most people in most situations, it will be a struggle.
Indeed. I think it'd often just explode table and index sizes, because HOT
pruning won't be able to make usable space in pages anymore (due to dead
items).
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: Disabling Heap-Only Tuples
2023-09-19 10:26 Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
2023-09-19 16:09 ` Re: Disabling Heap-Only Tuples Robert Haas <[email protected]>
2023-09-19 16:30 ` Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
2023-09-19 16:56 ` Re: Disabling Heap-Only Tuples Andres Freund <[email protected]>
@ 2023-09-19 17:33 ` Matthias van de Meent <[email protected]>
2023-09-19 18:55 ` Re: Disabling Heap-Only Tuples Andres Freund <[email protected]>
0 siblings, 1 reply; 91+ messages in thread
From: Matthias van de Meent @ 2023-09-19 17:33 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Laurenz Albe <[email protected]>; Matthias van de Meent <[email protected]>; Thom Brown <[email protected]>; PostgreSQL Hackers <[email protected]>
On Tue, 19 Sept 2023 at 18:56, Andres Freund <[email protected]> wrote:
>
> Hi,
>
> On 2023-09-19 18:30:44 +0200, Alvaro Herrera wrote:
> > This makes me think that maybe the logic needs to be a little more
> > complex to avoid the problem you describe: if an UPDATE is prevented
> > from being HOT because of this setting, but then it goes and consults
> > FSM and it gives the update a higher block number than the tuple's
> > current block (or it fails to give a block number at all so it is forced
> > to extend the relation), then the update should give up on that strategy
> > and use a HOT update after all. (I have not read the actual patch;
> > maybe it already does this? It sounds kinda obvious.)
>
> Yea, a setting like what's discussed here seems, uh, not particularly useful
> for achieving the goal of compacting tables. I don't think guiding this
> through SQL makes a lot of sense. For decent compaction you'd want to scan the
> table backwards, and move rows from the end to earlier, but stop once
> everything is filled up. You can somewhat do that from SQL, but it's going to
> be awkward and slow. I doubt you even want to use the normal UPDATE WAL
> logging.
We can't move tuples around (or, not that I know of) without using a
transaction ID to control the visibility of the two locations of that
tuple. Doing table compaction would thus likely require using
transactions to move these tuples around. Using a single backend and
bulk operations, it'll still lock each tuple that is being moved, and
that can be noticed by user DML queries. I'd rather make the user's
queries move the data around than this long-duration, locking
background operation.
> I think having explicit compaction support in VACUUM or somewhere similar
> would make sense, but I don't think the proposed GUC is a useful stepping
> stone.
The point of this GUC is that the compaction can happen organically in
the user's UPDATE workflow, so that there is no long locking operation
going on (as you would see with VACUUM FULL / CLUSTER / pg_repack).
> > > But without any kind of auto-tuning, in my opinion, it's a fairly poor
> > > feature. Sure, some people will get use out of it, if they're
> > > sufficiently knowledgeable and sufficiently determined. But I think
> > > for most people in most situations, it will be a struggle.
>
> Indeed. I think it'd often just explode table and index sizes, because HOT
> pruning won't be able to make usable space in pages anymore (due to dead
> items).
You seem to misunderstand the latest patch. It explicitly only blocks
local updates if the update can then move the new tuple to an earlier
page. If that is not possible, then it'll insert locally (assuming
that is still possible) and HOT can then still apply.
And yes, moving tuples to earlier pages will indeed increase index
bloat, because it does create dead tuples where previously we could've
applied HOT. But we do have VACUUM and REINDEX CONCURRENTLY to clean
that up without serious long-duration stop-the-world actions, while
the other builtin cleanup methods don't.
Kind regards,
Matthias van de Meent
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 91+ messages in thread
* Re: Disabling Heap-Only Tuples
2023-09-19 10:26 Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
2023-09-19 16:09 ` Re: Disabling Heap-Only Tuples Robert Haas <[email protected]>
2023-09-19 16:30 ` Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
2023-09-19 16:56 ` Re: Disabling Heap-Only Tuples Andres Freund <[email protected]>
2023-09-19 17:33 ` Re: Disabling Heap-Only Tuples Matthias van de Meent <[email protected]>
@ 2023-09-19 18:55 ` Andres Freund <[email protected]>
0 siblings, 0 replies; 91+ messages in thread
From: Andres Freund @ 2023-09-19 18:55 UTC (permalink / raw)
To: Matthias van de Meent <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Robert Haas <[email protected]>; Laurenz Albe <[email protected]>; Thom Brown <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi,
On 2023-09-19 19:33:22 +0200, Matthias van de Meent wrote:
> On Tue, 19 Sept 2023 at 18:56, Andres Freund <[email protected]> wrote:
> >
> > Hi,
> >
> > On 2023-09-19 18:30:44 +0200, Alvaro Herrera wrote:
> > > This makes me think that maybe the logic needs to be a little more
> > > complex to avoid the problem you describe: if an UPDATE is prevented
> > > from being HOT because of this setting, but then it goes and consults
> > > FSM and it gives the update a higher block number than the tuple's
> > > current block (or it fails to give a block number at all so it is forced
> > > to extend the relation), then the update should give up on that strategy
> > > and use a HOT update after all. (I have not read the actual patch;
> > > maybe it already does this? It sounds kinda obvious.)
> >
> > Yea, a setting like what's discussed here seems, uh, not particularly useful
> > for achieving the goal of compacting tables. I don't think guiding this
> > through SQL makes a lot of sense. For decent compaction you'd want to scan the
> > table backwards, and move rows from the end to earlier, but stop once
> > everything is filled up. You can somewhat do that from SQL, but it's going to
> > be awkward and slow. I doubt you even want to use the normal UPDATE WAL
> > logging.
>
> We can't move tuples around (or, not that I know of) without using a
> transaction ID to control the visibility of the two locations of that
> tuple.
Correct, otherwise you'd end up with broken visibility in scans (seeing the
same tuple twice or never).
> Doing table compaction would thus likely require using transactions to move
> these tuples around.
Yes - but I don't think that has to be a problem. I'd expect something like
this to use multiple transactions internally. Possibly optimizing xid usage by
checking if other transactions are currently waiting on the xid and committing
if that's the case. Processing a single page should be quite fast, so the
maximum delay on other sessions is quite small.
> Using a single backend and bulk operations, it'll still lock each tuple that
> is being moved, and that can be noticed by user DML queries. I'd rather make
> the user's queries move the data around than this long-duration, locking
> background operation.
I doubt that works well enough in practice. It's very common to have tuples
that aren't updated after some point. So you then end up with needing tooling
that triggers UPDATEs for tuples at the end of the relation.
> > I think having explicit compaction support in VACUUM or somewhere similar
> > would make sense, but I don't think the proposed GUC is a useful stepping
> > stone.
>
> The point of this GUC is that the compaction can happen organically in
> the user's UPDATE workflow, so that there is no long locking operation
> going on (as you would see with VACUUM FULL / CLUSTER / pg_repack).
It certainly shouldn't use an AEL. I think we could even get away without an
SUE (it's basically just UPDATEs after all), but whether it's worth doing that
I'm not sure.
> > > > But without any kind of auto-tuning, in my opinion, it's a fairly poor
> > > > feature. Sure, some people will get use out of it, if they're
> > > > sufficiently knowledgeable and sufficiently determined. But I think
> > > > for most people in most situations, it will be a struggle.
> >
> > Indeed. I think it'd often just explode table and index sizes, because HOT
> > pruning won't be able to make usable space in pages anymore (due to dead
> > items).
>
> You seem to misunderstand the latest patch. It explicitly only blocks
> local updates if the update can then move the new tuple to an earlier
> page. If that is not possible, then it'll insert locally (assuming
> that is still possible) and HOT can then still apply.
I indeed apparently had looked at the wrong patch. But I still don't think
this is a useful way of controlling this. I guess it could be a small part of
something larger, but you are going to need something that actively updates
tuples at the end of the table, otherwise it's very unlikely in practice that
you'll ever be able to shrink the table.
Leaving aside what process "moves" tuples, I doubt that controlling "moving"
via the table size is useful. Controlling via the amount free space in the FSM
would make more sense. If there's no known free space in the FSM, this
approach can't compact. Using the table size to control also means that the
value needs to be updated with the growth of the table. Whereas controlling
moving via a percentage of free space in the FSM would allow the same setting
to be used even for a growing (or shrinking) table.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 91+ messages in thread
end of thread, other threads:[~2023-09-19 18:55 UTC | newest]
Thread overview: 91+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-02-14 04:37 [bug fix] Cascaded standby cannot start after a clean shutdown Tsunakawa, Takayuki <[email protected]>
2018-02-16 07:19 ` Michael Paquier <[email protected]>
2018-02-16 08:06 ` Michael Paquier <[email protected]>
2018-02-19 03:01 ` Tsunakawa, Takayuki <[email protected]>
2018-02-22 07:55 ` Michael Paquier <[email protected]>
2018-02-23 02:26 ` Michael Paquier <[email protected]>
2018-02-23 14:02 ` Michael Paquier <[email protected]>
2018-02-26 02:57 ` Michael Paquier <[email protected]>
2018-02-26 07:25 ` Tsunakawa, Takayuki <[email protected]>
2018-02-26 08:08 ` Michael Paquier <[email protected]>
2018-02-26 09:19 ` Michael Paquier <[email protected]>
2018-02-27 05:15 ` Tsunakawa, Takayuki <[email protected]>
2018-03-14 05:27 ` Michael Paquier <[email protected]>
2018-03-16 05:27 ` Tsunakawa, Takayuki <[email protected]>
2018-03-16 05:47 ` Michael Paquier <[email protected]>
2018-03-16 06:02 ` Tsunakawa, Takayuki <[email protected]>
2018-03-17 23:49 ` Michael Paquier <[email protected]>
2019-02-04 16:43 [PATCH v2 2/2] Propagate replica identity to partitions Alvaro Herrera <[email protected]>
2019-02-04 16:43 [PATCH 2/2] Propagate replica identity to partitions Alvaro Herrera <[email protected]>
2019-02-04 16:43 [PATCH v3 2/2] Propagate replica identity to partitions Alvaro Herrera <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v32 06/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v18 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v20 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v28 06/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v12 10/11] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v13 7/8] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v23 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v14 7/8] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v34 06/15] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v35 6/7] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v10 8/9] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v31 06/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v25 07/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v11 8/9] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v9 09/11] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v33 06/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v30 06/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v26 07/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v27 06/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v22 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v19 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v21 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v36 6/7] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v16 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v17 08/10] pg_ls_logdir to ignore error if initial/top dir is missing Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v37 06/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2020-03-06 23:23 [PATCH v24 07/11] pg_ls_logdir to ignore error if initial/top dir is missing.. Justin Pryzby <[email protected]>
2023-09-19 10:26 Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
2023-09-19 16:09 ` Re: Disabling Heap-Only Tuples Robert Haas <[email protected]>
2023-09-19 16:30 ` Re: Disabling Heap-Only Tuples Alvaro Herrera <[email protected]>
2023-09-19 16:56 ` Re: Disabling Heap-Only Tuples Andres Freund <[email protected]>
2023-09-19 17:33 ` Re: Disabling Heap-Only Tuples Matthias van de Meent <[email protected]>
2023-09-19 18:55 ` Re: Disabling Heap-Only Tuples Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox