public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 2/2] Allow composite types in bootstrap
4+ messages / 3 participants
[nested] [flat]
* [PATCH 2/2] Allow composite types in bootstrap
@ 2020-11-17 15:28 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Justin Pryzby @ 2020-11-17 15:28 UTC (permalink / raw)
---
src/backend/bootstrap/bootstrap.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 1b940d9d27..a0fcbb3d83 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -916,6 +916,7 @@ gettype(char *type)
{
if (Typ != NIL)
{
+ static bool did_reread PG_USED_FOR_ASSERTS_ONLY = false; /* Already reread pg_types? */
ListCell *lc;
foreach (lc, Typ)
@@ -927,6 +928,33 @@ gettype(char *type)
return app->am_oid;
}
}
+
+ /*
+ * The type wasn't known; check again to handle composite
+ * types, added since first populating Typ.
+ */
+
+ /*
+ * Once all the types are populated and we handled composite
+ * types, shouldn't need to do that again.
+ */
+ Assert(!did_reread);
+ did_reread = true;
+
+ list_free_deep(Typ);
+ Typ = NIL;
+ populate_typ();
+
+ /* Need to avoid infinite recursion... */
+ foreach (lc, Typ)
+ {
+ struct typmap *app = lfirst(lc);
+ if (strncmp(NameStr(app->am_typ.typname), type, NAMEDATALEN) == 0)
+ {
+ Ap = app;
+ return app->am_oid;
+ }
+ }
}
else
{
--
2.17.0
--yQbNiKLmgenwUfTN--
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..)
@ 2023-04-11 23:18 Thomas Munro <[email protected]>
2023-04-11 23:37 ` Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..) Justin Pryzby <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Thomas Munro @ 2023-04-11 23:18 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: Thomas Munro <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>
On Wed, Apr 12, 2023 at 7:46 AM Justin Pryzby <[email protected]> wrote:
> Unfortunately:
> (gdb) p area->control->handle
> $3 = 0
> (gdb) p segment_map->header->magic
> value has been optimized out
> (gdb) p index
> $4 = <value optimized out>
Hmm, well index I can find from parameters:
> #2 0x0000000000991470 in ExceptionalCondition (conditionName=<value optimized out>, errorType=<value optimized out>, fileName=<value optimized out>, lineNumber=1770) at assert.c:69
> #3 0x00000000009b9f97 in get_segment_by_index (area=0x22818c0, index=<value optimized out>) at dsa.c:1769
> #4 0x00000000009ba192 in dsa_get_address (area=0x22818c0, dp=1099511703168) at dsa.c:953
We have dp=1099511703168 == 0x10000012680, so index == 1 and the rest
is the offset into that segment. It's not the initial segment in the
main shared memory area created by the postmaster with
dsa_create_in_place() (that'd be index 0), it's in an extra segment
that was created with shm_open(). We managed to open and mmap() that
segment, but it contains unexpected garbage.
Can you print *area->control? And then can you see that the DSM
handle is in index 1 in "segment_handles" in there? Then can you see
if your system has a file with that number in its name under
/dev/shm/, and can you tell me what "od -c /dev/shm/..." shows as the
first few lines of stuff at the top, so we can see what that
unexpected garbage looks like?
Side rant: I don't think there's any particular indication that it's
the issue here, but while it's on my mind: I really wish we didn't
use random numbers for DSM handles. I understand where it came from:
the need to manage SysV shmem keyspace (a DSM mode that almost nobody
uses, but whose limitations apply to all modes). We've debugged
issues relating to handle collisions before, causing unrelated DSM
segments to be confused, back when the random seed was not different
in each backend making collisions likely. For every other mode, we
could instead use something like (slot, generation) to keep collisions
as far apart as possible (generation wraparound), and avoid collisions
between unrelated clusters by using the pgdata path as a shm_open()
prefix. Another idea is to add a new DSM mode that would use memfd
and similar things and pass fds between backends, so that the segments
are entirely anonymous and don't need to be cleaned up after a crash
(I thought about that while studying the reasons why PostgreSQL can't
run on Capsicum (a capabilities research project) or Android (a
telephone), both of which banned SysV *and* POSIX shm because
system-global namespaces are bad).
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..)
2023-04-11 23:18 Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..) Thomas Munro <[email protected]>
@ 2023-04-11 23:37 ` Justin Pryzby <[email protected]>
2023-04-11 23:49 ` Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..) Thomas Munro <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Justin Pryzby @ 2023-04-11 23:37 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; +Cc: Thomas Munro <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>
On Wed, Apr 12, 2023 at 11:18:36AM +1200, Thomas Munro wrote:
> Can you print *area->control?
(gdb) p *area->control
$1 = {segment_header = {magic = 216163848, usable_pages = 62, size = 1048576, prev = 1, next = 18446744073709551615, bin = 4, freed = false}, handle = 0, segment_handles = {0, 3696856876, 433426374, 1403332952, 2754923922,
0 <repeats 1019 times>}, segment_bins = {18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 3, 4, 18446744073709551615, 18446744073709551615, 18446744073709551615,
18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615, 18446744073709551615}, pools = {{lock = {tranche = 72, state = {value = 536870912},
waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 2199025295360, 8192, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0,
2199025296088, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {
head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72,
state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0,
0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 2199025296648, 2199025298608, 0}}, {lock = {tranche = 72, state = {value = 536870912},
waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {
tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647,
tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {
value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0,
0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 2199025298496, 2199025298664, 2199025297936}}, {lock = {tranche = 72, state = {
value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0,
0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 8416, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647,
tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {
value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0,
0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647,
tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 8304, 0, 0}}, {lock = {tranche = 72, state = {
value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0,
0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647,
tail = 2147483647}}, spans = {0, 8528, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {
value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 8248, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0,
0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 8584, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {
head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72,
state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0, 0, 0, 0}}, {lock = {tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}, spans = {0,
8640, 0, 0}}}, total_segment_size = 9699328, max_total_segment_size = 18446744073709551615, high_segment_index = 4, refcnt = 8455469, pinned = true, freed_segment_counter = 0, lwlock_tranche_id = 72, lock = {
tranche = 72, state = {value = 536870912}, waiters = {head = 2147483647, tail = 2147483647}}}
> And then can you see that the DSM handle is in index 1 in "segment_handles"
> in there?
(gdb) p area->control->segment_handles
$2 = {0, 3696856876, 433426374, 1403332952, 2754923922, 0 <repeats 1019 times>}
> Then can you see if your system has a file with that number in its name under
> /dev/shm/,
$ ls /dev/shm/ |grep 3696856876 || echo not found
not found
(In case it matters: the vm has been up for 1558 days).
If it's helpful, I could provide the corefile, unstripped binaries, and
libc.so, which would be enough to use gdb on your side with "set
solib-search-path".
--
Justin
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..)
2023-04-11 23:18 Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..) Thomas Munro <[email protected]>
2023-04-11 23:37 ` Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..) Justin Pryzby <[email protected]>
@ 2023-04-11 23:49 ` Thomas Munro <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Thomas Munro @ 2023-04-11 23:49 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: pgsql-hackers; Andres Freund <[email protected]>
On Wed, Apr 12, 2023 at 11:37 AM Justin Pryzby <[email protected]> wrote:
> $ ls /dev/shm/ |grep 3696856876 || echo not found
> not found
Oh, of course it would have restarted after it crashed and unlinked
that... So the remaining traces of that memory *might* be in the core
file, depending (IIRC) on the core filter settings (you definitely get
shared anonymous memory like our main shm region by default, but IIRC
there's something extra needed if you want the shm_open'd DSM segments
to be dumped too...)
> (In case it matters: the vm has been up for 1558 days).
I will refrain from invoking cosmic radiation at this point :-)
> If it's helpful, I could provide the corefile, unstripped binaries, and
> libc.so, which would be enough to use gdb on your side with "set
> solib-search-path".
Sounds good, thanks, please send them over off-list and I'll see if I
can figure anything out ...
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2023-04-11 23:49 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-11-17 15:28 [PATCH 2/2] Allow composite types in bootstrap Justin Pryzby <[email protected]>
2023-04-11 23:18 Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..) Thomas Munro <[email protected]>
2023-04-11 23:37 ` Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..) Justin Pryzby <[email protected]>
2023-04-11 23:49 ` Re: v15b1: FailedAssertion("segment_map->header->magic == (DSA_SEGMENT_HEADER_MAGIC ^ area->control->handle ^ index)", File: "dsa.c", ..) Thomas Munro <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox