agora inbox for [email protected]help / color / mirror / Atom feed
Building 64-bit postgres with GSS support on windows 8+ messages / 3 participants [nested] [flat]
* Building 64-bit postgres with GSS support on windows @ 2018-10-17 07:47 Victor Wagner <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Victor Wagner @ 2018-10-17 07:47 UTC (permalink / raw) To: pgsql-hackers Colleagues, I've encountered some problems trying to enable gss support in MSVC build of Postgres (I've experemented with REL_10_STABLE branch, but code in question seems to be same in all supported releases, including master). As it is recommended in the documentation, I've downloaded MIT Kerberos from http://web.mit.edu/Kerberos/dist/index.html They distribute latest version (4.1) of Kerberos as .msi installer which installs into C:\Program Files. (32-bit version probably would install into C:\Program Files(x86), but I've not tried it yet). Here comes first problem. Project->AddLibrary unconditionally quotes paths with spaces. But some contrib modules which depends on PL language, use Mkvcbuild::AddTransformModule for generating dependences. It gets list of (already quoted) librariy name from dependency project and adds them to the current project. I haven't investigate whether adding kerberos, xml, icu etc libraries three times to linker command line does any good, but it works. And double quoting filename with quotes definitely breaks things. Fix is quite simple: diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 9817b94..3749c17 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -126,7 +126,7 @@ sub AddLibrary { my ($self, $lib, $dbgsuffix) = @_; - if ($lib =~ m/\s/) + if ($lib =~ m/\s/ && !$lib =~m/^\".*\"$/) { $lib = '"' . $lib . """; } I suppose this would help with any 3-rd party library installed into directory with spaces in the names, not just Kerberos. Second problem is that names of 32-bit libraries and library directories are hard-coded into Solution.pm $self->{options}->{gss} . '\lib\i386\krb5_32.lib' And for 64-bit build '\lib\amd64\krb5_64.lib' is needed (at least for this MIT Kerberos 4.1) There is similar platform differences with other libraries, such as ICU, and check if ($self->{platform} eq 'Win32') is used for them. But there is no such thing for gss libraries. And third problem is that Solution.pm expect includes in $self->{options}->{gss}.'\inc\krb5' but for Kerberos 4.1 $self->{options}->{gss}.'\include" is needed. I haven't dig into computer archeology and havent checked when this change occured, just check if directore '\include' exist and use it if so. Attached patch fixes these problems. Regrards, Victor. -- Attachments: [text/x-patch] gsswin.patch (1.5K, ../../[email protected]/2-gsswin.patch) download | inline diff: diff --git a/src/tools/msvc/Project.pm b/src/tools/msvc/Project.pm index 9817b94..3749c17 100644 --- a/src/tools/msvc/Project.pm +++ b/src/tools/msvc/Project.pm @@ -126,7 +126,7 @@ sub AddLibrary { my ($self, $lib, $dbgsuffix) = @_; - if ($lib =~ m/\s/) + if ($lib =~ m/\s/ && !$lib =~/^\".*\"$/) { $lib = '"' . $lib . """; } diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index ad75478..2adaafd 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -591,10 +591,22 @@ sub AddProject } if ($self->{options}->{gss}) { - $proj->AddIncludeDir($self->{options}->{gss} . '\inc\krb5'); - $proj->AddLibrary($self->{options}->{gss} . '\lib\i386\krb5_32.lib'); - $proj->AddLibrary($self->{options}->{gss} . '\lib\i386\comerr32.lib'); - $proj->AddLibrary($self->{options}->{gss} . '\lib\i386\gssapi32.lib'); + if ( -d $self->{options}->{gss}.'\include') { + $proj->AddIncludeDir($self->{options}->{gss} . '\include'); + } else { + $proj->AddIncludeDir($self->{options}->{gss} . '\inc\krb5'); + } + my ($gss_libdir,$suffix); + if ($self->{platform} eq 'Win32') { + $gss_libdir = $self->{options}->{gss}.'\lib\i386'; + $suffix='32'; + } else { + $gss_libdir = $self->{options}->{gss}.'\lib\amd64'; + $suffix='64'; + } + $proj->AddLibrary("$gss_libdir\\krb5_$suffix.lib"); + $proj->AddLibrary("$gss_libdir\\comerr$suffix.lib"); + $proj->AddLibrary("$gss_libdir\\gssapi$suffix.lib"); } if ($self->{options}->{iconv}) { ^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: speed up a logical replica setup @ 2022-02-24 07:05 Amit Kapila <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Amit Kapila @ 2022-02-24 07:05 UTC (permalink / raw) To: Euler Taveira <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On Mon, Feb 21, 2022 at 5:41 PM Euler Taveira <[email protected]> wrote: > > Logical replication has been used to migration with minimal downtime. However, > if you are dealing with a big database, the amount of required resources (disk > -- due to WAL retention) increases as the backlog (WAL) increases. Unless you > have a generous amount of resources and can wait for long period of time until > the new replica catches up, creating a logical replica is impracticable on > large databases. > > The general idea is to create and convert a physical replica or a base backup > (archived WAL files available) into a logical replica. The initial data copy > and catchup tends to be faster on a physical replica. This technique has been > successfully used in pglogical_create_subscriber [1]. > Sounds like a promising idea. > A new tool called pg_subscriber does this conversion and is tightly integrated > with Postgres. > > DESIGN > > The conversion requires 8 steps. > > 1. Check if the target data directory has the same system identifier than the > source data directory. > 2. Stop the target server if it is running as a standby server. (Modify > recovery parameters requires a restart.) > 3. Create one replication slot per specified database on the source server. One > additional replication slot is created at the end to get the consistent LSN > (This consistent LSN will be used as (a) a stopping point for the recovery > process and (b) a starting point for the subscriptions). > What is the need to create an extra slot other than the slot for each database? Can't we use the largest LSN returned by slots as the recovery-target-lsn and starting point for subscriptions? How, these additional slots will get freed or reused when say the server has crashed/stopped after creating the slots but before creating the subscriptions? Users won't even know the names of such slots as they are internally created. > 4. Write recovery parameters into the target data directory and start the > target server (Wait until the target server is promoted). > 5. Create one publication (FOR ALL TABLES) per specified database on the source > server. > 6. Create one subscription per specified database on the target server (Use > replication slot and publication created in a previous step. Don't enable the > subscriptions yet). > 7. Sets the replication progress to the consistent LSN that was got in a > previous step. > 8. Enable the subscription for each specified database on the target server. > > This tool does not take a base backup. It can certainly be included later. > There is already a tool do it: pg_basebackup. > The backup will take the backup of all the databases present on the source server. Do we need to provide the way/recommendation to remove the databases that are not required? Can we see some numbers with various sizes of databases (cluster) to see how it impacts the time for small to large size databases as compared to the traditional method? This might help giving users advice on when to use this tool? -- With Regards, Amit Kapila. ^ permalink raw reply [nested|flat] 8+ messages in thread
* [PATCH v3 1/3] pg_lfind32: process "tail" with SIMD intructions @ 2024-03-15 17:26 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Nathan Bossart @ 2024-03-15 17:26 UTC (permalink / raw) --- src/include/port/pg_lfind.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/include/port/pg_lfind.h b/src/include/port/pg_lfind.h index b8dfa66eef..9d21284724 100644 --- a/src/include/port/pg_lfind.h +++ b/src/include/port/pg_lfind.h @@ -103,7 +103,7 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) const uint32 nelem_per_iteration = 4 * nelem_per_vector; /* round down to multiple of elements per iteration */ - const uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); + uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); #if defined(USE_ASSERT_CHECKING) bool assert_result = false; @@ -117,9 +117,11 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) break; } } + i = 0; #endif - for (i = 0; i < tail_idx; i += nelem_per_iteration) +retry: + for (; i < tail_idx; i += nelem_per_iteration) { Vector32 vals1, vals2, @@ -157,6 +159,16 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) return true; } } + + if (i == nelem) + return false; + else if (tail_idx > 0) + { + tail_idx = nelem; + i = nelem - nelem_per_iteration; + goto retry; + } + #endif /* ! USE_NO_SIMD */ /* Process the remaining elements one at a time. */ -- 2.25.1 --gBBFr7Ir9EOA20Yy Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0002-add-avx2-support-in-simd.h.patch" ^ permalink raw reply [nested|flat] 8+ messages in thread
* [PATCH v3 1/3] pg_lfind32: process "tail" with SIMD intructions @ 2024-03-15 17:26 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Nathan Bossart @ 2024-03-15 17:26 UTC (permalink / raw) --- src/include/port/pg_lfind.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/include/port/pg_lfind.h b/src/include/port/pg_lfind.h index b8dfa66eef..9d21284724 100644 --- a/src/include/port/pg_lfind.h +++ b/src/include/port/pg_lfind.h @@ -103,7 +103,7 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) const uint32 nelem_per_iteration = 4 * nelem_per_vector; /* round down to multiple of elements per iteration */ - const uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); + uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); #if defined(USE_ASSERT_CHECKING) bool assert_result = false; @@ -117,9 +117,11 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) break; } } + i = 0; #endif - for (i = 0; i < tail_idx; i += nelem_per_iteration) +retry: + for (; i < tail_idx; i += nelem_per_iteration) { Vector32 vals1, vals2, @@ -157,6 +159,16 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) return true; } } + + if (i == nelem) + return false; + else if (tail_idx > 0) + { + tail_idx = nelem; + i = nelem - nelem_per_iteration; + goto retry; + } + #endif /* ! USE_NO_SIMD */ /* Process the remaining elements one at a time. */ -- 2.25.1 --gBBFr7Ir9EOA20Yy Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0002-add-avx2-support-in-simd.h.patch" ^ permalink raw reply [nested|flat] 8+ messages in thread
* [PATCH v3 1/3] pg_lfind32: process "tail" with SIMD intructions @ 2024-03-15 17:26 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Nathan Bossart @ 2024-03-15 17:26 UTC (permalink / raw) --- src/include/port/pg_lfind.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/include/port/pg_lfind.h b/src/include/port/pg_lfind.h index b8dfa66eef..9d21284724 100644 --- a/src/include/port/pg_lfind.h +++ b/src/include/port/pg_lfind.h @@ -103,7 +103,7 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) const uint32 nelem_per_iteration = 4 * nelem_per_vector; /* round down to multiple of elements per iteration */ - const uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); + uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); #if defined(USE_ASSERT_CHECKING) bool assert_result = false; @@ -117,9 +117,11 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) break; } } + i = 0; #endif - for (i = 0; i < tail_idx; i += nelem_per_iteration) +retry: + for (; i < tail_idx; i += nelem_per_iteration) { Vector32 vals1, vals2, @@ -157,6 +159,16 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) return true; } } + + if (i == nelem) + return false; + else if (tail_idx > 0) + { + tail_idx = nelem; + i = nelem - nelem_per_iteration; + goto retry; + } + #endif /* ! USE_NO_SIMD */ /* Process the remaining elements one at a time. */ -- 2.25.1 --gBBFr7Ir9EOA20Yy Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v3-0002-add-avx2-support-in-simd.h.patch" ^ permalink raw reply [nested|flat] 8+ messages in thread
* [PATCH v2 1/3] pg_lfind32: process "tail" with SIMD intructions @ 2024-03-15 17:26 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Nathan Bossart @ 2024-03-15 17:26 UTC (permalink / raw) --- src/include/port/pg_lfind.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/include/port/pg_lfind.h b/src/include/port/pg_lfind.h index b8dfa66eef..9d21284724 100644 --- a/src/include/port/pg_lfind.h +++ b/src/include/port/pg_lfind.h @@ -103,7 +103,7 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) const uint32 nelem_per_iteration = 4 * nelem_per_vector; /* round down to multiple of elements per iteration */ - const uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); + uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); #if defined(USE_ASSERT_CHECKING) bool assert_result = false; @@ -117,9 +117,11 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) break; } } + i = 0; #endif - for (i = 0; i < tail_idx; i += nelem_per_iteration) +retry: + for (; i < tail_idx; i += nelem_per_iteration) { Vector32 vals1, vals2, @@ -157,6 +159,16 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) return true; } } + + if (i == nelem) + return false; + else if (tail_idx > 0) + { + tail_idx = nelem; + i = nelem - nelem_per_iteration; + goto retry; + } + #endif /* ! USE_NO_SIMD */ /* Process the remaining elements one at a time. */ -- 2.25.1 --6c2NcOVqGQ03X4Wi Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0002-add-avx2-support-in-simd.h.patch" ^ permalink raw reply [nested|flat] 8+ messages in thread
* [PATCH v2 1/3] pg_lfind32: process "tail" with SIMD intructions @ 2024-03-15 17:26 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Nathan Bossart @ 2024-03-15 17:26 UTC (permalink / raw) --- src/include/port/pg_lfind.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/include/port/pg_lfind.h b/src/include/port/pg_lfind.h index b8dfa66eef..9d21284724 100644 --- a/src/include/port/pg_lfind.h +++ b/src/include/port/pg_lfind.h @@ -103,7 +103,7 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) const uint32 nelem_per_iteration = 4 * nelem_per_vector; /* round down to multiple of elements per iteration */ - const uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); + uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); #if defined(USE_ASSERT_CHECKING) bool assert_result = false; @@ -117,9 +117,11 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) break; } } + i = 0; #endif - for (i = 0; i < tail_idx; i += nelem_per_iteration) +retry: + for (; i < tail_idx; i += nelem_per_iteration) { Vector32 vals1, vals2, @@ -157,6 +159,16 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) return true; } } + + if (i == nelem) + return false; + else if (tail_idx > 0) + { + tail_idx = nelem; + i = nelem - nelem_per_iteration; + goto retry; + } + #endif /* ! USE_NO_SIMD */ /* Process the remaining elements one at a time. */ -- 2.25.1 --6c2NcOVqGQ03X4Wi Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0002-add-avx2-support-in-simd.h.patch" ^ permalink raw reply [nested|flat] 8+ messages in thread
* [PATCH v2 1/3] pg_lfind32: process "tail" with SIMD intructions @ 2024-03-15 17:26 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 8+ messages in thread From: Nathan Bossart @ 2024-03-15 17:26 UTC (permalink / raw) --- src/include/port/pg_lfind.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/include/port/pg_lfind.h b/src/include/port/pg_lfind.h index b8dfa66eef..9d21284724 100644 --- a/src/include/port/pg_lfind.h +++ b/src/include/port/pg_lfind.h @@ -103,7 +103,7 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) const uint32 nelem_per_iteration = 4 * nelem_per_vector; /* round down to multiple of elements per iteration */ - const uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); + uint32 tail_idx = nelem & ~(nelem_per_iteration - 1); #if defined(USE_ASSERT_CHECKING) bool assert_result = false; @@ -117,9 +117,11 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) break; } } + i = 0; #endif - for (i = 0; i < tail_idx; i += nelem_per_iteration) +retry: + for (; i < tail_idx; i += nelem_per_iteration) { Vector32 vals1, vals2, @@ -157,6 +159,16 @@ pg_lfind32(uint32 key, uint32 *base, uint32 nelem) return true; } } + + if (i == nelem) + return false; + else if (tail_idx > 0) + { + tail_idx = nelem; + i = nelem - nelem_per_iteration; + goto retry; + } + #endif /* ! USE_NO_SIMD */ /* Process the remaining elements one at a time. */ -- 2.25.1 --6c2NcOVqGQ03X4Wi Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v2-0002-add-avx2-support-in-simd.h.patch" ^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2024-03-15 17:26 UTC | newest] Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2018-10-17 07:47 Building 64-bit postgres with GSS support on windows Victor Wagner <[email protected]> 2022-02-24 07:05 Re: speed up a logical replica setup Amit Kapila <[email protected]> 2024-03-15 17:26 [PATCH v3 1/3] pg_lfind32: process "tail" with SIMD intructions Nathan Bossart <[email protected]> 2024-03-15 17:26 [PATCH v3 1/3] pg_lfind32: process "tail" with SIMD intructions Nathan Bossart <[email protected]> 2024-03-15 17:26 [PATCH v3 1/3] pg_lfind32: process "tail" with SIMD intructions Nathan Bossart <[email protected]> 2024-03-15 17:26 [PATCH v2 1/3] pg_lfind32: process "tail" with SIMD intructions Nathan Bossart <[email protected]> 2024-03-15 17:26 [PATCH v2 1/3] pg_lfind32: process "tail" with SIMD intructions Nathan Bossart <[email protected]> 2024-03-15 17:26 [PATCH v2 1/3] pg_lfind32: process "tail" with SIMD intructions Nathan Bossart <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox