agora inbox for [email protected]
help / color / mirror / Atom feed[PATCH 3/8] Optimize allocations in bringetbitmap
8+ messages / 5 participants
[nested] [flat]
* [PATCH 3/8] Optimize allocations in bringetbitmap
@ 2020-09-13 10:12 Tomas Vondra <[email protected]>
0 siblings, 0 replies; 8+ messages in thread
From: Tomas Vondra @ 2020-09-13 10:12 UTC (permalink / raw)
The bringetbitmap function allocates memory for various purposes, which
may be quite expensive, depending on the number of scan keys. Instead of
allocating them separately, allocate one bit chunk of memory an carve it
into smaller pieces as needed - all the pieces have the same lifespan,
and it saves quite a bit of CPU and memory overhead.
Author: Tomas Vondra <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/backend/access/brin/brin.c | 62 +++++++++++++++++++++++++++-------
1 file changed, 49 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 14da9ed17f..3735c41788 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -373,6 +373,9 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
int *nkeys,
*nnullkeys;
int keyno;
+ char *ptr;
+ Size len;
+ char *tmp PG_USED_FOR_ASSERTS_ONLY;
opaque = (BrinOpaque *) scan->opaque;
bdesc = opaque->bo_bdesc;
@@ -398,11 +401,50 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
* Make room for per-attribute lists of scan keys that we'll pass to the
* consistent support procedure. We keep null and regular keys separate,
* so that we can easily pass regular keys to the consistent function.
+ *
+ * To reduce the allocation overhead, we allocate one big chunk and then
+ * carve it into smaller arrays ourselves. All the pieces have exactly
+ * the same lifetime, so that's OK.
*/
- keys = palloc0(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts);
- nullkeys = palloc0(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts);
- nkeys = palloc0(sizeof(int) * bdesc->bd_tupdesc->natts);
- nnullkeys = palloc0(sizeof(int) * bdesc->bd_tupdesc->natts);
+ len =
+ /* regular keys */
+ MAXALIGN(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts) +
+ MAXALIGN(sizeof(ScanKey) * scan->numberOfKeys * bdesc->bd_tupdesc->natts) +
+ MAXALIGN(sizeof(int) * bdesc->bd_tupdesc->natts) +
+ /* NULL keys */
+ MAXALIGN(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts) +
+ MAXALIGN(sizeof(ScanKey) * scan->numberOfKeys * bdesc->bd_tupdesc->natts) +
+ MAXALIGN(sizeof(int) * bdesc->bd_tupdesc->natts);
+
+ ptr = palloc(len);
+ tmp = ptr;
+
+ keys = (ScanKey **) ptr;
+ ptr += MAXALIGN(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts);
+
+ nullkeys = (ScanKey **) ptr;
+ ptr += MAXALIGN(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts);
+
+ nkeys = (int *) ptr;
+ ptr += MAXALIGN(sizeof(int) * bdesc->bd_tupdesc->natts);
+
+ nnullkeys = (int *) ptr;
+ ptr += MAXALIGN(sizeof(int) * bdesc->bd_tupdesc->natts);
+
+ for (int i = 0; i < bdesc->bd_tupdesc->natts; i++)
+ {
+ keys[i] = (ScanKey *) ptr;
+ ptr += MAXALIGN(sizeof(ScanKey) * scan->numberOfKeys);
+
+ nullkeys[i] = (ScanKey *) ptr;
+ ptr += MAXALIGN(sizeof(ScanKey) * scan->numberOfKeys);
+ }
+
+ Assert(tmp + len == ptr);
+
+ /* zero the number of keys */
+ memset(nkeys, 0, sizeof(int) * bdesc->bd_tupdesc->natts);
+ memset(nnullkeys, 0, sizeof(int) * bdesc->bd_tupdesc->natts);
/*
* Preprocess the scan keys - split them into per-attribute arrays.
@@ -438,9 +480,9 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
{
FmgrInfo *tmp;
- /* No key/null arrays for this attribute. */
- Assert((keys[keyattno - 1] == NULL) && (nkeys[keyattno - 1] == 0));
- Assert((nullkeys[keyattno - 1] == NULL) && (nnullkeys[keyattno - 1] == 0));
+ /* First time we see this attribute, so no key/null keys. */
+ Assert(nkeys[keyattno - 1] == 0);
+ Assert(nnullkeys[keyattno - 1] == 0);
tmp = index_getprocinfo(idxRel, keyattno,
BRIN_PROCNUM_CONSISTENT);
@@ -451,17 +493,11 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
/* Add key to the proper per-attribute array. */
if (key->sk_flags & SK_ISNULL)
{
- if (!nullkeys[keyattno - 1])
- nullkeys[keyattno - 1] = palloc0(sizeof(ScanKey) * scan->numberOfKeys);
-
nullkeys[keyattno - 1][nnullkeys[keyattno - 1]] = key;
nnullkeys[keyattno - 1]++;
}
else
{
- if (!keys[keyattno - 1])
- keys[keyattno - 1] = palloc0(sizeof(ScanKey) * scan->numberOfKeys);
-
keys[keyattno - 1][nkeys[keyattno - 1]] = key;
nkeys[keyattno - 1]++;
}
--
2.26.2
--------------FA974B75A0C3E9CA18CE7207
Content-Type: text/x-patch; charset=UTF-8;
name="0004-BRIN-bloom-indexes-20210122.patch"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment;
filename="0004-BRIN-bloom-indexes-20210122.patch"
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: plperl version on the meson setup summary screen
@ 2024-11-29 08:26 Nazir Bilal Yavuz <[email protected]>
2024-12-04 16:10 ` Re: plperl version on the meson setup summary screen Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 8+ messages in thread
From: Nazir Bilal Yavuz @ 2024-11-29 08:26 UTC (permalink / raw)
To: Zharkov Roman <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; [email protected]
Hi,
On Fri, 29 Nov 2024 at 08:40, Zharkov Roman <[email protected]> wrote:
>
> Hello,
>
> On 2024-11-27 21:50, Andrew Dunstan wrote:
> > it should give the same answer
>
> Sometimes "version" and "api_versionstring" are different. Here are two
> simple examples from my windows system and from a linux system of one of
> my colleagues:
>
> C:\Temp>perl -MConfig -e "print \"$Config{api_versionstring}\n\"; print
> \"$Config{version}\n\""
> 5.32.0
> 5.32.1
> C:\Temp>perl -v
> This is perl 5, version 32, subversion 1 (v5.32.1) built for
> MSWin32-x64-multi-thread
>
> perl -MConfig -e 'print "$Config{api_versionstring}\n"; print
> "$Config{version}\n"'
> 5.38.0
> 5.38.2
> perl -v
> This is perl 5, version 38, subversion 2 (v5.38.2)
It is different for me too:
$ perl -MConfig -e 'print "$Config{api_versionstring}\n";
print"$Config{version}\n"'
5.38.0
5.38.2
$ perl -v
This is perl 5, version 38, subversion 2 (v5.38.2)
On Wed, 27 Nov 2024 at 17:51, Andrew Dunstan <[email protected]> wrote:
> On 2024-10-17 Th 10:02 PM, Zharkov Roman wrote:
> > With the attached patch we can see the "perlversion" in the summary
> > information table. But without a beautiful console colorize.
> >
> Yeah, the lack of version number has mildly annoyed me too, so let's fix
> it. I haven't found the right secret sauce to make the version number
> appear colorized, either. Maybe some meson guru can tell us how.
It seems that we can force ANSI colors:
- 'plperl': [perl_dep, perlversion],
+ 'plperl': [perl_dep, '\033[1;36m@0@\033[0m'.format(perlversion)],
But I think this is too hacky. I am not sure if that is the best way
or worth it.
--
Regards,
Nazir Bilal Yavuz
Microsoft
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: plperl version on the meson setup summary screen
2024-11-29 08:26 Re: plperl version on the meson setup summary screen Nazir Bilal Yavuz <[email protected]>
@ 2024-12-04 16:10 ` Andrew Dunstan <[email protected]>
2025-02-04 06:37 ` Re: plperl version on the meson setup summary screen Zharkov Roman <[email protected]>
0 siblings, 1 reply; 8+ messages in thread
From: Andrew Dunstan @ 2024-12-04 16:10 UTC (permalink / raw)
To: Nazir Bilal Yavuz <[email protected]>; Zharkov Roman <[email protected]>; +Cc: [email protected]
On 2024-11-29 Fr 3:26 AM, Nazir Bilal Yavuz wrote:
> Hi,
>
> On Fri, 29 Nov 2024 at 08:40, Zharkov Roman <[email protected]> wrote:
>> Hello,
>>
>> On 2024-11-27 21:50, Andrew Dunstan wrote:
>>> it should give the same answer
>> Sometimes "version" and "api_versionstring" are different. Here are two
>> simple examples from my windows system and from a linux system of one of
>> my colleagues:
>>
>> C:\Temp>perl -MConfig -e "print \"$Config{api_versionstring}\n\"; print
>> \"$Config{version}\n\""
>> 5.32.0
>> 5.32.1
>> C:\Temp>perl -v
>> This is perl 5, version 32, subversion 1 (v5.32.1) built for
>> MSWin32-x64-multi-thread
>>
>> perl -MConfig -e 'print "$Config{api_versionstring}\n"; print
>> "$Config{version}\n"'
>> 5.38.0
>> 5.38.2
>> perl -v
>> This is perl 5, version 38, subversion 2 (v5.38.2)
> It is different for me too:
>
> $ perl -MConfig -e 'print "$Config{api_versionstring}\n";
> print"$Config{version}\n"'
> 5.38.0
> 5.38.2
> $ perl -v
> This is perl 5, version 38, subversion 2 (v5.38.2)
>
> On Wed, 27 Nov 2024 at 17:51, Andrew Dunstan <[email protected]> wrote:
>> On 2024-10-17 Th 10:02 PM, Zharkov Roman wrote:
>>> With the attached patch we can see the "perlversion" in the summary
>>> information table. But without a beautiful console colorize.
>>>
>> Yeah, the lack of version number has mildly annoyed me too, so let's fix
>> it. I haven't found the right secret sauce to make the version number
>> appear colorized, either. Maybe some meson guru can tell us how.
> It seems that we can force ANSI colors:
>
> - 'plperl': [perl_dep, perlversion],
> + 'plperl': [perl_dep, '\033[1;36m@0@\033[0m'.format(perlversion)],
>
> But I think this is too hacky. I am not sure if that is the best way
> or worth it.
>
Yes, way too hacky. If we can't find a better way I'm good with Roman's
idea in principle.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: plperl version on the meson setup summary screen
2024-11-29 08:26 Re: plperl version on the meson setup summary screen Nazir Bilal Yavuz <[email protected]>
2024-12-04 16:10 ` Re: plperl version on the meson setup summary screen Andrew Dunstan <[email protected]>
@ 2025-02-04 06:37 ` Zharkov Roman <[email protected]>
2025-03-13 14:34 ` Re: plperl version on the meson setup summary screen vignesh C <[email protected]>
2025-03-14 06:08 ` Re: plperl version on the meson setup summary screen vignesh C <[email protected]>
0 siblings, 2 replies; 8+ messages in thread
From: Zharkov Roman @ 2025-02-04 06:37 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Nazir Bilal Yavuz <[email protected]>; [email protected]
Hello,
Here is a new patch version.
I tried to use perl 'version' instead of 'api_versionstring' to sync
with configure script.
--
Roman Zharkov
Attachments:
[text/x-diff] v2-0001-Show-plperl-version-in-the-meson-setup.patch (1.6K, ../../[email protected]/2-v2-0001-Show-plperl-version-in-the-meson-setup.patch)
download | inline diff:
From: Zharkov Roman <[email protected]>
Date: Thu, 17 Oct 2024 13:09:51 +0700
Subject: [PATCH] Show plperl version in the meson setup summary.
Use perl 'version' instead of 'api_versionstring' to sync with configure script.
---
meson.build | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/meson.build b/meson.build
index 8e128f4982a..436df7bb592 100644
--- a/meson.build
+++ b/meson.build
@@ -996,6 +996,7 @@ endif
perlopt = get_option('plperl')
perl_dep = not_found_dep
+perlversion = ''
if not perlopt.disabled()
perl_may_work = true
@@ -1013,7 +1014,7 @@ if not perlopt.disabled()
# Then inquire perl about its configuration
if perl_may_work
perl_conf_cmd = [perl, '-MConfig', '-e', 'print $Config{$ARGV[0]}']
- perlversion = run_command(perl_conf_cmd, 'api_versionstring', check: true).stdout()
+ perlversion = run_command(perl_conf_cmd, 'version', check: true).stdout()
archlibexp = run_command(perl_conf_cmd, 'archlibexp', check: true).stdout()
privlibexp = run_command(perl_conf_cmd, 'privlibexp', check: true).stdout()
useshrplib = run_command(perl_conf_cmd, 'useshrplib', check: true).stdout()
@@ -3711,7 +3712,7 @@ if meson.version().version_compare('>=0.57')
'nls': libintl,
'openssl': ssl,
'pam': pam,
- 'plperl': perl_dep,
+ 'plperl': [perl_dep, perlversion],
'plpython': python3_dep,
'pltcl': tcl_dep,
'readline': readline,
@@ -3722,6 +3723,7 @@ if meson.version().version_compare('>=0.57')
'zstd': zstd,
},
section: 'External libraries',
+ list_sep: ' ',
)
endif
--
2.34.0.windows.1
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: plperl version on the meson setup summary screen
2024-11-29 08:26 Re: plperl version on the meson setup summary screen Nazir Bilal Yavuz <[email protected]>
2024-12-04 16:10 ` Re: plperl version on the meson setup summary screen Andrew Dunstan <[email protected]>
2025-02-04 06:37 ` Re: plperl version on the meson setup summary screen Zharkov Roman <[email protected]>
@ 2025-03-13 14:34 ` vignesh C <[email protected]>
1 sibling, 0 replies; 8+ messages in thread
From: vignesh C @ 2025-03-13 14:34 UTC (permalink / raw)
To: Zharkov Roman <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Nazir Bilal Yavuz <[email protected]>; [email protected]
On Tue, 4 Feb 2025 at 12:07, Zharkov Roman <[email protected]> wrote:
>
> Hello,
>
> Here is a new patch version.
> I tried to use perl 'version' instead of 'api_versionstring' to sync
> with configure script.
Thanks for the patch, this looks good to me.
Regards,
Vignesh
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: plperl version on the meson setup summary screen
2024-11-29 08:26 Re: plperl version on the meson setup summary screen Nazir Bilal Yavuz <[email protected]>
2024-12-04 16:10 ` Re: plperl version on the meson setup summary screen Andrew Dunstan <[email protected]>
2025-02-04 06:37 ` Re: plperl version on the meson setup summary screen Zharkov Roman <[email protected]>
@ 2025-03-14 06:08 ` vignesh C <[email protected]>
2025-03-17 08:43 ` Re: plperl version on the meson setup summary screen Zharkov Roman <[email protected]>
1 sibling, 1 reply; 8+ messages in thread
From: vignesh C @ 2025-03-14 06:08 UTC (permalink / raw)
To: Zharkov Roman <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Nazir Bilal Yavuz <[email protected]>; [email protected]
On Tue, 4 Feb 2025 at 12:07, Zharkov Roman <[email protected]> wrote:
>
> Hello,
>
> Here is a new patch version.
> I tried to use perl 'version' instead of 'api_versionstring' to sync
> with configure script.
One suggestion, there are many other external libraries for which we
don't display the version, can we include the version for them too:
@@ -3711,7 +3712,7 @@ if meson.version().version_compare('>=0.57')
'nls': libintl,
'openssl': ssl,
'pam': pam,
- 'plperl': perl_dep,
+ 'plperl': [perl_dep, perlversion],
'plpython': python3_dep,
Regards,
Vignesh
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: plperl version on the meson setup summary screen
2024-11-29 08:26 Re: plperl version on the meson setup summary screen Nazir Bilal Yavuz <[email protected]>
2024-12-04 16:10 ` Re: plperl version on the meson setup summary screen Andrew Dunstan <[email protected]>
2025-02-04 06:37 ` Re: plperl version on the meson setup summary screen Zharkov Roman <[email protected]>
2025-03-14 06:08 ` Re: plperl version on the meson setup summary screen vignesh C <[email protected]>
@ 2025-03-17 08:43 ` Zharkov Roman <[email protected]>
2025-03-17 14:31 ` Re: plperl version on the meson setup summary screen Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 8+ messages in thread
From: Zharkov Roman @ 2025-03-17 08:43 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Nazir Bilal Yavuz <[email protected]>; [email protected]
On 2025-03-14 13:08, vignesh C wrote:
> One suggestion, there are many other external libraries for which we
> don't display the version, can we include the version for them too:
Hello,
We have seven "libraries" whose versions meson does not show: bonjour,
bsd_auth, docs, docs_pdf, nls, pam, plperl. And only plperl has a known
version.
bonjour, bsd_auth produces by declare_dependency() function, and I don't
know where to find their versions.
docs, docs_pdf are the complex of libraries: xmllint_bin, xsltproc_bin
and fop. Which version do we need?
nls can be formed in two ways: by declare_dependency() or by
cc.find_library(). It is not clear to me where to get a version.
pam also can be produced by dependency() or cc.find_library().
Best regards, Roman Zharkov
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: plperl version on the meson setup summary screen
2024-11-29 08:26 Re: plperl version on the meson setup summary screen Nazir Bilal Yavuz <[email protected]>
2024-12-04 16:10 ` Re: plperl version on the meson setup summary screen Andrew Dunstan <[email protected]>
2025-02-04 06:37 ` Re: plperl version on the meson setup summary screen Zharkov Roman <[email protected]>
2025-03-14 06:08 ` Re: plperl version on the meson setup summary screen vignesh C <[email protected]>
2025-03-17 08:43 ` Re: plperl version on the meson setup summary screen Zharkov Roman <[email protected]>
@ 2025-03-17 14:31 ` Andrew Dunstan <[email protected]>
0 siblings, 0 replies; 8+ messages in thread
From: Andrew Dunstan @ 2025-03-17 14:31 UTC (permalink / raw)
To: Zharkov Roman <[email protected]>; vignesh C <[email protected]>; +Cc: Nazir Bilal Yavuz <[email protected]>; [email protected]
On 2025-03-17 Mo 4:43 AM, Zharkov Roman wrote:
> On 2025-03-14 13:08, vignesh C wrote:
>> One suggestion, there are many other external libraries for which we
>> don't display the version, can we include the version for them too:
>
> Hello,
> We have seven "libraries" whose versions meson does not show: bonjour,
> bsd_auth, docs, docs_pdf, nls, pam, plperl. And only plperl has a
> known version.
> bonjour, bsd_auth produces by declare_dependency() function, and I
> don't know where to find their versions.
> docs, docs_pdf are the complex of libraries: xmllint_bin, xsltproc_bin
> and fop. Which version do we need?
> nls can be formed in two ways: by declare_dependency() or by
> cc.find_library(). It is not clear to me where to get a version.
> pam also can be produced by dependency() or cc.find_library().
>
>
>
Yeah. I'm planning to commit your patch shortly.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2025-03-17 14:31 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-09-13 10:12 [PATCH 3/8] Optimize allocations in bringetbitmap Tomas Vondra <[email protected]>
2024-11-29 08:26 Re: plperl version on the meson setup summary screen Nazir Bilal Yavuz <[email protected]>
2024-12-04 16:10 ` Re: plperl version on the meson setup summary screen Andrew Dunstan <[email protected]>
2025-02-04 06:37 ` Re: plperl version on the meson setup summary screen Zharkov Roman <[email protected]>
2025-03-13 14:34 ` Re: plperl version on the meson setup summary screen vignesh C <[email protected]>
2025-03-14 06:08 ` Re: plperl version on the meson setup summary screen vignesh C <[email protected]>
2025-03-17 08:43 ` Re: plperl version on the meson setup summary screen Zharkov Roman <[email protected]>
2025-03-17 14:31 ` Re: plperl version on the meson setup summary screen Andrew Dunstan <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox