agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feed[PATCH 3/9] Unify PostgresNode's new() and get_new_node() methods
213+ messages / 2 participants
[nested] [flat]
* [PATCH 3/9] Unify PostgresNode's new() and get_new_node() methods
@ 2021-05-19 22:11 Andrew Dunstan <andrew@dunslane.net>
0 siblings, 0 replies; 213+ messages in thread
From: Andrew Dunstan @ 2021-05-19 22:11 UTC (permalink / raw)
There is only one constructor now for PostgresNode, with the idiomatic
name 'new'. The method is not exported by the class, and must be called
as "PostgresNode->new('name',[args])". All the TAP tests that use
PostgresNode are modified accordingly. Third party scripts will need
adjusting, which is a fairly mechanical process (I just used a sed
script).
---
contrib/amcheck/t/001_verify_heapam.pl | 2 +-
contrib/auto_explain/t/001_auto_explain.pl | 2 +-
contrib/bloom/t/001_wal.pl | 4 +-
contrib/test_decoding/t/001_repl_stats.pl | 2 +-
src/bin/pg_amcheck/t/002_nonesuch.pl | 2 +-
src/bin/pg_amcheck/t/003_check.pl | 2 +-
src/bin/pg_amcheck/t/004_verify_heapam.pl | 2 +-
src/bin/pg_amcheck/t/005_opclass_damage.pl | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 4 +-
src/bin/pg_basebackup/t/020_pg_receivewal.pl | 2 +-
src/bin/pg_basebackup/t/030_pg_recvlogical.pl | 2 +-
src/bin/pg_checksums/t/002_actions.pl | 2 +-
.../pg_controldata/t/001_pg_controldata.pl | 2 +-
src/bin/pg_ctl/t/002_status.pl | 2 +-
src/bin/pg_ctl/t/003_promote.pl | 6 +-
src/bin/pg_ctl/t/004_logrotate.pl | 2 +-
src/bin/pg_dump/t/002_pg_dump.pl | 2 +-
src/bin/pg_dump/t/003_pg_dump_with_server.pl | 2 +-
src/bin/pg_dump/t/010_dump_connstr.pl | 6 +-
src/bin/pg_resetwal/t/001_basic.pl | 2 +-
src/bin/pg_resetwal/t/002_corrupted.pl | 2 +-
src/bin/pg_rewind/t/007_standby_source.pl | 4 +-
src/bin/pg_rewind/t/008_min_recovery_point.pl | 6 +-
src/bin/pg_rewind/t/RewindTest.pm | 4 +-
src/bin/pg_verifybackup/t/002_algorithm.pl | 2 +-
src/bin/pg_verifybackup/t/003_corruption.pl | 2 +-
src/bin/pg_verifybackup/t/004_options.pl | 2 +-
src/bin/pg_verifybackup/t/006_encoding.pl | 2 +-
src/bin/pg_verifybackup/t/007_wal.pl | 2 +-
src/bin/pgbench/t/001_pgbench_with_server.pl | 2 +-
src/bin/psql/t/010_tab_completion.pl | 2 +-
src/bin/scripts/t/010_clusterdb.pl | 2 +-
src/bin/scripts/t/011_clusterdb_all.pl | 2 +-
src/bin/scripts/t/020_createdb.pl | 2 +-
src/bin/scripts/t/040_createuser.pl | 2 +-
src/bin/scripts/t/050_dropdb.pl | 2 +-
src/bin/scripts/t/070_dropuser.pl | 2 +-
src/bin/scripts/t/080_pg_isready.pl | 2 +-
src/bin/scripts/t/090_reindexdb.pl | 2 +-
src/bin/scripts/t/091_reindexdb_all.pl | 2 +-
src/bin/scripts/t/100_vacuumdb.pl | 2 +-
src/bin/scripts/t/101_vacuumdb_all.pl | 2 +-
src/bin/scripts/t/102_vacuumdb_stages.pl | 2 +-
src/bin/scripts/t/200_connstr.pl | 2 +-
src/test/authentication/t/001_password.pl | 2 +-
src/test/authentication/t/002_saslprep.pl | 2 +-
src/test/kerberos/t/001_auth.pl | 2 +-
src/test/ldap/t/001_auth.pl | 2 +-
src/test/modules/brin/t/01_workitems.pl | 2 +-
src/test/modules/commit_ts/t/001_base.pl | 2 +-
src/test/modules/commit_ts/t/002_standby.pl | 4 +-
src/test/modules/commit_ts/t/003_standby_2.pl | 4 +-
src/test/modules/commit_ts/t/004_restart.pl | 2 +-
.../libpq_pipeline/t/001_libpq_pipeline.pl | 2 +-
.../ssl_passphrase_callback/t/001_testfunc.pl | 2 +-
.../test_misc/t/001_constraint_validation.pl | 2 +-
src/test/modules/test_pg_dump/t/001_base.pl | 2 +-
src/test/perl/PostgresNode.pm | 80 +++++++------------
src/test/recovery/t/001_stream_rep.pl | 6 +-
src/test/recovery/t/002_archiving.pl | 6 +-
src/test/recovery/t/003_recovery_targets.pl | 8 +-
src/test/recovery/t/004_timeline_switch.pl | 10 +--
src/test/recovery/t/005_replay_delay.pl | 4 +-
src/test/recovery/t/006_logical_decoding.pl | 2 +-
src/test/recovery/t/007_sync_rep.pl | 10 +--
src/test/recovery/t/008_fsm_truncation.pl | 4 +-
src/test/recovery/t/009_twophase.pl | 4 +-
.../t/010_logical_decoding_timelines.pl | 4 +-
src/test/recovery/t/011_crash_recovery.pl | 2 +-
src/test/recovery/t/012_subtransactions.pl | 4 +-
src/test/recovery/t/013_crash_restart.pl | 2 +-
src/test/recovery/t/014_unlogged_reinit.pl | 2 +-
src/test/recovery/t/015_promotion_pages.pl | 4 +-
src/test/recovery/t/016_min_consistency.pl | 4 +-
src/test/recovery/t/017_shm.pl | 2 +-
src/test/recovery/t/018_wal_optimize.pl | 2 +-
src/test/recovery/t/019_replslot_limit.pl | 8 +-
src/test/recovery/t/020_archive_status.pl | 6 +-
src/test/recovery/t/021_row_visibility.pl | 4 +-
src/test/recovery/t/022_crash_temp_files.pl | 2 +-
src/test/recovery/t/023_pitr_prepared_xact.pl | 4 +-
src/test/recovery/t/024_archive_recovery.pl | 4 +-
src/test/ssl/t/001_ssltests.pl | 2 +-
src/test/ssl/t/002_scram.pl | 2 +-
src/test/subscription/t/001_rep_changes.pl | 4 +-
src/test/subscription/t/002_types.pl | 4 +-
src/test/subscription/t/003_constraints.pl | 4 +-
src/test/subscription/t/004_sync.pl | 4 +-
src/test/subscription/t/005_encoding.pl | 4 +-
src/test/subscription/t/006_rewrite.pl | 4 +-
src/test/subscription/t/007_ddl.pl | 4 +-
src/test/subscription/t/008_diff_schema.pl | 4 +-
src/test/subscription/t/009_matviews.pl | 4 +-
src/test/subscription/t/010_truncate.pl | 4 +-
src/test/subscription/t/011_generated.pl | 4 +-
src/test/subscription/t/012_collation.pl | 4 +-
src/test/subscription/t/013_partition.pl | 6 +-
src/test/subscription/t/014_binary.pl | 4 +-
src/test/subscription/t/015_stream.pl | 4 +-
src/test/subscription/t/016_stream_subxact.pl | 4 +-
src/test/subscription/t/017_stream_ddl.pl | 4 +-
.../t/018_stream_subxact_abort.pl | 4 +-
.../t/019_stream_subxact_ddl_abort.pl | 4 +-
src/test/subscription/t/020_messages.pl | 4 +-
src/test/subscription/t/100_bugs.pl | 14 ++--
105 files changed, 200 insertions(+), 226 deletions(-)
diff --git a/contrib/amcheck/t/001_verify_heapam.pl b/contrib/amcheck/t/001_verify_heapam.pl
index 9bd66c07f4..4f720a7ed0 100644
--- a/contrib/amcheck/t/001_verify_heapam.pl
+++ b/contrib/amcheck/t/001_verify_heapam.pl
@@ -15,7 +15,7 @@ my ($node, $result);
#
# Test set-up
#
-$node = get_new_node('test');
+$node = PostgresNode->new('test');
$node->init;
$node->append_conf('postgresql.conf', 'autovacuum=off');
$node->start;
diff --git a/contrib/auto_explain/t/001_auto_explain.pl b/contrib/auto_explain/t/001_auto_explain.pl
index 9c4f1d0571..1773a37999 100644
--- a/contrib/auto_explain/t/001_auto_explain.pl
+++ b/contrib/auto_explain/t/001_auto_explain.pl
@@ -8,7 +8,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 4;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->append_conf('postgresql.conf',
"shared_preload_libraries = 'auto_explain'");
diff --git a/contrib/bloom/t/001_wal.pl b/contrib/bloom/t/001_wal.pl
index 9310af5c3d..55ad35926f 100644
--- a/contrib/bloom/t/001_wal.pl
+++ b/contrib/bloom/t/001_wal.pl
@@ -43,7 +43,7 @@ SELECT * FROM tst WHERE i = 7 AND t = 'e';
}
# Initialize primary node
-$node_primary = get_new_node('primary');
+$node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1);
$node_primary->start;
my $backup_name = 'my_backup';
@@ -52,7 +52,7 @@ my $backup_name = 'my_backup';
$node_primary->backup($backup_name);
# Create streaming standby linking to primary
-$node_standby = get_new_node('standby');
+$node_standby = PostgresNode->new('standby');
$node_standby->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
$node_standby->start;
diff --git a/contrib/test_decoding/t/001_repl_stats.pl b/contrib/test_decoding/t/001_repl_stats.pl
index 2dc5ef5f07..fdef6cb1ff 100644
--- a/contrib/test_decoding/t/001_repl_stats.pl
+++ b/contrib/test_decoding/t/001_repl_stats.pl
@@ -11,7 +11,7 @@ use TestLib;
use Test::More tests => 2;
# Test set-up
-my $node = get_new_node('test');
+my $node = PostgresNode->new('test');
$node->init(allows_streaming => 'logical');
$node->append_conf('postgresql.conf', 'synchronous_commit = on');
$node->start;
diff --git a/src/bin/pg_amcheck/t/002_nonesuch.pl b/src/bin/pg_amcheck/t/002_nonesuch.pl
index 5f712ee32a..5417959553 100644
--- a/src/bin/pg_amcheck/t/002_nonesuch.pl
+++ b/src/bin/pg_amcheck/t/002_nonesuch.pl
@@ -10,7 +10,7 @@ use Test::More tests => 72;
# Test set-up
my ($node, $port);
-$node = get_new_node('test');
+$node = PostgresNode->new('test');
$node->init;
$node->start;
$port = $node->port;
diff --git a/src/bin/pg_amcheck/t/003_check.pl b/src/bin/pg_amcheck/t/003_check.pl
index 817eb4e116..f3eb41ce3a 100644
--- a/src/bin/pg_amcheck/t/003_check.pl
+++ b/src/bin/pg_amcheck/t/003_check.pl
@@ -120,7 +120,7 @@ sub perform_all_corruptions()
}
# Test set-up
-$node = get_new_node('test');
+$node = PostgresNode->new('test');
$node->init;
$node->append_conf('postgresql.conf', 'autovacuum=off');
$node->start;
diff --git a/src/bin/pg_amcheck/t/004_verify_heapam.pl b/src/bin/pg_amcheck/t/004_verify_heapam.pl
index b3a96e8016..e4c0b83a1c 100644
--- a/src/bin/pg_amcheck/t/004_verify_heapam.pl
+++ b/src/bin/pg_amcheck/t/004_verify_heapam.pl
@@ -178,7 +178,7 @@ umask(0077);
# Set up the node. Once we create and corrupt the table,
# autovacuum workers visiting the table could crash the backend.
# Disable autovacuum so that won't happen.
-my $node = get_new_node('test');
+my $node = PostgresNode->new('test');
$node->init;
$node->append_conf('postgresql.conf', 'autovacuum=off');
diff --git a/src/bin/pg_amcheck/t/005_opclass_damage.pl b/src/bin/pg_amcheck/t/005_opclass_damage.pl
index b65becae9d..806335375d 100644
--- a/src/bin/pg_amcheck/t/005_opclass_damage.pl
+++ b/src/bin/pg_amcheck/t/005_opclass_damage.pl
@@ -10,7 +10,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 5;
-my $node = get_new_node('test');
+my $node = PostgresNode->new('test');
$node->init;
$node->start;
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 74f8c2c739..ecd6a5d374 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -18,7 +18,7 @@ program_options_handling_ok('pg_basebackup');
my $tempdir = TestLib::tempdir;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
# Set umask so test directories and files are created with default permissions
umask(0077);
@@ -268,7 +268,7 @@ SKIP:
skip "no tar program available", 1
if (!defined $tar || $tar eq '');
- my $node2 = get_new_node('replica');
+ my $node2 = PostgresNode->new('replica');
# Recover main data directory
$node2->init_from_backup($node, 'tarbackup2', tar_program => $tar);
diff --git a/src/bin/pg_basebackup/t/020_pg_receivewal.pl b/src/bin/pg_basebackup/t/020_pg_receivewal.pl
index 158f7d176f..81c49de22e 100644
--- a/src/bin/pg_basebackup/t/020_pg_receivewal.pl
+++ b/src/bin/pg_basebackup/t/020_pg_receivewal.pl
@@ -14,7 +14,7 @@ program_options_handling_ok('pg_receivewal');
# Set umask so test directories and files are created with default permissions
umask(0077);
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(allows_streaming => 1);
$primary->start;
diff --git a/src/bin/pg_basebackup/t/030_pg_recvlogical.pl b/src/bin/pg_basebackup/t/030_pg_recvlogical.pl
index bbbf9e21db..fe7fe76295 100644
--- a/src/bin/pg_basebackup/t/030_pg_recvlogical.pl
+++ b/src/bin/pg_basebackup/t/030_pg_recvlogical.pl
@@ -11,7 +11,7 @@ program_help_ok('pg_recvlogical');
program_version_ok('pg_recvlogical');
program_options_handling_ok('pg_recvlogical');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
# Initialize node without replication settings
$node->init(allows_streaming => 1, has_archiving => 1);
diff --git a/src/bin/pg_checksums/t/002_actions.pl b/src/bin/pg_checksums/t/002_actions.pl
index af88b94795..a18c104a94 100644
--- a/src/bin/pg_checksums/t/002_actions.pl
+++ b/src/bin/pg_checksums/t/002_actions.pl
@@ -92,7 +92,7 @@ sub check_relation_corruption
}
# Initialize node with checksums disabled.
-my $node = get_new_node('node_checksum');
+my $node = PostgresNode->new('node_checksum');
$node->init();
my $pgdata = $node->data_dir;
diff --git a/src/bin/pg_controldata/t/001_pg_controldata.pl b/src/bin/pg_controldata/t/001_pg_controldata.pl
index c3f3aca095..ce31cfbd3b 100644
--- a/src/bin/pg_controldata/t/001_pg_controldata.pl
+++ b/src/bin/pg_controldata/t/001_pg_controldata.pl
@@ -14,7 +14,7 @@ command_fails(['pg_controldata'], 'pg_controldata without arguments fails');
command_fails([ 'pg_controldata', 'nonexistent' ],
'pg_controldata with nonexistent directory fails');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
command_like([ 'pg_controldata', $node->data_dir ],
diff --git a/src/bin/pg_ctl/t/002_status.pl b/src/bin/pg_ctl/t/002_status.pl
index e69cb80134..56a06fafa3 100644
--- a/src/bin/pg_ctl/t/002_status.pl
+++ b/src/bin/pg_ctl/t/002_status.pl
@@ -14,7 +14,7 @@ my $tempdir_short = TestLib::tempdir_short;
command_exit_is([ 'pg_ctl', 'status', '-D', "$tempdir/nonexistent" ],
4, 'pg_ctl status with nonexistent directory');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
command_exit_is([ 'pg_ctl', 'status', '-D', $node->data_dir ],
diff --git a/src/bin/pg_ctl/t/003_promote.pl b/src/bin/pg_ctl/t/003_promote.pl
index 2d7e2fd5f3..547b3d8893 100644
--- a/src/bin/pg_ctl/t/003_promote.pl
+++ b/src/bin/pg_ctl/t/003_promote.pl
@@ -15,7 +15,7 @@ command_fails_like(
qr/directory .* does not exist/,
'pg_ctl promote with nonexistent directory');
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1);
command_fails_like(
@@ -30,7 +30,7 @@ command_fails_like(
qr/not in standby mode/,
'pg_ctl promote of primary instance fails');
-my $node_standby = get_new_node('standby');
+my $node_standby = PostgresNode->new('standby');
$node_primary->backup('my_backup');
$node_standby->init_from_backup($node_primary, 'my_backup',
has_streaming => 1);
@@ -47,7 +47,7 @@ ok( $node_standby->poll_query_until(
'promoted standby is not in recovery');
# same again with default wait option
-$node_standby = get_new_node('standby2');
+$node_standby = PostgresNode->new('standby2');
$node_standby->init_from_backup($node_primary, 'my_backup',
has_streaming => 1);
$node_standby->start;
diff --git a/src/bin/pg_ctl/t/004_logrotate.pl b/src/bin/pg_ctl/t/004_logrotate.pl
index 9a0492ae63..fa14b98c7d 100644
--- a/src/bin/pg_ctl/t/004_logrotate.pl
+++ b/src/bin/pg_ctl/t/004_logrotate.pl
@@ -10,7 +10,7 @@ use Test::More tests => 5;
use Time::HiRes qw(usleep);
# Set up node with logging collector
-my $node = get_new_node('primary');
+my $node = PostgresNode->new('primary');
$node->init();
$node->append_conf(
'postgresql.conf', qq(
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 448b1be26c..81a63882e1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -3503,7 +3503,7 @@ my %tests = (
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/pg_dump/t/003_pg_dump_with_server.pl b/src/bin/pg_dump/t/003_pg_dump_with_server.pl
index f9fea9ddcf..ba994aee82 100644
--- a/src/bin/pg_dump/t/003_pg_dump_with_server.pl
+++ b/src/bin/pg_dump/t/003_pg_dump_with_server.pl
@@ -11,7 +11,7 @@ use Test::More tests => 3;
my $tempdir = TestLib::tempdir;
my $tempdir_short = TestLib::tempdir_short;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
my $port = $node->port;
$node->init;
diff --git a/src/bin/pg_dump/t/010_dump_connstr.pl b/src/bin/pg_dump/t/010_dump_connstr.pl
index 6478894160..c4b60c5d2a 100644
--- a/src/bin/pg_dump/t/010_dump_connstr.pl
+++ b/src/bin/pg_dump/t/010_dump_connstr.pl
@@ -51,7 +51,7 @@ my $dbname4 = 'regression' . generate_ascii_string(203, 255);
my $src_bootstrap_super = 'regress_postgres';
my $dst_bootstrap_super = 'boot';
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init(extra =>
[ '-U', $src_bootstrap_super, '--locale=C', '--encoding=LATIN1' ]);
@@ -181,7 +181,7 @@ $restore_super =~ s/"//g
# Restore full dump through psql using environment variables for
# dbname/user connection parameters
-my $envar_node = get_new_node('destination_envar');
+my $envar_node = PostgresNode->new('destination_envar');
$envar_node->init(
extra =>
[ '-U', $dst_bootstrap_super, '--locale=C', '--encoding=LATIN1' ],
@@ -208,7 +208,7 @@ is($stderr, '', 'no dump errors');
# dbname/user connection parameters. "\connect dbname=" forgets
# user/port from command line.
-my $cmdline_node = get_new_node('destination_cmdline');
+my $cmdline_node = PostgresNode->new('destination_cmdline');
$cmdline_node->init(
extra =>
[ '-U', $dst_bootstrap_super, '--locale=C', '--encoding=LATIN1' ],
diff --git a/src/bin/pg_resetwal/t/001_basic.pl b/src/bin/pg_resetwal/t/001_basic.pl
index 9c08ade79f..f01a4d2d48 100644
--- a/src/bin/pg_resetwal/t/001_basic.pl
+++ b/src/bin/pg_resetwal/t/001_basic.pl
@@ -12,7 +12,7 @@ program_help_ok('pg_resetwal');
program_version_ok('pg_resetwal');
program_options_handling_ok('pg_resetwal');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
command_like([ 'pg_resetwal', '-n', $node->data_dir ],
diff --git a/src/bin/pg_resetwal/t/002_corrupted.pl b/src/bin/pg_resetwal/t/002_corrupted.pl
index 954790c28c..ac915ef91f 100644
--- a/src/bin/pg_resetwal/t/002_corrupted.pl
+++ b/src/bin/pg_resetwal/t/002_corrupted.pl
@@ -10,7 +10,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 6;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
my $pg_control = $node->data_dir . '/global/pg_control';
diff --git a/src/bin/pg_rewind/t/007_standby_source.pl b/src/bin/pg_rewind/t/007_standby_source.pl
index 44319a8204..2a717f5a2e 100644
--- a/src/bin/pg_rewind/t/007_standby_source.pl
+++ b/src/bin/pg_rewind/t/007_standby_source.pl
@@ -58,13 +58,13 @@ primary_psql("CHECKPOINT");
#
# A (primary) <--- B (standby) <--- C (standby)
$node_a->backup('my_backup');
-$node_b = get_new_node('node_b');
+$node_b = PostgresNode->new('node_b');
$node_b->init_from_backup($node_a, 'my_backup', has_streaming => 1);
$node_b->set_standby_mode();
$node_b->start;
$node_b->backup('my_backup');
-$node_c = get_new_node('node_c');
+$node_c = PostgresNode->new('node_c');
$node_c->init_from_backup($node_b, 'my_backup', has_streaming => 1);
$node_c->set_standby_mode();
$node_c->start;
diff --git a/src/bin/pg_rewind/t/008_min_recovery_point.pl b/src/bin/pg_rewind/t/008_min_recovery_point.pl
index 9ebcbad0d2..7a390f4abd 100644
--- a/src/bin/pg_rewind/t/008_min_recovery_point.pl
+++ b/src/bin/pg_rewind/t/008_min_recovery_point.pl
@@ -40,7 +40,7 @@ use File::Copy;
my $tmp_folder = TestLib::tempdir;
-my $node_1 = get_new_node('node_1');
+my $node_1 = PostgresNode->new('node_1');
$node_1->init(allows_streaming => 1);
$node_1->append_conf(
'postgresql.conf', qq(
@@ -60,11 +60,11 @@ $node_1->safe_psql('postgres', "INSERT INTO public.bar VALUES ('in both')");
my $backup_name = 'my_backup';
$node_1->backup($backup_name);
-my $node_2 = get_new_node('node_2');
+my $node_2 = PostgresNode->new('node_2');
$node_2->init_from_backup($node_1, $backup_name, has_streaming => 1);
$node_2->start;
-my $node_3 = get_new_node('node_3');
+my $node_3 = PostgresNode->new('node_3');
$node_3->init_from_backup($node_1, $backup_name, has_streaming => 1);
$node_3->start;
diff --git a/src/bin/pg_rewind/t/RewindTest.pm b/src/bin/pg_rewind/t/RewindTest.pm
index 938c661afc..367b99a438 100644
--- a/src/bin/pg_rewind/t/RewindTest.pm
+++ b/src/bin/pg_rewind/t/RewindTest.pm
@@ -128,7 +128,7 @@ sub setup_cluster
# Initialize primary, data checksums are mandatory
$node_primary =
- get_new_node('primary' . ($extra_name ? "_${extra_name}" : ''));
+ PostgresNode->new('primary' . ($extra_name ? "_${extra_name}" : ''));
# Set up pg_hba.conf and pg_ident.conf for the role running
# pg_rewind. This role is used for all the tests, and has
@@ -176,7 +176,7 @@ sub create_standby
my $extra_name = shift;
$node_standby =
- get_new_node('standby' . ($extra_name ? "_${extra_name}" : ''));
+ PostgresNode->new('standby' . ($extra_name ? "_${extra_name}" : ''));
$node_primary->backup('my_backup');
$node_standby->init_from_backup($node_primary, 'my_backup');
my $connstr_primary = $node_primary->connstr();
diff --git a/src/bin/pg_verifybackup/t/002_algorithm.pl b/src/bin/pg_verifybackup/t/002_algorithm.pl
index c2c4c3176a..7dc1af982a 100644
--- a/src/bin/pg_verifybackup/t/002_algorithm.pl
+++ b/src/bin/pg_verifybackup/t/002_algorithm.pl
@@ -12,7 +12,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 19;
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(allows_streaming => 1);
$primary->start;
diff --git a/src/bin/pg_verifybackup/t/003_corruption.pl b/src/bin/pg_verifybackup/t/003_corruption.pl
index 682b3b857e..509390f975 100644
--- a/src/bin/pg_verifybackup/t/003_corruption.pl
+++ b/src/bin/pg_verifybackup/t/003_corruption.pl
@@ -12,7 +12,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 44;
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(allows_streaming => 1);
$primary->start;
diff --git a/src/bin/pg_verifybackup/t/004_options.pl b/src/bin/pg_verifybackup/t/004_options.pl
index 3f6e84c221..d4210fd293 100644
--- a/src/bin/pg_verifybackup/t/004_options.pl
+++ b/src/bin/pg_verifybackup/t/004_options.pl
@@ -13,7 +13,7 @@ use TestLib;
use Test::More tests => 25;
# Start up the server and take a backup.
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(allows_streaming => 1);
$primary->start;
my $backup_path = $primary->backup_dir . '/test_options';
diff --git a/src/bin/pg_verifybackup/t/006_encoding.pl b/src/bin/pg_verifybackup/t/006_encoding.pl
index a821d52a6a..c0667b7f43 100644
--- a/src/bin/pg_verifybackup/t/006_encoding.pl
+++ b/src/bin/pg_verifybackup/t/006_encoding.pl
@@ -11,7 +11,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 5;
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(allows_streaming => 1);
$primary->start;
my $backup_path = $primary->backup_dir . '/test_encoding';
diff --git a/src/bin/pg_verifybackup/t/007_wal.pl b/src/bin/pg_verifybackup/t/007_wal.pl
index 28837b8503..adf60fb755 100644
--- a/src/bin/pg_verifybackup/t/007_wal.pl
+++ b/src/bin/pg_verifybackup/t/007_wal.pl
@@ -13,7 +13,7 @@ use TestLib;
use Test::More tests => 7;
# Start up the server and take a backup.
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(allows_streaming => 1);
$primary->start;
my $backup_path = $primary->backup_dir . '/test_wal';
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index 3aa9d5d753..ef53f6b2d9 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -10,7 +10,7 @@ use Test::More;
use Config;
# start a pgbench specific server
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl
index 3c58d50118..8695d22545 100644
--- a/src/bin/psql/t/010_tab_completion.pl
+++ b/src/bin/psql/t/010_tab_completion.pl
@@ -34,7 +34,7 @@ if ($@)
}
# start a new server
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/010_clusterdb.pl b/src/bin/scripts/t/010_clusterdb.pl
index 6d483be143..aae5ea985b 100644
--- a/src/bin/scripts/t/010_clusterdb.pl
+++ b/src/bin/scripts/t/010_clusterdb.pl
@@ -12,7 +12,7 @@ program_help_ok('clusterdb');
program_version_ok('clusterdb');
program_options_handling_ok('clusterdb');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/011_clusterdb_all.pl b/src/bin/scripts/t/011_clusterdb_all.pl
index c7e8514fb6..6ba71780de 100644
--- a/src/bin/scripts/t/011_clusterdb_all.pl
+++ b/src/bin/scripts/t/011_clusterdb_all.pl
@@ -8,7 +8,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 2;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/020_createdb.pl b/src/bin/scripts/t/020_createdb.pl
index 7261ebb2ef..3db2f13ae2 100644
--- a/src/bin/scripts/t/020_createdb.pl
+++ b/src/bin/scripts/t/020_createdb.pl
@@ -12,7 +12,7 @@ program_help_ok('createdb');
program_version_ok('createdb');
program_options_handling_ok('createdb');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/040_createuser.pl b/src/bin/scripts/t/040_createuser.pl
index 8fdd32d77b..3da8b7ae75 100644
--- a/src/bin/scripts/t/040_createuser.pl
+++ b/src/bin/scripts/t/040_createuser.pl
@@ -12,7 +12,7 @@ program_help_ok('createuser');
program_version_ok('createuser');
program_options_handling_ok('createuser');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/050_dropdb.pl b/src/bin/scripts/t/050_dropdb.pl
index 646cb4e82f..b9f0592bca 100644
--- a/src/bin/scripts/t/050_dropdb.pl
+++ b/src/bin/scripts/t/050_dropdb.pl
@@ -12,7 +12,7 @@ program_help_ok('dropdb');
program_version_ok('dropdb');
program_options_handling_ok('dropdb');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/070_dropuser.pl b/src/bin/scripts/t/070_dropuser.pl
index cbcb09b0ad..26dadaf4a5 100644
--- a/src/bin/scripts/t/070_dropuser.pl
+++ b/src/bin/scripts/t/070_dropuser.pl
@@ -12,7 +12,7 @@ program_help_ok('dropuser');
program_version_ok('dropuser');
program_options_handling_ok('dropuser');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/080_pg_isready.pl b/src/bin/scripts/t/080_pg_isready.pl
index e2e39ea2d4..4f1184953e 100644
--- a/src/bin/scripts/t/080_pg_isready.pl
+++ b/src/bin/scripts/t/080_pg_isready.pl
@@ -14,7 +14,7 @@ program_options_handling_ok('pg_isready');
command_fails(['pg_isready'], 'fails with no server running');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/090_reindexdb.pl b/src/bin/scripts/t/090_reindexdb.pl
index af5bdf352c..541504d8f0 100644
--- a/src/bin/scripts/t/090_reindexdb.pl
+++ b/src/bin/scripts/t/090_reindexdb.pl
@@ -12,7 +12,7 @@ program_help_ok('reindexdb');
program_version_ok('reindexdb');
program_options_handling_ok('reindexdb');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/091_reindexdb_all.pl b/src/bin/scripts/t/091_reindexdb_all.pl
index 299b198d15..34174e4d71 100644
--- a/src/bin/scripts/t/091_reindexdb_all.pl
+++ b/src/bin/scripts/t/091_reindexdb_all.pl
@@ -7,7 +7,7 @@ use warnings;
use PostgresNode;
use Test::More tests => 2;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/100_vacuumdb.pl b/src/bin/scripts/t/100_vacuumdb.pl
index 0addc97bf8..0f1e5bb278 100644
--- a/src/bin/scripts/t/100_vacuumdb.pl
+++ b/src/bin/scripts/t/100_vacuumdb.pl
@@ -12,7 +12,7 @@ program_help_ok('vacuumdb');
program_version_ok('vacuumdb');
program_options_handling_ok('vacuumdb');
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/101_vacuumdb_all.pl b/src/bin/scripts/t/101_vacuumdb_all.pl
index 504f252748..a60fc79aae 100644
--- a/src/bin/scripts/t/101_vacuumdb_all.pl
+++ b/src/bin/scripts/t/101_vacuumdb_all.pl
@@ -7,7 +7,7 @@ use warnings;
use PostgresNode;
use Test::More tests => 2;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/102_vacuumdb_stages.pl b/src/bin/scripts/t/102_vacuumdb_stages.pl
index 155c77edd9..c15057a763 100644
--- a/src/bin/scripts/t/102_vacuumdb_stages.pl
+++ b/src/bin/scripts/t/102_vacuumdb_stages.pl
@@ -7,7 +7,7 @@ use warnings;
use PostgresNode;
use Test::More tests => 4;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/bin/scripts/t/200_connstr.pl b/src/bin/scripts/t/200_connstr.pl
index b1ceab73bb..f702e32f00 100644
--- a/src/bin/scripts/t/200_connstr.pl
+++ b/src/bin/scripts/t/200_connstr.pl
@@ -23,7 +23,7 @@ my $dbname2 =
my $dbname3 = generate_ascii_string(130, 192);
my $dbname4 = generate_ascii_string(193, 255);
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init(extra => [ '--locale=C', '--encoding=LATIN1' ]);
$node->start;
diff --git a/src/test/authentication/t/001_password.pl b/src/test/authentication/t/001_password.pl
index 427a360198..9498c18d7d 100644
--- a/src/test/authentication/t/001_password.pl
+++ b/src/test/authentication/t/001_password.pl
@@ -62,7 +62,7 @@ sub test_role
}
# Initialize primary node
-my $node = get_new_node('primary');
+my $node = PostgresNode->new('primary');
$node->init;
$node->append_conf('postgresql.conf', "log_connections = on\n");
$node->start;
diff --git a/src/test/authentication/t/002_saslprep.pl b/src/test/authentication/t/002_saslprep.pl
index f080a0ccba..4799e927db 100644
--- a/src/test/authentication/t/002_saslprep.pl
+++ b/src/test/authentication/t/002_saslprep.pl
@@ -62,7 +62,7 @@ sub test_login
# Initialize primary node. Force UTF-8 encoding, so that we can use non-ASCII
# characters in the passwords below.
-my $node = get_new_node('primary');
+my $node = PostgresNode->new('primary');
$node->init(extra => [ '--locale=C', '--encoding=UTF8' ]);
$node->start;
diff --git a/src/test/kerberos/t/001_auth.pl b/src/test/kerberos/t/001_auth.pl
index b5594924ca..d6191498f0 100644
--- a/src/test/kerberos/t/001_auth.pl
+++ b/src/test/kerberos/t/001_auth.pl
@@ -167,7 +167,7 @@ END
note "setting up PostgreSQL instance";
-my $node = get_new_node('node');
+my $node = PostgresNode->new('node');
$node->init;
$node->append_conf(
'postgresql.conf', qq{
diff --git a/src/test/ldap/t/001_auth.pl b/src/test/ldap/t/001_auth.pl
index 0ae14e4c85..553041baa2 100644
--- a/src/test/ldap/t/001_auth.pl
+++ b/src/test/ldap/t/001_auth.pl
@@ -153,7 +153,7 @@ system_or_bail 'ldappasswd', '-x', '-y', $ldap_pwfile, '-s', 'secret2',
note "setting up PostgreSQL instance";
-my $node = get_new_node('node');
+my $node = PostgresNode->new('node');
$node->init;
$node->append_conf('postgresql.conf', "log_connections = on\n");
$node->start;
diff --git a/src/test/modules/brin/t/01_workitems.pl b/src/test/modules/brin/t/01_workitems.pl
index a4f603a9d5..ea2e2944a1 100644
--- a/src/test/modules/brin/t/01_workitems.pl
+++ b/src/test/modules/brin/t/01_workitems.pl
@@ -10,7 +10,7 @@ use TestLib;
use Test::More tests => 2;
use PostgresNode;
-my $node = get_new_node('tango');
+my $node = PostgresNode->new('tango');
$node->init;
$node->append_conf('postgresql.conf', 'autovacuum_naptime=1s');
$node->start;
diff --git a/src/test/modules/commit_ts/t/001_base.pl b/src/test/modules/commit_ts/t/001_base.pl
index dd41936658..0c504421d4 100644
--- a/src/test/modules/commit_ts/t/001_base.pl
+++ b/src/test/modules/commit_ts/t/001_base.pl
@@ -10,7 +10,7 @@ use TestLib;
use Test::More tests => 2;
use PostgresNode;
-my $node = get_new_node('foxtrot');
+my $node = PostgresNode->new('foxtrot');
$node->init;
$node->append_conf('postgresql.conf', 'track_commit_timestamp = on');
$node->start;
diff --git a/src/test/modules/commit_ts/t/002_standby.pl b/src/test/modules/commit_ts/t/002_standby.pl
index 2664b177ec..227eddeda2 100644
--- a/src/test/modules/commit_ts/t/002_standby.pl
+++ b/src/test/modules/commit_ts/t/002_standby.pl
@@ -11,7 +11,7 @@ use Test::More tests => 4;
use PostgresNode;
my $bkplabel = 'backup';
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(allows_streaming => 1);
$primary->append_conf(
@@ -22,7 +22,7 @@ $primary->append_conf(
$primary->start;
$primary->backup($bkplabel);
-my $standby = get_new_node('standby');
+my $standby = PostgresNode->new('standby');
$standby->init_from_backup($primary, $bkplabel, has_streaming => 1);
$standby->start;
diff --git a/src/test/modules/commit_ts/t/003_standby_2.pl b/src/test/modules/commit_ts/t/003_standby_2.pl
index 57ab9b1d46..27c5bfbfb7 100644
--- a/src/test/modules/commit_ts/t/003_standby_2.pl
+++ b/src/test/modules/commit_ts/t/003_standby_2.pl
@@ -11,7 +11,7 @@ use Test::More tests => 4;
use PostgresNode;
my $bkplabel = 'backup';
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(allows_streaming => 1);
$primary->append_conf(
'postgresql.conf', qq{
@@ -21,7 +21,7 @@ $primary->append_conf(
$primary->start;
$primary->backup($bkplabel);
-my $standby = get_new_node('standby');
+my $standby = PostgresNode->new('standby');
$standby->init_from_backup($primary, $bkplabel, has_streaming => 1);
$standby->start;
diff --git a/src/test/modules/commit_ts/t/004_restart.pl b/src/test/modules/commit_ts/t/004_restart.pl
index bc9931944f..5d92c5a2e6 100644
--- a/src/test/modules/commit_ts/t/004_restart.pl
+++ b/src/test/modules/commit_ts/t/004_restart.pl
@@ -8,7 +8,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 16;
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1);
$node_primary->append_conf('postgresql.conf', 'track_commit_timestamp = on');
$node_primary->start;
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
index 4101ef950e..49f211c827 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
@@ -9,7 +9,7 @@ use PostgresNode;
use TestLib;
use Test::More;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl b/src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
index 1e2455e82f..0a45fc1b75 100644
--- a/src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
+++ b/src/test/modules/ssl_passphrase_callback/t/001_testfunc.pl
@@ -20,7 +20,7 @@ my $rot13pass = "SbbOnE1";
# see the Makefile for how the certificate and key have been generated
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->append_conf('postgresql.conf',
"ssl_passphrase.passphrase = '$rot13pass'");
diff --git a/src/test/modules/test_misc/t/001_constraint_validation.pl b/src/test/modules/test_misc/t/001_constraint_validation.pl
index 3729906c1a..7c1929f805 100644
--- a/src/test/modules/test_misc/t/001_constraint_validation.pl
+++ b/src/test/modules/test_misc/t/001_constraint_validation.pl
@@ -10,7 +10,7 @@ use TestLib;
use Test::More tests => 42;
# Initialize a test cluster
-my $node = get_new_node('primary');
+my $node = PostgresNode->new('primary');
$node->init();
# Turn message level up to DEBUG1 so that we get the messages we want to see
$node->append_conf('postgresql.conf', 'client_min_messages = DEBUG1');
diff --git a/src/test/modules/test_pg_dump/t/001_base.pl b/src/test/modules/test_pg_dump/t/001_base.pl
index 8511da5169..ea7739d725 100644
--- a/src/test/modules/test_pg_dump/t/001_base.pl
+++ b/src/test/modules/test_pg_dump/t/001_base.pl
@@ -713,7 +713,7 @@ my %tests = (
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 6dd1e6ac03..2a889065ea 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -11,7 +11,7 @@ PostgresNode - class representing PostgreSQL server instance
use PostgresNode;
- my $node = PostgresNode->get_new_node('mynode');
+ my $node = PostgresNode->new('mynode');
# Create a data directory with initdb
$node->init();
@@ -61,9 +61,9 @@ PostgresNode - class representing PostgreSQL server instance
my $ret = $node->backup_fs_cold('testbackup3')
# Restore it to create a new independent node (not a replica)
- my $replica = get_new_node('replica');
- $replica->init_from_backup($node, 'testbackup');
- $replica->start;
+ my $other_node = PostgresNode->new('mycopy');
+ $other_node->init_from_backup($node, 'testbackup');
+ $other_node->start;
# Stop the server
$node->stop('fast');
@@ -110,7 +110,6 @@ use Time::HiRes qw(usleep);
use Scalar::Util qw(blessed);
our @EXPORT = qw(
- get_new_node
get_free_port
);
@@ -139,41 +138,6 @@ INIT
=over
-=item PostgresNode::new($class, $name, $pghost, $pgport)
-
-Create a new PostgresNode instance. Does not initdb or start it.
-
-You should generally prefer to use get_new_node() instead since it takes care
-of finding port numbers, registering instances for cleanup, etc.
-
-=cut
-
-sub new
-{
- my ($class, $name, $pghost, $pgport) = @_;
- my $testname = basename($0);
- $testname =~ s/\.[^.]+$//;
- my $self = {
- _port => $pgport,
- _host => $pghost,
- _basedir => "$TestLib::tmp_check/t_${testname}_${name}_data",
- _name => $name,
- _logfile_generation => 0,
- _logfile_base => "$TestLib::log_path/${testname}_${name}",
- _logfile => "$TestLib::log_path/${testname}_${name}.log"
- };
-
- bless $self, $class;
- mkdir $self->{_basedir}
- or
- BAIL_OUT("could not create data directory \"$self->{_basedir}\": $!");
- $self->dump_info;
-
- return $self;
-}
-
-=pod
-
=item $node->port()
Get the port number assigned to the host. This won't necessarily be a TCP port
@@ -1166,15 +1130,13 @@ sub _update_pid
=pod
-=item PostgresNode->get_new_node(node_name, %params)
+=item PostgresNode->new(node_name, %params)
Build a new object of class C<PostgresNode> (or of a subclass, if you have
one), assigning a free port number. Remembers the node, to prevent its port
number from being reused for another node, and to ensure that it gets
shut down when the test script exits.
-You should generally use this instead of C<PostgresNode::new(...)>.
-
=over
=item port => [1,65535]
@@ -1199,15 +1161,11 @@ not provided, Postgres binaries will be found in the caller's PATH.
=back
-For backwards compatibility, it is also exported as a standalone function,
-which can only create objects of class C<PostgresNode>.
-
=cut
-sub get_new_node
+sub new
{
- my $class = 'PostgresNode';
- $class = shift if scalar(@_) % 2 != 1;
+ my $class = shift;
my ($name, %params) = @_;
# Select a port.
@@ -1242,14 +1200,30 @@ sub get_new_node
}
}
- # Lock port number found by creating a new node
- my $node = $class->new($name, $host, $port);
+ my $testname = basename($0);
+ $testname =~ s/\.[^.]+$//;
+ my $node = {
+ _port => $port,
+ _host => $host,
+ _basedir => "$TestLib::tmp_check/t_${testname}_${name}_data",
+ _name => $name,
+ _logfile_generation => 0,
+ _logfile_base => "$TestLib::log_path/${testname}_${name}",
+ _logfile => "$TestLib::log_path/${testname}_${name}.log"
+ };
if ($params{install_path})
{
$node->{_install_path} = $params{install_path};
}
+ bless $node, $class;
+ mkdir $node->{_basedir}
+ or
+ BAIL_OUT("could not create data directory \"$node->{_basedir}\": $!");
+
+ $node->dump_info;
+
# Add node to list of nodes
push(@all_nodes, $node);
@@ -1320,7 +1294,7 @@ sub _set_pg_version
# the remainder are# set. Then the PATH and (DY)LD_LIBRARY_PATH are adjusted
# if the node's install path is set, and the copy environment is returned.
#
-# The install path set in get_new_node needs to be a directory containing
+# The install path set in new() needs to be a directory containing
# bin and lib subdirectories as in a standard PostgreSQL installation, so this
# can't be used with installations where the bin and lib directories don't have
# a common parent directory.
@@ -1405,7 +1379,7 @@ sub installed_command
=item get_free_port()
Locate an unprivileged (high) TCP port that's not currently bound to
-anything. This is used by get_new_node, and is also exported for use
+anything. This is used by new(), and is also exported for use
by test cases that need to start other, non-Postgres servers.
Ports assigned to existing PostgresNode objects are automatically
diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl
index df6fdc20d1..ac581c1c07 100644
--- a/src/test/recovery/t/001_stream_rep.pl
+++ b/src/test/recovery/t/001_stream_rep.pl
@@ -9,7 +9,7 @@ use TestLib;
use Test::More tests => 49;
# Initialize primary node
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
# A specific role is created to perform some tests related to replication,
# and it needs proper authentication configuration.
$node_primary->init(
@@ -22,7 +22,7 @@ my $backup_name = 'my_backup';
$node_primary->backup($backup_name);
# Create streaming standby linking to primary
-my $node_standby_1 = get_new_node('standby_1');
+my $node_standby_1 = PostgresNode->new('standby_1');
$node_standby_1->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
$node_standby_1->start;
@@ -37,7 +37,7 @@ $node_standby_1->backup('my_backup_2');
$node_primary->start;
# Create second standby node linking to standby 1
-my $node_standby_2 = get_new_node('standby_2');
+my $node_standby_2 = PostgresNode->new('standby_2');
$node_standby_2->init_from_backup($node_standby_1, $backup_name,
has_streaming => 1);
$node_standby_2->start;
diff --git a/src/test/recovery/t/002_archiving.pl b/src/test/recovery/t/002_archiving.pl
index c675c0886c..ce60159f03 100644
--- a/src/test/recovery/t/002_archiving.pl
+++ b/src/test/recovery/t/002_archiving.pl
@@ -10,7 +10,7 @@ use Test::More tests => 3;
use File::Copy;
# Initialize primary node, doing archives
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(
has_archiving => 1,
allows_streaming => 1);
@@ -23,7 +23,7 @@ $node_primary->start;
$node_primary->backup($backup_name);
# Initialize standby node from backup, fetching WAL from archives
-my $node_standby = get_new_node('standby');
+my $node_standby = PostgresNode->new('standby');
$node_standby->init_from_backup($node_primary, $backup_name,
has_restoring => 1);
$node_standby->append_conf('postgresql.conf',
@@ -62,7 +62,7 @@ is($result, qq(1000), 'check content from archives');
# promoted.
$node_standby->promote;
-my $node_standby2 = get_new_node('standby2');
+my $node_standby2 = PostgresNode->new('standby2');
$node_standby2->init_from_backup($node_primary, $backup_name,
has_restoring => 1);
$node_standby2->start;
diff --git a/src/test/recovery/t/003_recovery_targets.pl b/src/test/recovery/t/003_recovery_targets.pl
index 84e977bd6d..7bd500ed95 100644
--- a/src/test/recovery/t/003_recovery_targets.pl
+++ b/src/test/recovery/t/003_recovery_targets.pl
@@ -21,7 +21,7 @@ sub test_recovery_standby
my $num_rows = shift;
my $until_lsn = shift;
- my $node_standby = get_new_node($node_name);
+ my $node_standby = PostgresNode->new($node_name);
$node_standby->init_from_backup($node_primary, 'my_backup',
has_restoring => 1);
@@ -50,7 +50,7 @@ sub test_recovery_standby
}
# Initialize primary node
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(has_archiving => 1, allows_streaming => 1);
# Bump the transaction ID epoch. This is useful to stress the portability
@@ -136,7 +136,7 @@ test_recovery_standby('LSN', 'standby_5', $node_primary, \@recovery_params,
test_recovery_standby('multiple overriding settings',
'standby_6', $node_primary, \@recovery_params, "3000", $lsn3);
-my $node_standby = get_new_node('standby_7');
+my $node_standby = PostgresNode->new('standby_7');
$node_standby->init_from_backup($node_primary, 'my_backup',
has_restoring => 1);
$node_standby->append_conf(
@@ -156,7 +156,7 @@ ok($logfile =~ qr/multiple recovery targets specified/,
# Check behavior when recovery ends before target is reached
-$node_standby = get_new_node('standby_8');
+$node_standby = PostgresNode->new('standby_8');
$node_standby->init_from_backup(
$node_primary, 'my_backup',
has_restoring => 1,
diff --git a/src/test/recovery/t/004_timeline_switch.pl b/src/test/recovery/t/004_timeline_switch.pl
index c101980e9e..07b1527043 100644
--- a/src/test/recovery/t/004_timeline_switch.pl
+++ b/src/test/recovery/t/004_timeline_switch.pl
@@ -15,7 +15,7 @@ $ENV{PGDATABASE} = 'postgres';
# on a new timeline.
# Initialize primary node
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1);
$node_primary->start;
@@ -24,11 +24,11 @@ my $backup_name = 'my_backup';
$node_primary->backup($backup_name);
# Create two standbys linking to it
-my $node_standby_1 = get_new_node('standby_1');
+my $node_standby_1 = PostgresNode->new('standby_1');
$node_standby_1->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
$node_standby_1->start;
-my $node_standby_2 = get_new_node('standby_2');
+my $node_standby_2 = PostgresNode->new('standby_2');
$node_standby_2->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
$node_standby_2->start;
@@ -76,7 +76,7 @@ is($result, qq(2000), 'check content of standby 2');
# when WAL archiving is enabled.
# Initialize primary node
-my $node_primary_2 = get_new_node('primary_2');
+my $node_primary_2 = PostgresNode->new('primary_2');
$node_primary_2->init(allows_streaming => 1, has_archiving => 1);
$node_primary_2->append_conf(
'postgresql.conf', qq(
@@ -88,7 +88,7 @@ $node_primary_2->start;
$node_primary_2->backup($backup_name);
# Create standby node
-my $node_standby_3 = get_new_node('standby_3');
+my $node_standby_3 = PostgresNode->new('standby_3');
$node_standby_3->init_from_backup($node_primary_2, $backup_name,
has_streaming => 1);
diff --git a/src/test/recovery/t/005_replay_delay.pl b/src/test/recovery/t/005_replay_delay.pl
index bd7ed4e304..aa418dda8f 100644
--- a/src/test/recovery/t/005_replay_delay.pl
+++ b/src/test/recovery/t/005_replay_delay.pl
@@ -10,7 +10,7 @@ use TestLib;
use Test::More tests => 3;
# Initialize primary node
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1);
$node_primary->start;
@@ -23,7 +23,7 @@ my $backup_name = 'my_backup';
$node_primary->backup($backup_name);
# Create streaming standby from backup
-my $node_standby = get_new_node('standby');
+my $node_standby = PostgresNode->new('standby');
my $delay = 3;
$node_standby->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
diff --git a/src/test/recovery/t/006_logical_decoding.pl b/src/test/recovery/t/006_logical_decoding.pl
index 827a7b488e..cc116062c2 100644
--- a/src/test/recovery/t/006_logical_decoding.pl
+++ b/src/test/recovery/t/006_logical_decoding.pl
@@ -14,7 +14,7 @@ use Test::More tests => 14;
use Config;
# Initialize primary node
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1);
$node_primary->append_conf(
'postgresql.conf', qq(
diff --git a/src/test/recovery/t/007_sync_rep.pl b/src/test/recovery/t/007_sync_rep.pl
index 81098dcf00..9d00e17f9f 100644
--- a/src/test/recovery/t/007_sync_rep.pl
+++ b/src/test/recovery/t/007_sync_rep.pl
@@ -49,7 +49,7 @@ sub start_standby_and_wait
}
# Initialize primary node
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1);
$node_primary->start;
my $backup_name = 'primary_backup';
@@ -61,19 +61,19 @@ $node_primary->backup($backup_name);
# the ordering of each one of them in the WAL sender array of the primary.
# Create standby1 linking to primary
-my $node_standby_1 = get_new_node('standby1');
+my $node_standby_1 = PostgresNode->new('standby1');
$node_standby_1->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
start_standby_and_wait($node_primary, $node_standby_1);
# Create standby2 linking to primary
-my $node_standby_2 = get_new_node('standby2');
+my $node_standby_2 = PostgresNode->new('standby2');
$node_standby_2->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
start_standby_and_wait($node_primary, $node_standby_2);
# Create standby3 linking to primary
-my $node_standby_3 = get_new_node('standby3');
+my $node_standby_3 = PostgresNode->new('standby3');
$node_standby_3->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
start_standby_and_wait($node_primary, $node_standby_3);
@@ -123,7 +123,7 @@ standby3|3|sync),
start_standby_and_wait($node_primary, $node_standby_1);
# Create standby4 linking to primary
-my $node_standby_4 = get_new_node('standby4');
+my $node_standby_4 = PostgresNode->new('standby4');
$node_standby_4->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
$node_standby_4->start;
diff --git a/src/test/recovery/t/008_fsm_truncation.pl b/src/test/recovery/t/008_fsm_truncation.pl
index 14b4b97e9e..b1082546a8 100644
--- a/src/test/recovery/t/008_fsm_truncation.pl
+++ b/src/test/recovery/t/008_fsm_truncation.pl
@@ -12,7 +12,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 1;
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1);
$node_primary->append_conf(
@@ -28,7 +28,7 @@ autovacuum = off
$node_primary->start;
$node_primary->backup('primary_backup');
-my $node_standby = get_new_node('standby');
+my $node_standby = PostgresNode->new('standby');
$node_standby->init_from_backup($node_primary, 'primary_backup',
has_streaming => 1);
$node_standby->start;
diff --git a/src/test/recovery/t/009_twophase.pl b/src/test/recovery/t/009_twophase.pl
index 3ee012226d..78d4ef5b54 100644
--- a/src/test/recovery/t/009_twophase.pl
+++ b/src/test/recovery/t/009_twophase.pl
@@ -29,7 +29,7 @@ sub configure_and_reload
# Set up two nodes, which will alternately be primary and replication standby.
# Setup london node
-my $node_london = get_new_node("london");
+my $node_london = PostgresNode->new("london");
$node_london->init(allows_streaming => 1);
$node_london->append_conf(
'postgresql.conf', qq(
@@ -40,7 +40,7 @@ $node_london->start;
$node_london->backup('london_backup');
# Setup paris node
-my $node_paris = get_new_node('paris');
+my $node_paris = PostgresNode->new('paris');
$node_paris->init_from_backup($node_london, 'london_backup',
has_streaming => 1);
$node_paris->start;
diff --git a/src/test/recovery/t/010_logical_decoding_timelines.pl b/src/test/recovery/t/010_logical_decoding_timelines.pl
index 12edbf760e..2a9cf3b79c 100644
--- a/src/test/recovery/t/010_logical_decoding_timelines.pl
+++ b/src/test/recovery/t/010_logical_decoding_timelines.pl
@@ -34,7 +34,7 @@ use Scalar::Util qw(blessed);
my ($stdout, $stderr, $ret);
# Initialize primary node
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1, has_archiving => 1);
$node_primary->append_conf(
'postgresql.conf', q[
@@ -74,7 +74,7 @@ $node_primary->backup_fs_hot($backup_name);
$node_primary->safe_psql('postgres',
q[SELECT pg_create_physical_replication_slot('phys_slot');]);
-my $node_replica = get_new_node('replica');
+my $node_replica = PostgresNode->new('replica');
$node_replica->init_from_backup(
$node_primary, $backup_name,
has_streaming => 1,
diff --git a/src/test/recovery/t/011_crash_recovery.pl b/src/test/recovery/t/011_crash_recovery.pl
index a26e99500b..72fc603e6d 100644
--- a/src/test/recovery/t/011_crash_recovery.pl
+++ b/src/test/recovery/t/011_crash_recovery.pl
@@ -13,7 +13,7 @@ use Config;
plan tests => 3;
-my $node = get_new_node('primary');
+my $node = PostgresNode->new('primary');
$node->init(allows_streaming => 1);
$node->start;
diff --git a/src/test/recovery/t/012_subtransactions.pl b/src/test/recovery/t/012_subtransactions.pl
index aa84073311..a3655a076b 100644
--- a/src/test/recovery/t/012_subtransactions.pl
+++ b/src/test/recovery/t/012_subtransactions.pl
@@ -10,7 +10,7 @@ use TestLib;
use Test::More tests => 12;
# Setup primary node
-my $node_primary = get_new_node("primary");
+my $node_primary = PostgresNode->new("primary");
$node_primary->init(allows_streaming => 1);
$node_primary->append_conf(
'postgresql.conf', qq(
@@ -22,7 +22,7 @@ $node_primary->backup('primary_backup');
$node_primary->psql('postgres', "CREATE TABLE t_012_tbl (id int)");
# Setup standby node
-my $node_standby = get_new_node('standby');
+my $node_standby = PostgresNode->new('standby');
$node_standby->init_from_backup($node_primary, 'primary_backup',
has_streaming => 1);
$node_standby->start;
diff --git a/src/test/recovery/t/013_crash_restart.pl b/src/test/recovery/t/013_crash_restart.pl
index 868a50b33d..b5e3457753 100644
--- a/src/test/recovery/t/013_crash_restart.pl
+++ b/src/test/recovery/t/013_crash_restart.pl
@@ -27,7 +27,7 @@ plan tests => 18;
# is really wrong.
my $psql_timeout = IPC::Run::timer(60);
-my $node = get_new_node('primary');
+my $node = PostgresNode->new('primary');
$node->init(allows_streaming => 1);
$node->start();
diff --git a/src/test/recovery/t/014_unlogged_reinit.pl b/src/test/recovery/t/014_unlogged_reinit.pl
index b629d549dd..4c22663b64 100644
--- a/src/test/recovery/t/014_unlogged_reinit.pl
+++ b/src/test/recovery/t/014_unlogged_reinit.pl
@@ -12,7 +12,7 @@ use PostgresNode;
use TestLib;
use Test::More tests => 12;
-my $node = get_new_node('main');
+my $node = PostgresNode->new('main');
$node->init;
$node->start;
diff --git a/src/test/recovery/t/015_promotion_pages.pl b/src/test/recovery/t/015_promotion_pages.pl
index 2b72dc2334..562c4cd3e4 100644
--- a/src/test/recovery/t/015_promotion_pages.pl
+++ b/src/test/recovery/t/015_promotion_pages.pl
@@ -12,7 +12,7 @@ use TestLib;
use Test::More tests => 1;
# Initialize primary node
-my $alpha = get_new_node('alpha');
+my $alpha = PostgresNode->new('alpha');
$alpha->init(allows_streaming => 1);
# Setting wal_log_hints to off is important to get invalid page
# references.
@@ -25,7 +25,7 @@ $alpha->start;
# setup/start a standby
$alpha->backup('bkp');
-my $bravo = get_new_node('bravo');
+my $bravo = PostgresNode->new('bravo');
$bravo->init_from_backup($alpha, 'bkp', has_streaming => 1);
$bravo->append_conf('postgresql.conf', <<EOF);
checkpoint_timeout=1h
diff --git a/src/test/recovery/t/016_min_consistency.pl b/src/test/recovery/t/016_min_consistency.pl
index 60ecd3dbe7..23f4c42745 100644
--- a/src/test/recovery/t/016_min_consistency.pl
+++ b/src/test/recovery/t/016_min_consistency.pl
@@ -43,7 +43,7 @@ sub find_largest_lsn
}
# Initialize primary node
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(allows_streaming => 1);
# Set shared_buffers to a very low value to enforce discard and flush
@@ -61,7 +61,7 @@ $primary->start;
# setup/start a standby
$primary->backup('bkp');
-my $standby = get_new_node('standby');
+my $standby = PostgresNode->new('standby');
$standby->init_from_backup($primary, 'bkp', has_streaming => 1);
$standby->start;
diff --git a/src/test/recovery/t/017_shm.pl b/src/test/recovery/t/017_shm.pl
index 6fe9b97f6e..5548a1556d 100644
--- a/src/test/recovery/t/017_shm.pl
+++ b/src/test/recovery/t/017_shm.pl
@@ -43,7 +43,7 @@ sub log_ipcs
}
# Node setup.
-my $gnat = PostgresNode->get_new_node('gnat');
+my $gnat = PostgresNode->new('gnat');
$gnat->init;
# Create a shmem segment that will conflict with gnat's first choice
diff --git a/src/test/recovery/t/018_wal_optimize.pl b/src/test/recovery/t/018_wal_optimize.pl
index 53d9864309..4aa1bf8f54 100644
--- a/src/test/recovery/t/018_wal_optimize.pl
+++ b/src/test/recovery/t/018_wal_optimize.pl
@@ -43,7 +43,7 @@ sub run_wal_optimize
{
my $wal_level = shift;
- my $node = get_new_node("node_$wal_level");
+ my $node = PostgresNode->new("node_$wal_level");
$node->init;
$node->append_conf(
'postgresql.conf', qq(
diff --git a/src/test/recovery/t/019_replslot_limit.pl b/src/test/recovery/t/019_replslot_limit.pl
index d4b9ff705f..f421bd7c90 100644
--- a/src/test/recovery/t/019_replslot_limit.pl
+++ b/src/test/recovery/t/019_replslot_limit.pl
@@ -17,7 +17,7 @@ use Time::HiRes qw(usleep);
$ENV{PGDATABASE} = 'postgres';
# Initialize primary node, setting wal-segsize to 1MB
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1, extra => ['--wal-segsize=1']);
$node_primary->append_conf(
'postgresql.conf', qq(
@@ -41,7 +41,7 @@ my $backup_name = 'my_backup';
$node_primary->backup($backup_name);
# Create a standby linking to it using the replication slot
-my $node_standby = get_new_node('standby_1');
+my $node_standby = PostgresNode->new('standby_1');
$node_standby->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
$node_standby->append_conf('postgresql.conf', "primary_slot_name = 'rep1'");
@@ -214,7 +214,7 @@ ok($failed, 'check that replication has been broken');
$node_primary->stop;
$node_standby->stop;
-my $node_primary2 = get_new_node('primary2');
+my $node_primary2 = PostgresNode->new('primary2');
$node_primary2->init(allows_streaming => 1);
$node_primary2->append_conf(
'postgresql.conf', qq(
@@ -235,7 +235,7 @@ max_slot_wal_keep_size = 0
));
$node_primary2->start;
-$node_standby = get_new_node('standby_2');
+$node_standby = PostgresNode->new('standby_2');
$node_standby->init_from_backup($node_primary2, $backup_name,
has_streaming => 1);
$node_standby->append_conf('postgresql.conf', "primary_slot_name = 'rep1'");
diff --git a/src/test/recovery/t/020_archive_status.pl b/src/test/recovery/t/020_archive_status.pl
index 777bf05274..cea65735a3 100644
--- a/src/test/recovery/t/020_archive_status.pl
+++ b/src/test/recovery/t/020_archive_status.pl
@@ -11,7 +11,7 @@ use TestLib;
use Test::More tests => 16;
use Config;
-my $primary = get_new_node('primary');
+my $primary = PostgresNode->new('primary');
$primary->init(
has_archiving => 1,
allows_streaming => 1);
@@ -138,7 +138,7 @@ $primary->poll_query_until('postgres',
or die "Timed out while waiting for archiving to finish";
# Test standby with archive_mode = on.
-my $standby1 = get_new_node('standby');
+my $standby1 = PostgresNode->new('standby');
$standby1->init_from_backup($primary, 'backup', has_restoring => 1);
$standby1->append_conf('postgresql.conf', "archive_mode = on");
my $standby1_data = $standby1->data_dir;
@@ -174,7 +174,7 @@ ok( -f "$standby1_data/$segment_path_2_done",
# command to fail to persist the .ready files. Note that this node
# has inherited the archive command of the previous cold backup that
# will cause archiving failures.
-my $standby2 = get_new_node('standby2');
+my $standby2 = PostgresNode->new('standby2');
$standby2->init_from_backup($primary, 'backup', has_restoring => 1);
$standby2->append_conf('postgresql.conf', 'archive_mode = always');
my $standby2_data = $standby2->data_dir;
diff --git a/src/test/recovery/t/021_row_visibility.pl b/src/test/recovery/t/021_row_visibility.pl
index fcff0a2feb..7f40977976 100644
--- a/src/test/recovery/t/021_row_visibility.pl
+++ b/src/test/recovery/t/021_row_visibility.pl
@@ -12,7 +12,7 @@ use Test::More tests => 10;
use Config;
# Initialize primary node
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(allows_streaming => 1);
$node_primary->append_conf('postgresql.conf', 'max_prepared_transactions=10');
$node_primary->start;
@@ -26,7 +26,7 @@ my $backup_name = 'my_backup';
$node_primary->backup($backup_name);
# Create streaming standby from backup
-my $node_standby = get_new_node('standby');
+my $node_standby = PostgresNode->new('standby');
$node_standby->init_from_backup($node_primary, $backup_name,
has_streaming => 1);
$node_standby->append_conf('postgresql.conf', 'max_prepared_transactions=10');
diff --git a/src/test/recovery/t/022_crash_temp_files.pl b/src/test/recovery/t/022_crash_temp_files.pl
index 157ddba8cf..cc8c8664e2 100644
--- a/src/test/recovery/t/022_crash_temp_files.pl
+++ b/src/test/recovery/t/022_crash_temp_files.pl
@@ -26,7 +26,7 @@ else
# is really wrong.
my $psql_timeout = IPC::Run::timer(60);
-my $node = get_new_node('node_crash');
+my $node = PostgresNode->new('node_crash');
$node->init();
$node->start();
diff --git a/src/test/recovery/t/023_pitr_prepared_xact.pl b/src/test/recovery/t/023_pitr_prepared_xact.pl
index 9190a38f93..a1ff97d070 100644
--- a/src/test/recovery/t/023_pitr_prepared_xact.pl
+++ b/src/test/recovery/t/023_pitr_prepared_xact.pl
@@ -10,7 +10,7 @@ use Test::More tests => 1;
use File::Compare;
# Initialize and start primary node with WAL archiving
-my $node_primary = get_new_node('primary');
+my $node_primary = PostgresNode->new('primary');
$node_primary->init(has_archiving => 1, allows_streaming => 1);
$node_primary->append_conf(
'postgresql.conf', qq{
@@ -24,7 +24,7 @@ $node_primary->backup($backup_name);
# Initialize node for PITR targeting a very specific restore point, just
# after a PREPARE TRANSACTION is issued so as we finish with a promoted
# node where this 2PC transaction needs an explicit COMMIT PREPARED.
-my $node_pitr = get_new_node('node_pitr');
+my $node_pitr = PostgresNode->new('node_pitr');
$node_pitr->init_from_backup(
$node_primary, $backup_name,
standby => 0,
diff --git a/src/test/recovery/t/024_archive_recovery.pl b/src/test/recovery/t/024_archive_recovery.pl
index f06ed8c8a1..32be5c4251 100644
--- a/src/test/recovery/t/024_archive_recovery.pl
+++ b/src/test/recovery/t/024_archive_recovery.pl
@@ -11,7 +11,7 @@ use Time::HiRes qw(usleep);
# Initialize and start node with wal_level = replica and WAL archiving
# enabled.
-my $node = get_new_node('orig');
+my $node = PostgresNode->new('orig');
$node->init(has_archiving => 1, allows_streaming => 1);
my $replica_config = q[
wal_level = replica
@@ -66,7 +66,7 @@ sub test_recovery_wal_level_minimal
{
my ($node_name, $node_text, $standby_setting) = @_;
- my $recovery_node = get_new_node($node_name);
+ my $recovery_node = PostgresNode->new($node_name);
$recovery_node->init_from_backup(
$node, $backup_name,
has_restoring => 1,
diff --git a/src/test/ssl/t/001_ssltests.pl b/src/test/ssl/t/001_ssltests.pl
index 44daefb002..3bc711f4a7 100644
--- a/src/test/ssl/t/001_ssltests.pl
+++ b/src/test/ssl/t/001_ssltests.pl
@@ -64,7 +64,7 @@ push @keys, 'client_wrongperms';
#### Set up the server.
note "setting up data directory";
-my $node = get_new_node('primary');
+my $node = PostgresNode->new('primary');
$node->init;
# PGHOST is enforced here to set up the node, subsequent connections
diff --git a/src/test/ssl/t/002_scram.pl b/src/test/ssl/t/002_scram.pl
index 9143fa515f..1dfa2b91f3 100644
--- a/src/test/ssl/t/002_scram.pl
+++ b/src/test/ssl/t/002_scram.pl
@@ -38,7 +38,7 @@ my $common_connstr;
# Set up the server.
note "setting up data directory";
-my $node = get_new_node('primary');
+my $node = PostgresNode->new('primary');
$node->init;
# PGHOST is enforced here to set up the node, subsequent connections
diff --git a/src/test/subscription/t/001_rep_changes.pl b/src/test/subscription/t/001_rep_changes.pl
index dee5f5c30a..0c84d87873 100644
--- a/src/test/subscription/t/001_rep_changes.pl
+++ b/src/test/subscription/t/001_rep_changes.pl
@@ -9,12 +9,12 @@ use TestLib;
use Test::More tests => 32;
# Initialize publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/002_types.pl b/src/test/subscription/t/002_types.pl
index 96669f869b..420876099a 100644
--- a/src/test/subscription/t/002_types.pl
+++ b/src/test/subscription/t/002_types.pl
@@ -10,12 +10,12 @@ use TestLib;
use Test::More tests => 4;
# Initialize publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/003_constraints.pl b/src/test/subscription/t/003_constraints.pl
index 1182a1288c..3767d24f92 100644
--- a/src/test/subscription/t/003_constraints.pl
+++ b/src/test/subscription/t/003_constraints.pl
@@ -9,12 +9,12 @@ use TestLib;
use Test::More tests => 6;
# Initialize publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/004_sync.pl b/src/test/subscription/t/004_sync.pl
index b3c91af21d..f07c306e5b 100644
--- a/src/test/subscription/t/004_sync.pl
+++ b/src/test/subscription/t/004_sync.pl
@@ -9,12 +9,12 @@ use TestLib;
use Test::More tests => 8;
# Initialize publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->append_conf('postgresql.conf',
"wal_retrieve_retry_interval = 1ms");
diff --git a/src/test/subscription/t/005_encoding.pl b/src/test/subscription/t/005_encoding.pl
index a3f56a452f..9df474beb5 100644
--- a/src/test/subscription/t/005_encoding.pl
+++ b/src/test/subscription/t/005_encoding.pl
@@ -8,13 +8,13 @@ use PostgresNode;
use TestLib;
use Test::More tests => 1;
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(
allows_streaming => 'logical',
extra => [ '--locale=C', '--encoding=UTF8' ]);
$node_publisher->start;
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(
allows_streaming => 'logical',
extra => [ '--locale=C', '--encoding=LATIN1' ]);
diff --git a/src/test/subscription/t/006_rewrite.pl b/src/test/subscription/t/006_rewrite.pl
index 37e05a401a..ec4d641230 100644
--- a/src/test/subscription/t/006_rewrite.pl
+++ b/src/test/subscription/t/006_rewrite.pl
@@ -8,11 +8,11 @@ use PostgresNode;
use TestLib;
use Test::More tests => 2;
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/007_ddl.pl b/src/test/subscription/t/007_ddl.pl
index dd10d5cffa..1a3a1dcf14 100644
--- a/src/test/subscription/t/007_ddl.pl
+++ b/src/test/subscription/t/007_ddl.pl
@@ -8,11 +8,11 @@ use PostgresNode;
use TestLib;
use Test::More tests => 1;
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/008_diff_schema.pl b/src/test/subscription/t/008_diff_schema.pl
index a04a798a18..3af3d25604 100644
--- a/src/test/subscription/t/008_diff_schema.pl
+++ b/src/test/subscription/t/008_diff_schema.pl
@@ -9,12 +9,12 @@ use TestLib;
use Test::More tests => 5;
# Create publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/009_matviews.pl b/src/test/subscription/t/009_matviews.pl
index 92c7d18be8..2e7d0d4413 100644
--- a/src/test/subscription/t/009_matviews.pl
+++ b/src/test/subscription/t/009_matviews.pl
@@ -8,11 +8,11 @@ use PostgresNode;
use TestLib;
use Test::More tests => 1;
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/010_truncate.pl b/src/test/subscription/t/010_truncate.pl
index 5617469a2c..0e6ecf9c2f 100644
--- a/src/test/subscription/t/010_truncate.pl
+++ b/src/test/subscription/t/010_truncate.pl
@@ -10,11 +10,11 @@ use Test::More tests => 14;
# setup
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->append_conf('postgresql.conf',
qq(max_logical_replication_workers = 6));
diff --git a/src/test/subscription/t/011_generated.pl b/src/test/subscription/t/011_generated.pl
index 29108cbcf2..a8e7fbd9da 100644
--- a/src/test/subscription/t/011_generated.pl
+++ b/src/test/subscription/t/011_generated.pl
@@ -10,11 +10,11 @@ use Test::More tests => 2;
# setup
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/012_collation.pl b/src/test/subscription/t/012_collation.pl
index 8137a165ee..0987391188 100644
--- a/src/test/subscription/t/012_collation.pl
+++ b/src/test/subscription/t/012_collation.pl
@@ -18,13 +18,13 @@ else
plan skip_all => 'ICU not supported by this build';
}
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(
allows_streaming => 'logical',
extra => [ '--locale=C', '--encoding=UTF8' ]);
$node_publisher->start;
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(
allows_streaming => 'logical',
extra => [ '--locale=C', '--encoding=UTF8' ]);
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index 3478e4db8f..c89d495221 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -10,15 +10,15 @@ use Test::More tests => 62;
# setup
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
-my $node_subscriber1 = get_new_node('subscriber1');
+my $node_subscriber1 = PostgresNode->new('subscriber1');
$node_subscriber1->init(allows_streaming => 'logical');
$node_subscriber1->start;
-my $node_subscriber2 = get_new_node('subscriber2');
+my $node_subscriber2 = PostgresNode->new('subscriber2');
$node_subscriber2->init(allows_streaming => 'logical');
$node_subscriber2->start;
diff --git a/src/test/subscription/t/014_binary.pl b/src/test/subscription/t/014_binary.pl
index 7260378f5e..4e8aeb2e41 100644
--- a/src/test/subscription/t/014_binary.pl
+++ b/src/test/subscription/t/014_binary.pl
@@ -10,12 +10,12 @@ use TestLib;
use Test::More tests => 5;
# Create and initialize a publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
# Create and initialize subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/015_stream.pl b/src/test/subscription/t/015_stream.pl
index 6cc8b4a8d2..da6761f81e 100644
--- a/src/test/subscription/t/015_stream.pl
+++ b/src/test/subscription/t/015_stream.pl
@@ -9,14 +9,14 @@ use TestLib;
use Test::More tests => 4;
# Create publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->append_conf('postgresql.conf',
'logical_decoding_work_mem = 64kB');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/016_stream_subxact.pl b/src/test/subscription/t/016_stream_subxact.pl
index 0245b0685b..4aae98a66e 100644
--- a/src/test/subscription/t/016_stream_subxact.pl
+++ b/src/test/subscription/t/016_stream_subxact.pl
@@ -9,14 +9,14 @@ use TestLib;
use Test::More tests => 2;
# Create publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->append_conf('postgresql.conf',
'logical_decoding_work_mem = 64kB');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/017_stream_ddl.pl b/src/test/subscription/t/017_stream_ddl.pl
index 35b146827d..50bd6e3117 100644
--- a/src/test/subscription/t/017_stream_ddl.pl
+++ b/src/test/subscription/t/017_stream_ddl.pl
@@ -9,14 +9,14 @@ use TestLib;
use Test::More tests => 3;
# Create publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->append_conf('postgresql.conf',
'logical_decoding_work_mem = 64kB');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/018_stream_subxact_abort.pl b/src/test/subscription/t/018_stream_subxact_abort.pl
index 7fc60b5bde..63fe248a04 100644
--- a/src/test/subscription/t/018_stream_subxact_abort.pl
+++ b/src/test/subscription/t/018_stream_subxact_abort.pl
@@ -9,14 +9,14 @@ use TestLib;
use Test::More tests => 4;
# Create publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->append_conf('postgresql.conf',
'logical_decoding_work_mem = 64kB');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/019_stream_subxact_ddl_abort.pl b/src/test/subscription/t/019_stream_subxact_ddl_abort.pl
index 81149b86a9..176a702486 100644
--- a/src/test/subscription/t/019_stream_subxact_ddl_abort.pl
+++ b/src/test/subscription/t/019_stream_subxact_ddl_abort.pl
@@ -10,14 +10,14 @@ use TestLib;
use Test::More tests => 2;
# Create publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->append_conf('postgresql.conf',
'logical_decoding_work_mem = 64kB');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/020_messages.pl b/src/test/subscription/t/020_messages.pl
index 0e218e0048..ecf9b192a3 100644
--- a/src/test/subscription/t/020_messages.pl
+++ b/src/test/subscription/t/020_messages.pl
@@ -9,13 +9,13 @@ use TestLib;
use Test::More tests => 5;
# Create publisher node
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->append_conf('postgresql.conf', 'autovacuum = off');
$node_publisher->start;
# Create subscriber node
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
diff --git a/src/test/subscription/t/100_bugs.pl b/src/test/subscription/t/100_bugs.pl
index 21eabceb2f..baa4a90771 100644
--- a/src/test/subscription/t/100_bugs.pl
+++ b/src/test/subscription/t/100_bugs.pl
@@ -19,11 +19,11 @@ use Test::More tests => 5;
# fix was to avoid the constant expressions simplification in
# RelationGetIndexAttrBitmap(), so it's safe to call in more contexts.
-my $node_publisher = get_new_node('publisher');
+my $node_publisher = PostgresNode->new('publisher');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
-my $node_subscriber = get_new_node('subscriber');
+my $node_subscriber = PostgresNode->new('subscriber');
$node_subscriber->init(allows_streaming => 'logical');
$node_subscriber->start;
@@ -81,7 +81,7 @@ $node_subscriber->stop('fast');
# identity set before accepting updates. If it did not it would cause
# an error when an update was attempted.
-$node_publisher = get_new_node('publisher2');
+$node_publisher = PostgresNode->new('publisher2');
$node_publisher->init(allows_streaming => 'logical');
$node_publisher->start;
@@ -108,7 +108,7 @@ $node_publisher->stop('fast');
#
# Initial sync doesn't complete; the protocol was not being followed per
# expectations after commit 07082b08cc5d.
-my $node_twoways = get_new_node('twoways');
+my $node_twoways = PostgresNode->new('twoways');
$node_twoways->init(allows_streaming => 'logical');
$node_twoways->start;
for my $db (qw(d1 d2))
@@ -158,15 +158,15 @@ is($node_twoways->safe_psql('d2', "SELECT count(f) FROM t2"),
# Verify table data is synced with cascaded replication setup. This is mainly
# to test whether the data written by tablesync worker gets replicated.
-my $node_pub = get_new_node('testpublisher1');
+my $node_pub = PostgresNode->new('testpublisher1');
$node_pub->init(allows_streaming => 'logical');
$node_pub->start;
-my $node_pub_sub = get_new_node('testpublisher_subscriber');
+my $node_pub_sub = PostgresNode->new('testpublisher_subscriber');
$node_pub_sub->init(allows_streaming => 'logical');
$node_pub_sub->start;
-my $node_sub = get_new_node('testsubscriber1');
+my $node_sub = PostgresNode->new('testsubscriber1');
$node_sub->init(allows_streaming => 'logical');
$node_sub->start;
--
2.25.4
--------------4E4CE6CCDFE521456659ABE6
Content-Type: text/x-patch; charset=UTF-8;
name="0004-Remove-the-last-vestiges-of-Exporter-from-PostgresNo.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="0004-Remove-the-last-vestiges-of-Exporter-from-PostgresNo.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v4 2/8] Address space reservation for shared memory
@ 2024-10-16 18:21 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2024-10-16 18:21 UTC (permalink / raw)
Currently the kernel is responsible to chose an address, where to place each
shared memory mapping, which is the lowest possible address that do not clash
with any other mappings. This is considered to be the most portable approach,
but one of the downsides is that there is no place to resize allocated mappings
anymore. Here is how it looks like for one mapping in /proc/$PID/maps,
/dev/zero represents the anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
...
7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
By specifying the mapping address directly it's possible to place the
mapping in a way that leaves room for resizing. The idea is:
* To reserve some address space via mmap'ing a large chunk of memory
with PROT_NONE and MAP_NORESERVE. This way we prepare a playground for
preparing shared memory layout without risking anything interfering
with that.
* To slice the reserved space up into sections, one to use for each
shared segment.
* Allocate shared memory segments out of corresponding slices and
leaving unclaimed space in between them. This is implemented via
mmap'ing memory at a specified address from the reserved space with
MAP_FIXED.
The result looks like this:
012d9000-0133e000 [heap]
7f443a800000-7f444196c000 /dev/zero (deleted)
7f444196c000-7f470a800000 # reserved space
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
Things like address space randomization should not be a problem in this
context, since the randomization is applied to the mmap base, which is
one per process.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21250648 kB
VmRSS: 22948 kB
RssAnon: 768 kB
RssFile: 10404 kB
RssShmem: 11776 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
17637376 (~16.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
---
src/backend/port/sysv_shmem.c | 284 ++++++++++++++++++++++++----
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/storage/pg_shmem.h | 4 +-
6 files changed, 271 insertions(+), 39 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..a0f03ff868f 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -108,6 +108,66 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping placing (/dev/zero (deleted) below) looks like this:
+ *
+ * 00400000-00490000 /path/bin/postgres
+ * ...
+ * 012d9000-0133e000 [heap]
+ * 7f443a800000-7f470a800000 /dev/zero (deleted)
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * ...
+ * 7f471aef2000-7f471aef9000 /dev/shm/PostgreSQL.3859891842
+ * 7f471aef9000-7f471aefa000 /SYSV007dbf7d (deleted)
+ * ...
+ *
+ * We would like to place multiple mappings in such a way, that there will be
+ * enough space between them in the address space to be able to resize up to
+ * certain size, but without counting towards the total memory consumption.
+ *
+ * To achieve that we first reserve some shared memory address space by
+ * mmap'ing a segment of MaxAvailableMemory size with PROT_NONE and
+ * MAP_NORESERVE (these flags allow to make sure this space will not be used by
+ * anything else, yet do not count against memory limits). Having the reserved
+ * space, we allocate out of it actual chunks of shared memory as usual,
+ * updating a pointer to the current available reserved space for the next
+ * allocation with the gap between segments in mind.
+ *
+ * The result would look like this:
+ *
+ * 012d9000-0133e000 [heap]
+ * 7f4426f54000-7f442e010000 /dev/zero (deleted)
+ * 7f442e010000-7f443a800000 # reserved empty space
+ * 7f443a800000-7f444196c000 /dev/zero (deleted)
+ * 7f444196c000-7f470a800000 # reserved empty space
+ * 7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
+ * 7f4718400000-7f4718401000 /usr/lib64/libicudata.so.74.2
+ * [...]
+ *
+ * The reserved space pointer is calculated to slice up the total reserved
+ * space into fixed fractions of address space for each segment, as specified
+ * in the SHMEM_RESIZE_RATIO array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Offset from the beginning of the reserved space, which indicates currently
+ * available range. New shared memory segments have to be allocated at this
+ * offset related to the reserved space.
+ */
+static Size reserved_offset = 0;
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -626,39 +686,198 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
*
* This function will modify mapping size to the actual size of the allocation,
* if it ends up allocating a segment that is larger than requested.
+ *
+ * Note that we do not switch from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
-CreateAnonymousSegment(AnonymousMapping *mapping)
+CreateAnonymousSegment(AnonymousMapping *mapping, Pointer base)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
int mmap_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS;
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* ReserveAnonymousMemory should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
GetHugePageSize(&hugepagesize, &mmap_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
+ }
+#endif
+
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
+ /*
+ * Try to create mapping at an address out of the reserved range, which
+ * will allow to extend it later. Use reserved_offset to allocate the
+ * segment, then update currently available reserved range.
+ *
+ * If the last step has failed, fallback to the regular mapping
+ * creation and signal that shared buffers could not be resized without
+ * a restart.
+ */
+ ptr = mmap(base + reserved_offset, allocsize, PROT_READ | PROT_WRITE,
+ mmap_flags | MAP_FIXED, -1, 0);
+ mmap_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+ elog(DEBUG1, "segment[%s]: mmap(%zu) at address %p failed: %m, "
+ "fallback to the non-resizable allocation",
+ MappingName(mapping->shmem_segment), allocsize, base + reserved_offset);
+
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
+ PG_MMAP_FLAGS, -1, 0);
+ mmap_errno = errno;
+ }
+ else
+ {
+ Size total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+
+ reserved_offset += total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+ }
+
+ if (ptr == MAP_FAILED)
+ {
+ errno = mmap_errno;
+ DebugMappings();
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment)),
+ (mmap_errno == ENOMEM) ?
+ errhint("This error usually means that PostgreSQL's request "
+ "for a shared memory segment exceeded available memory, "
+ "swap space, or huge pages. To reduce the request size "
+ "(currently %zu bytes), reduce PostgreSQL's shared "
+ "memory usage, perhaps by reducing \"shared_buffers\" or "
+ "\"max_connections\".",
+ allocsize) : 0));
+ }
+
+ mapping->shmem = ptr;
+ mapping->shmem_size = allocsize;
+}
+
+/*
+ * ReserveAnonymousMemory
+ *
+ * Reserve shared memory address space, from which shared memory segments are
+ * going to be sliced out. The goal of this exercise is to support segments
+ * resizing, for which we need a reserved space free of potential clashes with
+ * other mmap'd areas that are not under our control. Reservation is done via
+ * mmap, and will not allocate any memory until it will be actually used, and
+ * MAP_NORESERVE allows to make it not counting againt kernel reservation
+ * limits (e.g. in cgroups or for huge pages). Do not get confused because of
+ * MAP_NORESERVE -- we need to reserve some space, but not the actual memory,
+ * and that is that this flag is about.
+ *
+ * Note, that with MAP_NORESERVE a reservation with hugetlb will succeed even
+ * if there is actually not enough huge pages. Hence this function is
+ * responsible for deciding whether to use huge pages or not. To achieve that
+ * we need to probe first and try to allocate needed memory for all segments --
+ * if this succeeds, we unmap the probe segment and use hugetlb; if it fails,
+ * we proceed with the regular memory.
+ */
+void *
+ReserveAnonymousMemory(Size reserve_size)
+{
+ Size allocsize = reserve_size;
+ void *ptr = MAP_FAILED;
+ int mmap_errno = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ *
+ * We could actually have a mix and match of segments with and without
+ * huge pages. But in that case we need to have multiple reservation
+ * spaces to use corresponding memory (hugetlb adress space reserved
+ * for hugetlb segments, regular memory for others), and it doesn't
+ * seem to worth the complexity for now.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
mmap_errno = errno;
if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
{
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
+ /* No huge pages, we will go with the regular page size */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", total_size);
+ }
+ else
+ {
+ /*
+ * All fine, unmap the temporary segment and proceed with reserving
+ * using huge pages.
+ */
+ if (munmap(ptr, total_size) < 0)
+ elog(LOG, "reservice space: munmap(%p, %zu) failed: %m",
+ ptr, total_size);
+
+ /* Round up the requested size to a suitable large value. */
+ if (allocsize % hugepagesize != 0)
+ allocsize += hugepagesize - (allocsize % hugepagesize);
+
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB",
+ allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | MAP_NORESERVE | mmap_flags,
+ -1, 0);
+ mmap_errno = errno;
+
+ /* This should not happen, but handle errors anyway */
+ if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
+ {
+ elog(DEBUG1, "reserving space: mmap(%zu) with MAP_HUGETLB "
+ "failed, huge pages disabled: %m", allocsize);
+ }
}
}
#endif
@@ -666,10 +885,12 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
/*
* Report whether huge pages are in use. This needs to be tracked before
* the second mmap() call if attempting to use huge pages failed
- * previously.
+ * previously. At this point ptr is either pointing to the probe segment,
+ * if we couldn't mmap it, or the reservation space.
*/
SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
{
@@ -677,10 +898,11 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
* Use the original size, not the rounded-up value, when falling back
* to non-huge pages.
*/
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
+ allocsize = reserve_size;
+
+ elog(DEBUG1, "reserving space: mmap(%zu)", allocsize);
+ ptr = mmap(NULL, allocsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
}
if (ptr == MAP_FAILED)
@@ -688,20 +910,18 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
errno = mmap_errno;
DebugMappings();
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
- MappingName(mapping->shmem_segment)),
+ (errmsg("reserving space: could not map anonymous shared "
+ "memory: %m"),
(mmap_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
- "for a shared memory segment exceeded available memory, "
- "swap space, or huge pages. To reduce the request size "
- "(currently %zu bytes), reduce PostgreSQL's shared "
- "memory usage, perhaps by reducing \"shared_buffers\" or "
- "\"max_connections\".",
+ "for a reserved shared memory address space exceeded "
+ "available memory, swap space, or huge pages. To "
+ "reduce the request reservation size (currently %zu "
+ "bytes), reduce PostgreSQL's \"maximum_shared_buffers\".",
allocsize) : 0));
}
- mapping->shmem = ptr;
- mapping->shmem_size = allocsize;
+ return ptr;
}
/*
@@ -740,7 +960,7 @@ AnonymousShmemDetach(int status, Datum arg)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
@@ -760,14 +980,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -782,7 +994,7 @@ PGSharedMemoryCreate(Size size,
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
/* On success, mapping data will be modified. */
- CreateAnonymousSegment(mapping);
+ CreateAnonymousSegment(mapping, base);
next_free_segment++;
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..ce719f1b412 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -205,7 +205,7 @@ EnableLockPagesPrivilege(int elevel)
*/
PGShmemHeader *
PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim)
+ PGShmemHeader **shim, Pointer base)
{
void *memAddress;
PGShmemHeader *hdr;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..076888c0172 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -203,9 +203,12 @@ CreateSharedMemoryAndSemaphores(void)
PGShmemHeader *seghdr;
Size size;
int numSemas;
+ void *base;
Assert(!IsUnderPostmaster);
+ base = ReserveAnonymousMemory((Size) MaxAvailableMemory * BLCKSZ);
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -217,7 +220,7 @@ CreateSharedMemoryAndSemaphores(void)
*
* XXX: Do multiple shims are needed, one per segment?
*/
- seghdr = PGSharedMemoryCreate(size, &shim);
+ seghdr = PGSharedMemoryCreate(size, &shim, base);
/*
* Make sure that huge pages are never reported as "unknown" while the
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 2152aad97d9..1d42a5856c0 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 131072;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4eaeca89f2c..dede37f7905 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2364,6 +2364,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 131072, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 138078c29c5..4a83e255652 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -60,6 +60,7 @@ extern PGDLLIMPORT ShmemSegment Segments[ANON_MAPPINGS];
extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -100,10 +101,11 @@ extern void PGSharedMemoryNoReAttach(void);
#endif
extern PGShmemHeader *PGSharedMemoryCreate(Size size,
- PGShmemHeader **shim);
+ PGShmemHeader **shim, Pointer base);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+void *ReserveAnonymousMemory(Size reserve_size);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.45.1
--vninua6xybvzgrci
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v4-0003-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
* [PATCH v5 05/10] Address space reservation for shared memory
@ 2025-06-17 09:47 Dmitrii Dolgov <9erthalion6@gmail.com>
0 siblings, 0 replies; 213+ messages in thread
From: Dmitrii Dolgov @ 2025-06-17 09:47 UTC (permalink / raw)
Currently the shared memory layout is designed to pack everything tight
together, leaving no space between mappings for resizing. Here is how it
looks like for one mapping in /proc/$PID/maps, /dev/zero represents the
anonymous shared memory we talk about:
00400000-00490000 /path/bin/postgres
...
012d9000-0133e000 [heap]
7f443a800000-7f470a800000 /dev/zero (deleted)
7f470a800000-7f471831d000 /usr/lib/locale/locale-archive
7f4718400000-7f4718401000 /usr/lib64/libstdc++.so.6.0.34
...
Make the layout more dynamic via splitting every shared memory segment
into two parts:
* An anonymous file, which actually contains shared memory content. Such
an anonymous file is created via memfd_create, it lives in memory,
behaves like a regular file and semantically equivalent to an
anonymous memory allocated via mmap with MAP_ANONYMOUS.
* A reservation mapping, which size is much larger than required shared
segment size. This mapping is created with flags PROT_NONE (which
makes sure the reserved space is not used), and MAP_NORESERVE (to not
count the reserved space against memory limits). The anonymous file is
mapped into this reservation mapping.
The resulting layout looks like this:
00400000-00490000 /path/bin/postgres
...
3f526000-3f590000 rw-p [heap]
7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted) -- anon file
7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted) -- reservation
7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
To resize a shared memory segment in this layout it's possible to use ftruncate
on the anonymous file, adjusting access permissions on the reserved space as
needed.
This approach also do not impact the actual memory usage as reported by
the kernel. Here is the output of /proc/$PID/status for the master
version with shared_buffers = 128 MB:
// Peak virtual memory size, which is described as total pages
// mapped in mm_struct. It corresponds to the mapped reserved space
// and is the only number that grows with it.
VmPeak: 2043192 kB
// Size of memory portions. It contains RssAnon + RssFile + RssShmem
VmRSS: 22908 kB
// Size of resident anonymous memory
RssAnon: 768 kB
// Size of resident file mappings
RssFile: 10364 kB
// Size of resident shmem memory (includes SysV shm, mapping of tmpfs and
// shared anonymous mappings)
RssShmem: 11776 kB
Here is the same for the patch when reserving 20GB of space:
VmPeak: 21255824 kB
VmRSS: 25020 kB
RssAnon: 768 kB
RssFile: 10812 kB
RssShmem: 13440 kB
Cgroup v2 doesn't have any problems with that as well. To verify a new cgroup
was created with the memory limit 256 MB, then PostgreSQL was launched withing
this cgroup with shared_buffers = 128 MB:
$ cd /sys/fs/cgroup
$ mkdir postgres
$ cd postres
$ echo 268435456 > memory.max
$ echo $MASTER_PID_SHELL > cgroup.procs
# postgres from the master branch has being successfully launched
# from that shell
$ cat memory.current
17465344 (~16.6 MB)
# stop postgres
$ echo $PATCH_PID_SHELL > cgroup.procs
# postgres from the patch has being successfully launched from that shell
$ cat memory.current
20770816 (~19.8 MB)
To control the amount of space reserved a new GUC max_available_memory
is introduced. Ideally it should be based on the maximum available
memory, hense the name.
There are also few unrelated advantages of using anon files:
* We've got a file descriptor, which could be used for regular file
operations (modification, truncation, you name it).
* The file could be given a name, which improves readability when it
comes to process maps.
* By default, Linux will not add file-backed shared mappings into a core dump,
making it more convenient to work with them in PostgreSQL: no more huge dumps
to process.
The downside is that memfd_create is Linux specific.
---
src/backend/port/sysv_shmem.c | 290 ++++++++++++++++++++++------
src/backend/port/win32_shmem.c | 2 +-
src/backend/storage/ipc/ipci.c | 5 +-
src/backend/storage/ipc/shmem.c | 2 +-
src/backend/utils/init/globals.c | 1 +
src/backend/utils/misc/guc_tables.c | 14 ++
src/include/miscadmin.h | 1 +
src/include/portability/mem.h | 2 +-
src/include/storage/pg_shmem.h | 5 +-
9 files changed, 262 insertions(+), 60 deletions(-)
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 56af0231d24..363ddfd1fca 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -97,10 +97,12 @@ void *UsedShmemSegAddr = NULL;
typedef struct AnonymousMapping
{
int shmem_segment;
- Size shmem_size; /* Size of the mapping */
+ Size shmem_size; /* Size of the actually used memory */
+ Size shmem_reserved; /* Size of the reserved mapping */
Pointer shmem; /* Pointer to the start of the mapped memory */
Pointer seg_addr; /* SysV shared memory for the header */
unsigned long seg_id; /* IPC key */
+ int segment_fd; /* fd for the backing anon file */
} AnonymousMapping;
static AnonymousMapping Mappings[ANON_MAPPINGS];
@@ -108,6 +110,49 @@ static AnonymousMapping Mappings[ANON_MAPPINGS];
/* Keeps track of used mapping segments */
static int next_free_segment = 0;
+/*
+ * Anonymous mapping layout we use looks like this:
+ *
+ * 00400000-00c2a000 r-xp /bin/postgres
+ * ...
+ * 3f526000-3f590000 rw-p [heap]
+ * 7fbd827fe000-7fbd8bdde000 rw-s /memfd:main (deleted)
+ * 7fbd8bdde000-7fbe82800000 ---s /memfd:main (deleted)
+ * 7fbe82800000-7fbe90670000 r--p /usr/lib/locale/locale-archive
+ * 7fbe90800000-7fbe90941000 r-xp /usr/lib64/libstdc++.so.6.0.34
+ * ...
+ *
+ * We need to place shared memory mappings in such a way, that there will be
+ * gaps between them in the address space. Those gaps have to be large enough
+ * to resize the mapping up to certain size, without counting towards the total
+ * memory consumption.
+ *
+ * To achieve this, for each shared memory segment we first create an anonymous
+ * file of specified size using memfd_create, which will accomodate actual
+ * shared memory mapping content. It is represented by the first /memfd:main
+ * with rw permissions. Then we create a mapping for this file using mmap, with
+ * size much larger than required and flags PROT_NONE (allows to make sure the
+ * reserved space will not be used) and MAP_NORESERVE (prevents the space from
+ * being counted against memory limits). The mapping serves as an address space
+ * reservation, into which shared memory segment can be extended and is
+ * represented by the second /memfd:main with no permissions.
+ *
+ * The reserved space for each segment is calculated as a fraction of the total
+ * reserved space (MaxAvailableMemory), as specified in the SHMEM_RESIZE_RATIO
+ * array.
+ */
+static double SHMEM_RESIZE_RATIO[1] = {
+ 1.0, /* MAIN_SHMEM_SLOT */
+};
+
+/*
+ * Flag telling that we have decided to use huge pages.
+ *
+ * XXX: It's possible to use GetConfigOption("huge_pages_status", false, false)
+ * instead, but it feels like an overkill.
+ */
+static bool huge_pages_on = false;
+
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
@@ -503,19 +548,20 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
* hugepage sizes, we might want to think about more invasive strategies,
* such as increasing shared_buffers to absorb the extra space.
*
- * Returns the (real, assumed or config provided) page size into
- * *hugepagesize, and the hugepage-related mmap flags to use into
- * *mmap_flags if requested by the caller. If huge pages are not supported,
- * *hugepagesize and *mmap_flags are set to 0.
+ * Returns the (real, assumed or config provided) page size into *hugepagesize,
+ * the hugepage-related mmap and memfd flags to use into *mmap_flags and
+ * *memfd_flags if requested by the caller. If huge pages are not supported,
+ * *hugepagesize, *mmap_flags and *memfd_flags are set to 0.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
#ifdef MAP_HUGETLB
Size default_hugepagesize = 0;
Size hugepagesize_local = 0;
int mmap_flags_local = 0;
+ int memfd_flags_local = 0;
/*
* System-dependent code to find out the default huge page size.
@@ -574,6 +620,7 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
}
mmap_flags_local = MAP_HUGETLB;
+ memfd_flags_local = MFD_HUGETLB;
/*
* On recent enough Linux, also include the explicit page size, if
@@ -584,7 +631,16 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
{
int shift = pg_ceil_log2_64(hugepagesize_local);
- mmap_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
+ }
+#endif
+
+#if defined(MFD_HUGE_MASK) && defined(MFD_HUGE_SHIFT)
+ if (hugepagesize_local != default_hugepagesize)
+ {
+ int shift = pg_ceil_log2_64(hugepagesize_local);
+
+ memfd_flags_local |= (shift & MAP_HUGE_MASK) << MAP_HUGE_SHIFT;
}
#endif
@@ -593,6 +649,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*mmap_flags = mmap_flags_local;
if (hugepagesize)
*hugepagesize = hugepagesize_local;
+ if (memfd_flags)
+ *memfd_flags = memfd_flags_local;
#else
@@ -600,6 +658,8 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
*hugepagesize = 0;
if (mmap_flags)
*mmap_flags = 0;
+ if (memfd_flags)
+ *memfd_flags = 0;
#endif /* MAP_HUGETLB */
}
@@ -625,72 +685,90 @@ check_huge_page_size(int *newval, void **extra, GucSource source)
* Creates an anonymous mmap()ed shared memory segment.
*
* This function will modify mapping size to the actual size of the allocation,
- * if it ends up allocating a segment that is larger than requested.
+ * if it ends up allocating a segment that is larger than requested. If needed,
+ * it also rounds up the mapping reserved size to be a multiple of huge page
+ * size.
+ *
+ * Note that we do not fallback from huge pages to regular pages in this
+ * function, this decision was already made in ReserveAnonymousMemory and we
+ * stick to it.
*/
static void
CreateAnonymousSegment(AnonymousMapping *mapping)
{
Size allocsize = mapping->shmem_size;
void *ptr = MAP_FAILED;
- int mmap_errno = 0;
+ int save_errno = 0;
+ int mmap_flags = PG_MMAP_FLAGS, memfd_flags = 0;
+
+ elog(DEBUG1, "segment[%s]: size %zu, reserved %zu",
+ MappingName(mapping->shmem_segment), mapping->shmem_size,
+ mapping->shmem_reserved);
#ifndef MAP_HUGETLB
- /* PGSharedMemoryCreate should have dealt with this case */
- Assert(huge_pages != HUGE_PAGES_ON);
+ /* PrepareHugePages should have dealt with this case */
+ Assert(huge_pages != HUGE_PAGES_ON && !huge_pages_on);
#else
- if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ if (huge_pages_on)
{
- /*
- * Round up the request size to a suitable large value.
- */
Size hugepagesize;
- int mmap_flags;
- GetHugePageSize(&hugepagesize, &mmap_flags);
+ /* Make sure nothing is messed up */
+ Assert(huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY);
+
+ /* Round up the request size to a suitable large value */
+ GetHugePageSize(&hugepagesize, &mmap_flags, &memfd_flags);
if (allocsize % hugepagesize != 0)
allocsize += hugepagesize - (allocsize % hugepagesize);
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS | mmap_flags, -1, 0);
- mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
- {
- DebugMappings();
- elog(DEBUG1, "segment[%s]: mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
- MappingName(mapping->shmem_segment), allocsize);
- }
+ /*
+ * The reserved space is multiple of BLCKSZ. We know the huge page
+ * size, round up the reserved space to it.
+ */
+ mapping->shmem_reserved = mapping->shmem_reserved + hugepagesize -
+ (mapping->shmem_reserved % hugepagesize);
+
+ /* Verify that the new size is withing the reserved boundaries */
+ if (mapping->shmem_reserved < mapping->shmem_size)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_RESOURCES),
+ errmsg("not enough shared memory is reserved"),
+ errhint("You may need to increase \"max_available_memory\".")));
+
+ mmap_flags = PG_MMAP_FLAGS | mmap_flags;
}
#endif
/*
- * Report whether huge pages are in use. This needs to be tracked before
- * the second mmap() call if attempting to use huge pages failed
- * previously.
+ * Prepare an anonymous file backing the segment. Its size will be
+ * specified later via ftruncate.
+ *
+ * The file behaves like a regular file, but lives in memory. Once all
+ * references to the file are dropped, it is automatically released.
+ * Anonymous memory is used for all backing pages of the file, thus it has
+ * the same semantics as anonymous memory allocations using mmap with the
+ * MAP_ANONYMOUS flag.
*/
- SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
- PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ mapping->segment_fd = memfd_create(MappingName(mapping->shmem_segment),
+ memfd_flags);
- if (ptr == MAP_FAILED && huge_pages != HUGE_PAGES_ON)
+ /*
+ * Specify the segment file size using allocsize, which contains
+ * potentially modified value.
+ */
+ if(ftruncate(mapping->segment_fd, allocsize) == -1)
{
- /*
- * Use the original size, not the rounded-up value, when falling back
- * to non-huge pages.
- */
- allocsize = mapping->shmem_size;
- ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
- PG_MMAP_FLAGS, -1, 0);
- mmap_errno = errno;
- }
+ save_errno = errno;
- if (ptr == MAP_FAILED)
- {
- errno = mmap_errno;
DebugMappings();
+ close(mapping->segment_fd);
+
+ errno = save_errno;
ereport(FATAL,
- (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ (errmsg("segment[%s]: could not truncate anonymous file: %m",
MappingName(mapping->shmem_segment)),
- (mmap_errno == ENOMEM) ?
+ (save_errno == ENOMEM) ?
errhint("This error usually means that PostgreSQL's request "
"for a shared memory segment exceeded available memory, "
"swap space, or huge pages. To reduce the request size "
@@ -700,10 +778,112 @@ CreateAnonymousSegment(AnonymousMapping *mapping)
allocsize) : 0));
}
+ elog(DEBUG1, "segment[%s]: mmap(%zu)",
+ MappingName(mapping->shmem_segment), allocsize);
+
+ /*
+ * Create a reservation mapping.
+ */
+ ptr = mmap(NULL, mapping->shmem_reserved, PROT_NONE,
+ mmap_flags | MAP_NORESERVE, mapping->segment_fd, 0);
+ save_errno = errno;
+
+ if (ptr == MAP_FAILED)
+ {
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not map anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
+ /* Make the memory accessible */
+ if(mprotect(ptr, allocsize, PROT_READ | PROT_WRITE) == -1)
+ {
+ save_errno = errno;
+ DebugMappings();
+
+ errno = save_errno;
+ ereport(FATAL,
+ (errmsg("segment[%s]: could not mprotect anonymous shared memory: %m",
+ MappingName(mapping->shmem_segment))));
+ }
+
mapping->shmem = ptr;
mapping->shmem_size = allocsize;
}
+/*
+ * PrepareHugePages
+ *
+ * Figure out if there are enough huge pages to allocate all shared memory
+ * segments, and report that information via huge_pages_status and
+ * huge_pages_on. It needs to be called before creating shared memory segments.
+ *
+ * It is necessary to maintain the same semantic (simple on/off) for
+ * huge_pages_status, even if there are multiple shared memory segments: all
+ * segments either use huge pages or not, there is no mix of segments with
+ * different page size. The latter might be actually beneficial, in particular
+ * because only some segments may require large amount of memory, but for now
+ * we go with a simple solution.
+ */
+void
+PrepareHugePages()
+{
+ void *ptr = MAP_FAILED;
+
+ /* Reset to handle reinitialization */
+ next_free_segment = 0;
+
+ /* Complain if hugepages demanded but we can't possibly support them */
+#if !defined(MAP_HUGETLB)
+ if (huge_pages == HUGE_PAGES_ON)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("huge pages not supported on this platform")));
+#else
+ if (huge_pages == HUGE_PAGES_ON || huge_pages == HUGE_PAGES_TRY)
+ {
+ Size hugepagesize, total_size = 0;
+ int mmap_flags;
+
+ GetHugePageSize(&hugepagesize, &mmap_flags, NULL);
+
+ /*
+ * Figure out how much memory is needed for all segments, keeping in
+ * mind that for every segment this value will be rounding up by the
+ * huge page size. The resulting value will be used to probe memory and
+ * decide whether we will allocate huge pages or not.
+ */
+ for(int segment = 0; segment < ANON_MAPPINGS; segment++)
+ {
+ int numSemas;
+ Size segment_size = CalculateShmemSize(&numSemas, segment);
+
+ if (segment_size % hugepagesize != 0)
+ segment_size += hugepagesize - (segment_size % hugepagesize);
+
+ total_size += segment_size;
+ }
+
+ /* Map total amount of memory to test its availability. */
+ elog(DEBUG1, "reserving space: probe mmap(%zu) with MAP_HUGETLB",
+ total_size);
+ ptr = mmap(NULL, total_size, PROT_NONE,
+ PG_MMAP_FLAGS | MAP_ANONYMOUS | mmap_flags, -1, 0);
+ }
+#endif
+
+ /*
+ * Report whether huge pages are in use. This needs to be tracked before
+ * creating shared memory segments.
+ */
+ SetConfigOption("huge_pages_status", (ptr == MAP_FAILED) ? "off" : "on",
+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+ huge_pages_on = ptr != MAP_FAILED;
+}
+
/*
* AnonymousShmemDetach --- detach from an anonymous mmap'd block
* (called as an on_shmem_exit callback, hence funny argument list)
@@ -746,7 +926,7 @@ PGSharedMemoryCreate(Size size,
void *memAddress;
PGShmemHeader *hdr;
struct stat statbuf;
- Size sysvsize;
+ Size sysvsize, total_reserved;
AnonymousMapping *mapping = &Mappings[next_free_segment];
/*
@@ -760,14 +940,6 @@ PGSharedMemoryCreate(Size size,
errmsg("could not stat data directory \"%s\": %m",
DataDir)));
- /* Complain if hugepages demanded but we can't possibly support them */
-#if !defined(MAP_HUGETLB)
- if (huge_pages == HUGE_PAGES_ON)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("huge pages not supported on this platform")));
-#endif
-
/* For now, we don't support huge pages in SysV memory */
if (huge_pages == HUGE_PAGES_ON && shared_memory_type != SHMEM_TYPE_MMAP)
ereport(ERROR,
@@ -776,8 +948,16 @@ PGSharedMemoryCreate(Size size,
/* Room for a header? */
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
+
+ /* Prepare the mapping information */
mapping->shmem_size = size;
mapping->shmem_segment = next_free_segment;
+ total_reserved = (Size) MaxAvailableMemory * BLCKSZ;
+ mapping->shmem_reserved = total_reserved * SHMEM_RESIZE_RATIO[next_free_segment];
+
+ /* Round up to be a multiple of BLCKSZ */
+ mapping->shmem_reserved = mapping->shmem_reserved + BLCKSZ -
+ (mapping->shmem_reserved % BLCKSZ);
if (shared_memory_type == SHMEM_TYPE_MMAP)
{
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 4dee856d6bd..732fedee87e 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -627,7 +627,7 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
* use GetLargePageMinimum() instead.
*/
void
-GetHugePageSize(Size *hugepagesize, int *mmap_flags)
+GetHugePageSize(Size *hugepagesize, int *mmap_flags, int *memfd_flags)
{
if (hugepagesize)
*hugepagesize = 0;
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8b38e985327..b60f7ef9ce2 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -206,6 +206,9 @@ CreateSharedMemoryAndSemaphores(void)
Assert(!IsUnderPostmaster);
+ /* Decide if we use huge pages or regular size pages */
+ PrepareHugePages();
+
for(int segment = 0; segment < ANON_MAPPINGS; segment++)
{
/* Compute the size of the shared-memory block */
@@ -377,7 +380,7 @@ InitializeShmemGUCs(void)
/*
* Calculate the number of huge pages required.
*/
- GetHugePageSize(&hp_size, NULL);
+ GetHugePageSize(&hp_size, NULL, NULL);
if (hp_size != 0)
{
Size hp_required;
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 72255a1c5ca..8d025f0e907 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -817,7 +817,7 @@ pg_get_shmem_pagesize(void)
Assert(huge_pages_status != HUGE_PAGES_UNKNOWN);
if (huge_pages_status == HUGE_PAGES_ON)
- GetHugePageSize(&os_page_size, NULL);
+ GetHugePageSize(&os_page_size, NULL, NULL);
return os_page_size;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index d31cb45a058..90d3feb547c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -140,6 +140,7 @@ int max_parallel_maintenance_workers = 2;
* register background workers.
*/
int NBuffers = 16384;
+int MaxAvailableMemory = 524288;
int MaxConnections = 100;
int max_worker_processes = 8;
int max_parallel_workers = 8;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f04bfedb2fd..a221e446d6a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -2376,6 +2376,20 @@ struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"max_available_memory", PGC_SIGHUP, RESOURCES_MEM,
+ gettext_noop("Sets the upper limit for the shared_buffers value."),
+ gettext_noop("Shared memory could be resized at runtime, this "
+ "parameters sets the upper limit for it, beyond which "
+ "resizing would not be supported. Normally this value "
+ "would be the same as the total available memory."),
+ GUC_UNIT_BLOCKS
+ },
+ &MaxAvailableMemory,
+ 524288, 16, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"vacuum_buffer_usage_limit", PGC_USERSET, RESOURCES_MEM,
gettext_noop("Sets the buffer pool size for VACUUM, ANALYZE, and autovacuum."),
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1bef98471c3..a0c37a7749e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -173,6 +173,7 @@ extern PGDLLIMPORT char *DataDir;
extern PGDLLIMPORT int data_directory_mode;
extern PGDLLIMPORT int NBuffers;
+extern PGDLLIMPORT int MaxAvailableMemory;
extern PGDLLIMPORT int MaxBackends;
extern PGDLLIMPORT int MaxConnections;
extern PGDLLIMPORT int max_worker_processes;
diff --git a/src/include/portability/mem.h b/src/include/portability/mem.h
index ef9800732d9..40588ff6968 100644
--- a/src/include/portability/mem.h
+++ b/src/include/portability/mem.h
@@ -38,7 +38,7 @@
#define MAP_NOSYNC 0
#endif
-#define PG_MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_HASSEMAPHORE)
+#define PG_MMAP_FLAGS (MAP_SHARED|MAP_HASSEMAPHORE)
/* Some really old systems don't define MAP_FAILED. */
#ifndef MAP_FAILED
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 2348c59b5a0..79b0b1ef9eb 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -61,6 +61,7 @@ extern PGDLLIMPORT int shared_memory_type;
extern PGDLLIMPORT int huge_pages;
extern PGDLLIMPORT int huge_page_size;
extern PGDLLIMPORT int huge_pages_status;
+extern PGDLLIMPORT int MaxAvailableMemory;
/* Possible values for huge_pages and huge_pages_status */
typedef enum
@@ -104,7 +105,9 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
-extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags);
+extern void GetHugePageSize(Size *hugepagesize, int *mmap_flags,
+ int *memfd_flags);
+void PrepareHugePages(void);
/* The main segment, contains everything except buffer blocks and related data. */
#define MAIN_SHMEM_SEGMENT 0
--
2.49.0
--bwrlgp6w2ubxykjq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v5-0006-Introduce-multiple-shmem-segments-for-shared-buff.patch"
^ permalink raw reply [nested|flat] 213+ messages in thread
end of thread, other threads:[~2025-06-17 09:47 UTC | newest]
Thread overview: 213+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-05-19 22:11 [PATCH 3/9] Unify PostgresNode's new() and get_new_node() methods Andrew Dunstan <andrew@dunslane.net>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2024-10-16 18:21 [PATCH v4 2/8] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
2025-06-17 09:47 [PATCH v5 05/10] Address space reservation for shared memory Dmitrii Dolgov <9erthalion6@gmail.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox