Received: from localhost (unknown [200.46.204.184]) by postgresql.org (Postfix) with ESMTP id 5CF0E9FCBA1 for ; Sun, 11 Nov 2007 04:46:52 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 28886-03 for ; Sun, 11 Nov 2007 04:46:43 -0400 (AST) Received: from mail.clickspace.com (torch1.clickspace.com [207.148.169.68]) by postgresql.org (Postfix) with ESMTP id 654C79FCBE3 for ; Sun, 11 Nov 2007 04:46:28 -0400 (AST) Received: from [10.0.1.198] ([68.147.223.171]) (authenticated user brendan@clickspace.com) by mail.clickspace.com (using TLSv1/SSLv3 with cipher AES128-SHA (128 bits)); Sun, 11 Nov 2007 01:46:22 -0700 Cc: Maximilian Tyrtania , "pgsql-admin@postgresql.org" Message-Id: <8B947BD2-11D8-42CF-A14E-DCB895C7551E@clickspace.com> From: Brendan Duddridge To: Tom Lane In-Reply-To: <4731.1193665210@sss.pgh.pa.us> Content-Type: multipart/signed; boundary=Apple-Mail-3--432267045; micalg=sha1; protocol="application/pkcs7-signature" Mime-Version: 1.0 (Apple Message framework v912) Subject: Re: shared memory settings on MAC OS X Date: Sun, 11 Nov 2007 01:46:21 -0700 References: <4731.1193665210@sss.pgh.pa.us> X-Mailer: Apple Mail (2.912) X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200711/90 X-Sequence-Number: 27333 --Apple-Mail-3--432267045 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Either way, when you do manage to change your shared memory settings, you still won't be able to use more than about 4 GB of shared buffer space in Postgres, or any other program for that matter. We thought Leopard would help with this, but it hasn't. We have a 32 GB dual 3 Ghz Intel Xeon Xserve, but we can't set our shared buffers as high as we'd like. We also have a similarly spec'd Linux machine and it has no problem allowing us to specify lots of shared buffers. For some reason, OS X gives an error when you try to attach to anything more than 4 GB. Here's a program we wrote to demonstrate the problem, which we have submitted to Apple as a bug (Radar #5547628): /*** create_shared_memory.c ***/ /* Takes as input from standard in a size in bytes for a shared memory segment that this program will attempt to create and attach to. If any errors occur trying to create or attach to the shared memory, they are printed to standard out. This program will then attempt to detach from and delete the shared memory it allocated and/or attached to as appropriate. On Mac OS X 10.5 Server (9A557) on an Intel Quad core Xserve, compiling this program with gcc version i686-apple-darwin9-gcc-4.0.1 as follows #gcc -arch x86_64 create_shared_memory.c you will see the maximum shared memory segment that can be attached to is 4292870144. I do not know if this shared memory limit applies to the final version of Leopard Server or on other hardware. Note: You will need greater than 4GB of memory on your machine and large enough values set for kern.sysv.shmall and kern.sysv.shmmax to verify this shared memory limit. Known limitations: This program does not provide any output if the allocated shared memory cannot be deallocated or if the attached shared memory cannot be detached. Written by: Brian Wipf ClickSpace Interactive Inc. Date: October 18, 2007 */ #include #include #include #include #include typedef struct { int iPID; } shared_t; main() { int key = 912367102; /* Some random key for the shared memory segment. */ size_t size; int shmflg=IPC_CREAT | 0600; int shmid; shared_t *shared_mem; /* If you want to specify your own key, uncomment the next two lines of code. */ /* printf("Enter an integer key for shared memory segment: "); scanf("%d", &key); */ printf("Enter size for shared memory segment (in bytes): "); scanf("%ld", &size); /* Create the shared memory segment */ printf("\nAttempting to create the shared memory segment...\n"); shmid=shmget(key, size, shmflg); if (errno != 0) { printf("There was an error trying to allocate the requested amount of shared memory\n"); printf("errno: %d\n", errno); } else { printf("Shared memory created successfully\n"); /* Attach to the shared memory segment */ printf("\nAttempting to attach to the shared memory segment... \n"); shared_mem = (shared_t *) shmat ( shmid, 0, 0); if (errno != 0) { printf("Error trying to attach to the shared memory segment. \n"); printf("errno: %d\n\n", errno); } else { printf("Shared memory attached to successfully\n"); /* Detach from shared memory segment */ printf("\nDetaching from the shared memory segment\n\n"); shmdt(shared_mem); } /* Delete the shared memory segment */ printf("Deleting the shared memory segment\n"); shmctl(shmid, IPC_RMID, NULL); } return 0; } ____________________________________________________________________ Brendan Duddridge | CTO | 403-277-5591 x24 | brendan@clickspace.com ClickSpace Interactive Inc. Suite L100, 239 - 10th Ave. SE Calgary, AB T2G 0V9 http://www.clickspace.com On Oct 29, 2007, at 7:40 AM, Tom Lane wrote: > Maximilian Tyrtania writes: >> Tom, I suspect you have Mac OS X Server installed, right? That's >> probably >> why your /etc/sysctl.conf file mentions that /etc/sysctl- >> macosxserver.conf >> file, while mine doesn't. > > Uh, no, I'm looking at my laptop. Curious that yours has no reference > to the other file. > >> And frankly, to me it looks as if it means "if there is a /etc/ >> sysctl.conf >> file, then read it and accept its settings. Then overwrite the sysctl >> settings with the default values, no matter what." > > You're forgetting the point I made that the first complete set of > shmem > settings wins. If we could change the settings on the fly after that, > all this would be a whole lot easier, but the OSX kernel locks them > down > somehow. > > BTW, I dunno if you read awk at all, but that awk command effectively > says "print lines that contain = and do not contain #". You didn't > try > appending comments to the setting lines in your file did you? > > regards, tom lane > > ---------------------------(end of > broadcast)--------------------------- > TIP 2: Don't 'kill -9' the postmaster > --Apple-Mail-3--432267045 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGLTCCAuYw ggJPoAMCAQICEEW/yqG9v5vC6/ZX+WWZEAowDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UEBhMCWkEx JTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQ ZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MTIwNjAwMTk1MFoXDTA3MTIwNjAwMTk1 MFowSDEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjElMCMGCSqGSIb3DQEJARYWYnJl bmRhbkBjbGlja3NwYWNlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANJqEPAB NJUnGcb8NSmp3dDmO+4Tg/5zUJ7lFjxMra4+v36huV6i7QC968c7ajVRrUtkEx+j6WgcjBzuNsMo mGiC9NBOsN2w+TTa37YtjfkgqUF5EUG/FoFmF5CRZbSz9HIyn5DQFYrCCXb9y3O/LkfKOEgooYs5 u0XOVWFrcHk3YIFIFAbZaiocdCOKSBMR0I8uvcUbM8ZsV3sphdczdGrP9eXfpugrsn65ksXpmHuU HyEhqOpUwmu25WqY9JgNWWdOaAWu2T0s9RIFb6VM5hcCWJwaQkZXHzpPXN+Q0OUBmK4GAJ0MmBQz 8G/+euTX7pp8dChDwQrzdcLMDWhjoXUCAwEAAaMzMDEwIQYDVR0RBBowGIEWYnJlbmRhbkBjbGlj a3NwYWNlLmNvbTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBQUAA4GBACYCarc7PUs9uxZ2P9WY T2S9ykWDywn9P2HdN4HsfaszMcraxGdp8X5YuKQamsuq8hYgRH6z+hb+nXrwhCnQcbCT64QhG3gg LN5jm3g86MjhVCwDTFaAbz+6EBVqXBpg/iFYqD6AeDMVjsdbMVxajKBC+ucJcMo0RPQ5Xj1eipfU MIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0BAQUFADCB0TELMAkGA1UEBhMCWkExFTATBgNVBAgT DFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMRowGAYDVQQKExFUaGF3dGUgQ29uc3Vs dGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMb VGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENBMSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVl bWFpbEB0aGF3dGUuY29tMB4XDTAzMDcxNzAwMDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1Ro YXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCB iQKBgQDEpjxVc1X7TrnKmVoeaMB1BHCd3+n/ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1os iRVwjt3J8CuFWqo/cVbLrzwLB+fxH5E2JCoTzyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpU hQ9IBH+nttE8YQRAHmQZcmC3+wIDAQABo4GUMIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0f BDwwOjA4oDagNIYyaHR0cDovL2NybC50aGF3dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxD QS5jcmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwy LTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQg+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4Nh+q LZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsAxRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowgT2Vfldr3 94fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXeJLHTHUb/XV9lTzGCAxAwggMMAgEBMHYwYjELMAkG A1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhBFv8qhvb+bwuv2V/llmRAKMAkG BSsOAwIaBQCgggFvMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTA3 MTExMTA4NDYyMVowIwYJKoZIhvcNAQkEMRYEFJZSfB3S0ISrvVJOZpoQYQDq4GFgMIGFBgkrBgEE AYI3EAQxeDB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5 KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQRb/K ob2/m8Lr9lf5ZZkQCjCBhwYLKoZIhvcNAQkQAgsxeKB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQK ExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg RnJlZW1haWwgSXNzdWluZyBDQQIQRb/Kob2/m8Lr9lf5ZZkQCjANBgkqhkiG9w0BAQEFAASCAQCe 0Qu7K/MrcjsluPbUmSMG8fYr1O+4mXzDfEeWC+aeeD0uTlL0e21LM1Qk62xQuZjqEtU+Y68TEhTZ MQCnVJ/EPR4NLSm4QOE+bDu0KoKrnFt5H1glBCINaqIhhVyYeqWQe4AweiC4n8TcgaQQeu5Qd/a8 WuS12GU+B9bc7jwwc/j/nmE27rCfWsoh7lIeDEwxI96lMze070L2HmvQjkpGtjrJjUNDB3SEOn/B GQkyLR0ZwVEozrEiXRwP2tTP1UTN74eCRg21nQaZhd/RW+dzFlPFipqBr5REdTkLFhKMIomS+o38 El6eTJxbdywY0iMwegk6yfOWU3t5UPVg8YKTAAAAAAAA --Apple-Mail-3--432267045--