Received: from localhost (unknown [200.46.204.183]) by postgresql.org (Postfix) with ESMTP id 8CF1F64FD32 for ; Sun, 31 Aug 2008 12:54:58 -0300 (ADT) Received: from postgresql.org ([200.46.204.86]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 25385-02 for ; Sun, 31 Aug 2008 12:54:52 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from outmail137079.authsmtp.co.uk (outmail137079.authsmtp.co.uk [62.13.137.79]) by postgresql.org (Postfix) with ESMTP id 01E5164FCF9 for ; Sun, 31 Aug 2008 12:54:51 -0300 (ADT) Received: from mail-c187.authsmtp.com (mail-c187.authsmtp.com [62.13.128.33]) by punt4.authsmtp.com (8.14.2/8.14.2/Kp) with ESMTP id m7VFsnww017586 for ; Sun, 31 Aug 2008 16:54:49 +0100 (BST) Received: from [192.168.10.87] (LAubervilliers-151-12-98-86.w193-252.abo.wanadoo.fr [193.252.201.86]) (authenticated bits=0) by mail.authsmtp.com (8.14.2/8.14.2/Kp) with ESMTP id m7VFseoM081694 for ; Sun, 31 Aug 2008 16:54:42 +0100 (BST) Subject: rmgr hooks and contrib/rmgr_hook From: Simon Riggs To: List pgsql-patches Content-Type: multipart/mixed; boundary="=-1wM3qrVXC7wyle9kqO5m" Date: Sun, 31 Aug 2008 10:27:47 +0100 Message-Id: <1220174867.4371.107.camel@ebony.2ndQuadrant> Mime-Version: 1.0 X-Mailer: Evolution 2.12.0 X-Server-Quench: 24074a2a-7775-11dd-8155-001185d377ca X-AuthRoute: OCdxZQATClZOTQEd DAteCiNZVAwpPBRK HVkIKg5MOFUSTAAU Kk9GJVpeK3UEQkpF VCReGBUITgIzDi12 ahUgKFJCYEpOVxVr VkJMRlhWHUtpCB8f BBscTBtydhtOfmFy YERnQXAVAUMcAwUo HTNUHWwAbGMua2kc UkYWdQdUdwcbfE5Y bVd4UW4MYDFTe3Jh QlJ2Z2hoZGUCJH8K HQoEO0gXTlxDHjk7 RAtKFDMjVUYCQTkp LhZhMFsTBnERNUIx WQBh X-Authentic-SMTP: 61633235383639.squirrel.dmpriest.net.uk:1186/Kp X-Report-SPAM: If SPAM / abuse - report it at: http://www.authsmtp.com/abuse X-Virus-Status: No virus detected - but ensure you scan with your own anti-virus system! X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=1.854 tagged_above=0 required=5 tests=DATE_IN_PAST_06_12=1.854 X-Spam-Level: * X-Archive-Number: 200808/35 X-Sequence-Number: 7011 --=-1wM3qrVXC7wyle9kqO5m Content-Type: text/plain Content-Transfer-Encoding: 7bit As previously discussed on -hackers on Aug 19, "Proposed Resource Manager Changes". Enclosed are two closely related items: 1) A refactoring of calls to Rmgr code from xlog.c, and having isolated the code for rmgrs then to allow rmgr plugins to modify and/or add rmgrs to Postgres. Includes additional code to generate log messages so we can see what is happening after plugin has executed. Introduces a shared memory area for Rmgrs that allows each backend to read which RmgrIds are valid for the currently running server, allowing call to be made during XLogInsert() to validate rmgrid. (The validation uses a fixed length BitMapSet, a minor new invention for this patch, but that is begging to be refactored - I await advice and/or comments on the fastest way to do this if that isn't it.) (I'd like to rip out WAL_DEBUG completely in favour of this new mechanism, but I haven't done that here). 2) contrib module that contains an example rmgr_hook - actually two examples in one module These have both been tested in normal mode, WAL_DEBUG mode and in warm standby recovery, so not a WIP progress patch. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support --=-1wM3qrVXC7wyle9kqO5m Content-Disposition: attachment; filename=rmgr_hook.tar Content-Type: application/x-tar; name=rmgr_hook.tar Content-Transfer-Encoding: base64 cm1ncl9ob29rLwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAwMDA3NTUAMDAwMTc1 MAAwMDAwMTQ0ADAwMDAwMDAwMDAwADExMDU2NDU2NDYwADAxMjI3MQAgNQAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB1c3RhciAgAHNyaWdncwAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAdXNlcnMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABy bWdyX2hvb2svcm1ncl9ob29rLmMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMDAwMDc1NQAwMDAxNzUw ADAwMDAxNDQAMDAwMDAwMDYxNzUAMTEwNTY0NTY0NTQAMDE0NDQzACAwAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHVzdGFyICAAc3JpZ2dzAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAB1c2VycwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC8q LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLQogKgogKiBybWdyX2hvb2tzLmMKICoKICoJICBQb3N0Z3JlcyBSZXNv dXJjZSBNYW5hZ2VyIGhvb2tzIAogKgogKgkgICRQb3N0Z3JlU1FMJAogKgogKgkgIFR3byBleGFt cGxlcyBzaG93bgogKgogKgkJKiBFWEFNUExFX1dBTF9ERUJVRyBzaG93cyBob3cgeW91IG1pZ2h0 IG91dHB1dCBkZWJ1ZyBpbmZvcm1hdGlvbgogKgkJZm9yIHNlbGVjdGVkIFdBTCByZWNvcmRzIGF0 IHRoZSBzYW1lIHRpbWUgYXMgY29udGludWluZyB0byAKICoJCWFwcGx5IFdBTCByZWNvcmRzLgog KgogKgkJKiBFWEFNUExFX0hFQVBfRklMVEVSIHNob3dzIGhvdyB0byBmaWx0ZXIgc3BlY2lmaWMg aGVhcCBXQUwgcmVjb3JkcwogKiAKICogCSAgc2ltb25AMm5kcXVhZHJhbnQuY29tCiAqLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLQogKi8KCiNpbmNsdWRlICJwb3N0Z3Jlcy5oIgoKI2luY2x1ZGUgImZ1bmNhcGku aCIKI2luY2x1ZGUgIm1pc2NhZG1pbi5oIgoKI2luY2x1ZGUgImFjY2Vzcy9odHVwLmgiCiNpbmNs dWRlICJhY2Nlc3MveGxvZ19pbnRlcm5hbC5oIgoKUEdfTU9EVUxFX01BR0lDOwkvKiBUZWxsIHRo ZSBzZXJ2ZXIgYWJvdXQgb3VyIGNvbXBpbGUgZW52aXJvbm1lbnQgKi8KCi8qIGV4dGVybmFsIGNh bGxzICovCnZvaWQgcGx1Z2luX3JtZ3JfaG9vayhSbWdyRGF0YSAqUm1nclRhYmxlKTsKdm9pZCBf UEdfaW5pdCh2b2lkKTsKCnN0YXRpYyB2b2lkIHBsdWdpbl94bG9nX2RlYnVnKFhMb2dSZWNQdHIg bHNuLCBYTG9nUmVjb3JkICpycHRyKTsKCiNkZWZpbmUgRVhBTVBMRV9XQUxfREVCVUcKCiNpZmRl ZiBFWEFNUExFX1dBTF9ERUJVRwpzdGF0aWMgdm9pZCBwbHVnaW5fYnRyZWVfcmVkbyhYTG9nUmVj UHRyIGxzbiwgWExvZ1JlY29yZCAqcnB0cik7CiNlbmRpZgoKI2lmZGVmIEVYQU1QTEVfSEVBUF9G SUxURVIKc3RhdGljIGJvb2wgd2FudF9yZWNvcmQoWExvZ1JlY29yZCAqcnB0cik7CnN0YXRpYyB2 b2lkIHBsdWdpbl9oZWFwX3JlZG8oWExvZ1JlY1B0ciBsc24sIFhMb2dSZWNvcmQgKnJwdHIpOwpz dGF0aWMgdm9pZCBwbHVnaW5faGVhcDJfcmVkbyhYTG9nUmVjUHRyIGxzbiwgWExvZ1JlY29yZCAq cnB0cik7CiNlbmRpZgoKdm9pZApfUEdfaW5pdCh2b2lkKQp7CglybWdyX2hvb2sgPSAmcGx1Z2lu X3JtZ3JfaG9vazsKfQoKLyoKICogcGx1Z2luX3JtZ3JfaG9vaygpCiAqLwp2b2lkCnBsdWdpbl9y bWdyX2hvb2soUm1nckRhdGEgKlJtZ3JUYWJsZSkKewojaWZkZWYgRVhBTVBMRV9XQUxfREVCVUcK CVJtZ3JUYWJsZVtSTV9CVFJFRV9JRF0ucm1fcmVkbyA9IHBsdWdpbl9idHJlZV9yZWRvOwojZW5k aWYgCgojaWZkZWYgRVhBTVBMRV9IRUFQX0ZJTFRFUgoJUm1nclRhYmxlW1JNX0hFQVAyX0lEXS5y bV9yZWRvID0gcGx1Z2luX2hlYXAyX3JlZG87CglSbWdyVGFibGVbUk1fSEVBUF9JRF0ucm1fcmVk byA9IHBsdWdpbl9oZWFwX3JlZG87CgoJZWxvZyhMT0csICJybWdyX3BsdWdpbjogZmlsdGVyaW5n IGF3YXkgZXZlcnl0aGluZyBhcGFydCBmcm9tIGRhdGFiYXNlIDExNTA1Iik7CiNlbmRpZgp9Cgpz dGF0aWMgdm9pZCAKcGx1Z2luX3hsb2dfZGVidWcoWExvZ1JlY1B0ciBsc24sIFhMb2dSZWNvcmQg KnJwdHIpCnsKCWludCBpOwoJU3RyaW5nSW5mb0RhdGEgYnVmOwoKCWluaXRTdHJpbmdJbmZvKCZi dWYpOwoJYXBwZW5kU3RyaW5nSW5mbygmYnVmLCAiTFNOICVYLyVYOiAiLAoJCQkJCSBsc24ueGxv Z2lkLCBsc24ueHJlY29mZik7CgoJYXBwZW5kU3RyaW5nSW5mbygmYnVmLCAieGlkICV1IiwKCQkJ CSBycHRyLT54bF94aWQpOwoKCWZvciAoaSA9IDA7IGkgPCBYTFJfTUFYX0JLUF9CTE9DS1M7IGkr KykKCXsKCQlpZiAocnB0ci0+eGxfaW5mbyAmIFhMUl9TRVRfQktQX0JMT0NLKGkpKQoJCQlhcHBl bmRTdHJpbmdJbmZvKCZidWYsICI7IGJrcGIlZCIsIGkgKyAxKTsKCX0KCglhcHBlbmRTdHJpbmdJ bmZvKCZidWYsICI6ICVzIiwgUm1nck5hbWUocnB0ci0+eGxfcm1pZCkpOwoJYXBwZW5kU3RyaW5n SW5mbygmYnVmLCAiIC0gIik7CglSbWdyRGVzYygmYnVmLCBycHRyLCBYTG9nUmVjR2V0RGF0YShy cHRyKSk7CgllbG9nKExPRywgIiVzIiwgYnVmLmRhdGEpOwoJcGZyZWUoYnVmLmRhdGEpOwp9Cgoj aWZkZWYgRVhBTVBMRV9XQUxfREVCVUcKc3RhdGljIHZvaWQgCnBsdWdpbl9idHJlZV9yZWRvKFhM b2dSZWNQdHIgbHNuLCBYTG9nUmVjb3JkICpycHRyKQp7CglwbHVnaW5feGxvZ19kZWJ1Zyhsc24s IHJwdHIpOwoKCVJtZ3JSZWRvKGxzbiwgcnB0cik7Cn0KI2VuZGlmCgojaWZkZWYgRVhBTVBMRV9I RUFQX0ZJTFRFUgpzdGF0aWMgdm9pZCAKcGx1Z2luX2hlYXBfcmVkbyhYTG9nUmVjUHRyIGxzbiwg WExvZ1JlY29yZCAqcnB0cikKewoJLyoKCSAqIGFwcGx5IGZpbHRlciwgaWYgcmVjb3JkIG5vdCBy ZXF1aXJlZCwgc2tpcAoJICovCglpZiAoIXdhbnRfcmVjb3JkKHJwdHIpKQoJCXJldHVybjsKCglo ZWFwX3JlZG8obHNuLCBycHRyKTsKfQoKc3RhdGljIHZvaWQgCnBsdWdpbl9oZWFwMl9yZWRvKFhM b2dSZWNQdHIgbHNuLCBYTG9nUmVjb3JkICpycHRyKQp7CgkvKgoJICogYXBwbHkgZmlsdGVyLCBp ZiByZWNvcmQgbm90IHJlcXVpcmVkLCBza2lwCgkgKi8KCWlmICghd2FudF9yZWNvcmQocnB0cikp CgkJcmV0dXJuOwoKCWhlYXAyX3JlZG8obHNuLCBycHRyKTsKfQoKLyoKICogd2FudF9yZWNvcmQo KQogKiAKICogRnVuY3Rpb24gZGVjaWRlcyB3aGV0aGVyIHdlIHdhbnQgdGhpcyB4bG9nIHJlY29y ZCBvciBub3QuCiAqIFVzZSB0aGUgZmFjdCB0aGF0IGFsbCBIRUFQIGFuZCBIRUFQMiByZWNvcmRz IGJlZ2luIHdpdGggYSBSZWxGaWxlTm9kZQogKiBpbiBlYWNoIG9mIHRoZWlyIHZhcmlvdXMgc3Ry dWN0dXJlcy4KICovCnN0YXRpYyBib29sCndhbnRfcmVjb3JkKFhMb2dSZWNvcmQgKnJwdHIpCnsK CVJlbEZpbGVOb2RlICp4bHJmbiA9IChSZWxGaWxlTm9kZSAqKSBYTG9nUmVjR2V0RGF0YShycHRy KTsKCglpZiAoeGxyZm4tPmRiTm9kZSAhPSAxMTUwNSkKCXsKCQllbG9nKExPRywgIm5vdCBmb3Ig dGhpcyBkYXRhYmFzZSIpOwoJCXJldHVybiBmYWxzZTsKCX0KCglyZXR1cm4gdHJ1ZTsKfQojZW5k aWYKAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAcm1ncl9ob29r L01ha2VmaWxlAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAwMDA3NTUAMDAwMTc1MAAwMDAwMTQ0 ADAwMDAwMDAwNDU0ADExMDU0NzQxNjM1ADAxMzczNgAgMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAB1c3RhciAgAHNyaWdncwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAdXNlcnMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAjICRQb3N0Z3Jl U1FMJAoKTU9EVUxFUyA9IHJtZ3JfaG9vawpPQkpTCT0gcm1ncl9ob29rLnNvCgppZmRlZiBVU0Vf UEdYUwpQR19DT05GSUcgPSBwZ19jb25maWcKUEdYUyA6PSAkKHNoZWxsICQoUEdfQ09ORklHKSAt LXBneHMpCQppbmNsdWRlICQoUEdYUykKZWxzZQpzdWJkaXIgPSBjb250cmliL3JtZ3JfaG9vawp0 b3BfYnVpbGRkaXIgPSAuLi8uLgppbmNsdWRlICQodG9wX2J1aWxkZGlyKS9zcmMvTWFrZWZpbGUu Z2xvYmFsCmluY2x1ZGUgJCh0b3Bfc3JjZGlyKS9jb250cmliL2NvbnRyaWItZ2xvYmFsLm1rCmVu ZGlmCgoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== --=-1wM3qrVXC7wyle9kqO5m Content-Disposition: attachment; filename=rmgr_plugin.v4.patch Content-Type: text/x-patch; name=rmgr_plugin.v4.patch; charset=UTF-8 Content-Transfer-Encoding: 7bit Index: src/backend/access/transam/rmgr.c =================================================================== RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/transam/rmgr.c,v retrieving revision 1.25 diff -c -r1.25 rmgr.c *** src/backend/access/transam/rmgr.c 5 Nov 2006 22:42:07 -0000 1.25 --- src/backend/access/transam/rmgr.c 31 Aug 2008 08:25:37 -0000 *************** *** 7,12 **** --- 7,13 ---- */ #include "postgres.h" + #include "miscadmin.h" #include "access/clog.h" #include "access/gin.h" #include "access/gist_private.h" *************** *** 16,28 **** #include "access/nbtree.h" #include "access/xact.h" #include "access/xlog_internal.h" #include "commands/dbcommands.h" #include "commands/sequence.h" #include "commands/tablespace.h" #include "storage/smgr.h" ! const RmgrData RmgrTable[RM_MAX_ID + 1] = { {"XLOG", xlog_redo, xlog_desc, NULL, NULL, NULL}, {"Transaction", xact_redo, xact_desc, NULL, NULL, NULL}, {"Storage", smgr_redo, smgr_desc, NULL, NULL, NULL}, --- 17,30 ---- #include "access/nbtree.h" #include "access/xact.h" #include "access/xlog_internal.h" + #include "nodes/bitmapset.h" #include "commands/dbcommands.h" #include "commands/sequence.h" #include "commands/tablespace.h" #include "storage/smgr.h" ! const RmgrData FixedRmgrTable[RM_MAX_FIXED_ID + 1] = { {"XLOG", xlog_redo, xlog_desc, NULL, NULL, NULL}, {"Transaction", xact_redo, xact_desc, NULL, NULL, NULL}, {"Storage", smgr_redo, smgr_desc, NULL, NULL, NULL}, *************** *** 40,42 **** --- 42,365 ---- {"Gist", gist_redo, gist_desc, gist_xlog_startup, gist_xlog_cleanup, gist_safe_restartpoint}, {"Sequence", seq_redo, seq_desc, NULL, NULL, NULL} }; + + /* Main table of Resource Managers */ + RmgrData *RmgrTable; + + /* Hook for plugins to get control in RmgrInitialize() */ + rmgr_hook_type rmgr_hook = NULL; + + #define MAX_NUM_RMGRS 255 + #define RMGR_BITS_PER_WORD 32 + #define RMGR_BITMAP_WORDS ((MAX_NUM_RMGRS + 1) / RMGR_BITS_PER_WORD) + typedef struct RmgrCtlData + { + uint32 RmgrBitmap[RMGR_BITMAP_WORDS]; /* fixed size bitmapset */ + } RmgrCtlData; + + static RmgrCtlData *RmgrCtl = NULL; + + /* + * Initialization of shared memory for Rmgrs + */ + Size + RmgrShmemSize(void) + { + return sizeof(RmgrCtlData); + } + + void + RmgrShmemInit(void) + { + bool foundRmgr; + + RmgrCtl = (RmgrCtlData *) + ShmemInitStruct("Rmgr Ctl", RmgrShmemSize(), &foundRmgr); + + if (foundRmgr) + return; + + memset(RmgrCtl, 0, sizeof(RmgrCtlData)); + } + + /* + * RmgrInitialize() + * + * Create RmgrTable, populate RmgrTable with fixed Rmgrs, call plugin + * and then initialize shared list of current resource managers. + * + * RmgrInitialize must run before any other Rmgr function. Normally, + * it runs only in the Startup process (once), ensuring that the RmgrTable + * and its functions cannot be accessed by normal backends. + * + * If we have WAL_DEBUG set then the plugin will be called multiple + * times and so must be designed to return same answer every time. + */ + void + RmgrInitialize(void) + { + int rmid; + int num_rmgrs = 0; + + if (RmgrTable) + return; + + /* + * Create local copy of RmgrTable. Memory is never freed. + */ + RmgrTable = malloc(sizeof(RmgrData) * (MAX_NUM_RMGRS + 1)); + + for (rmid = 0; rmid <= MAX_NUM_RMGRS; rmid++) + { + if (rmid <= RM_MAX_FIXED_ID) + { + RmgrTable[rmid].rm_name = FixedRmgrTable[rmid].rm_name; + RmgrTable[rmid].rm_redo = FixedRmgrTable[rmid].rm_redo; + RmgrTable[rmid].rm_desc = FixedRmgrTable[rmid].rm_desc; + RmgrTable[rmid].rm_startup = FixedRmgrTable[rmid].rm_startup; + RmgrTable[rmid].rm_cleanup = FixedRmgrTable[rmid].rm_cleanup; + RmgrTable[rmid].rm_safe_restartpoint = + FixedRmgrTable[rmid].rm_safe_restartpoint; + } + else + { + RmgrTable[rmid].rm_name = NULL; + RmgrTable[rmid].rm_redo = NULL; + RmgrTable[rmid].rm_desc = NULL; + RmgrTable[rmid].rm_startup = NULL; + RmgrTable[rmid].rm_cleanup = NULL; + RmgrTable[rmid].rm_safe_restartpoint = NULL; + } + } + + /* + * Call plugin, if present and output log message to assist with + * diagnosis of recovery issues. + * + * What is the plugin allowed to do? + * All RMs are assigned to a single RmgrId. All RM functions are + * optional, though we will only allow WAL inserts for RmgrIds that + * have a defined rm_redo function. The fixed RMs can be overridden + * by providing new routines that do similar, yet slightly different + * actions such as filters. There is no direct connection between + * an RM and and entry in pg_am, so it is possible to create a new + * index type but then not be able to insert into that index, iff + * the new index code issues any WAL inserts. That error is quickly + * discovered in practice, even if the docs are ignored. That sounds + * a little strange, but then pg_am is not readable at the time we + * need to know which RM functions to use for recovery. + * + * XXX we cannot yet issue an index rebuild from rm_cleanup() + * though that feature is expected to be achieved one day + */ + if (rmgr_hook) + { + elog(LOG, "Executing Rmgr hook to extend and/or modify resource managers"); + (*rmgr_hook) (RmgrTable); + } + + /* + * Now create data structure to allow checking of RmgrId in each backend. + */ + for (rmid = 0; rmid <= MAX_NUM_RMGRS; rmid++) + { + if (RmgrTable[rmid].rm_redo != NULL) + { + int wordnum = rmid / RMGR_BITS_PER_WORD; + int bitnum = rmid % RMGR_BITS_PER_WORD; + RmgrCtl->RmgrBitmap[wordnum] |= ((uint32) 1 << bitnum); + + num_rmgrs++; + + /* + * Keep track of new or modified RMs for diagnostic purposes + */ + if (rmid <= RM_MAX_FIXED_ID) + { + if (RmgrTable[rmid].rm_name != FixedRmgrTable[rmid].rm_name || + RmgrTable[rmid].rm_redo != FixedRmgrTable[rmid].rm_redo || + RmgrTable[rmid].rm_startup != FixedRmgrTable[rmid].rm_startup || + RmgrTable[rmid].rm_cleanup != FixedRmgrTable[rmid].rm_cleanup || + RmgrTable[rmid].rm_safe_restartpoint != + FixedRmgrTable[rmid].rm_safe_restartpoint) + elog(LOG, "Rmgr (%d) %s has been modified", rmid, + FixedRmgrTable[rmid].rm_name); + } + else + elog(LOG, "Rmgr (%d) %s defined", rmid, + (RmgrTable[rmid].rm_name != NULL ? + RmgrTable[rmid].rm_name : "")); + } + } + + if (num_rmgrs == 0) + elog(FATAL, "No valid resource managers defined"); + } + + + /* + * RmgrStartup() + * + * Allow resource managers to do any required startup. + * Run RM startup functions once, only ever runs in Startup process. + */ + void + RmgrStartup(void) + { + int rmid; + + Assert(RmgrTable); + for (rmid = 0; rmid <= MAX_NUM_RMGRS; rmid++) + { + if (RmgrTable[rmid].rm_startup != NULL) + { + elog(DEBUG2, "RmgrStartup (%d) %s", rmid, RmgrTable[rmid].rm_name); + RmgrTable[rmid].rm_startup(); + } + } + } + + /* + * RmgrCleanup() + * + * Allow resource managers to do any required cleanup. + * Run RM cleanup functions once, only ever runs in Startup process. + * + * Called once prior to start of normal running + */ + void + RmgrCleanup(void) + { + int rmid; + + Assert(RmgrTable); + for (rmid = 0; rmid <= MAX_NUM_RMGRS; rmid++) + { + if (RmgrTable[rmid].rm_cleanup != NULL) + { + elog(DEBUG2, "RmgrCleanup (%d) %s", rmid, RmgrTable[rmid].rm_name); + RmgrTable[rmid].rm_cleanup(); + } + } + } + + /* + * RmgrSafeRestartpoint() + * + * Is it safe to mark a restartpoint? We must ask each of the resource + * managers whether they have any partial state information that might + * prevent a correct restart from the supplied LSN. + * + * Called once every 5 minutes with default settings, sometimes more + * frequently if last call returned false. + */ + bool + RmgrSafeRestartpoint(XLogRecPtr checkPointRedo) + { + int rmid; + + Assert(RmgrTable); + for (rmid = 0; rmid <= MAX_NUM_RMGRS; rmid++) + { + if (RmgrTable[rmid].rm_safe_restartpoint != NULL) + if (!(RmgrTable[rmid].rm_safe_restartpoint())) + { + elog(DEBUG2, "RM (%d) %s not safe to record restart point at %X/%X", + rmid, + RmgrTable[rmid].rm_name, + checkPointRedo.xlogid, + checkPointRedo.xrecoff); + return false; + } + } + + return true; + } + + /* + * RmgrRedo + * + * Call the RM's redo function to apply WAL records. The RmgrId should + * previously have been checked using RmgrIdIsValid(). + */ + void + RmgrRedo(XLogRecPtr lsn, XLogRecord *rptr) + { + RmgrTable[rptr->xl_rmid].rm_redo(lsn, rptr); + } + + /* + * Error context callback for errors occurring during rm_redo(). + */ + void + rm_redo_error_callback(void *arg) + { + XLogRecord *record = (XLogRecord *) arg; + StringInfoData buf; + + initStringInfo(&buf) ; + Assert(RmgrTable); + if (RmgrTable[record->xl_rmid].rm_desc != NULL) + RmgrTable[record->xl_rmid].rm_desc(&buf, + record->xl_info, + XLogRecGetData(record)); + + /* don't bother emitting empty description */ + if (buf.len > 0) + errcontext("xlog redo %s", buf.data); + + pfree(buf.data); + } + + /* + * RmgrDesc + * + * Call the RM's desc function to print details of WAL records + * during error handling or when WAL_DEBUG is set + */ + void + RmgrDesc(StringInfo buf, XLogRecord *rptr, char *rec) + { + if (RmgrTable[rptr->xl_rmid].rm_desc != NULL) + RmgrTable[rptr->xl_rmid].rm_desc(buf, rptr->xl_info, rec); + } + + /* + * RmgrName + * + * Get the RM's namefor use when WAL_DEBUG is set + */ + const char * + RmgrName(RmgrId rmid) + { + Assert(RmgrTable); + return RmgrTable[rmid].rm_name; + } + + /* + * RmgrIsIsValid + * + * Check whether the RmgrId has a valid rm_redo function currently + * Caller throws any error, not here. + */ + bool + RmgrIdIsValid(RmgrId rmid) + { + int wordnum = rmid / RMGR_BITS_PER_WORD; + int bitnum = rmid % RMGR_BITS_PER_WORD; + + /* + * If not already done so, get set of valid RmgrIds as a bms + */ + if (!RmgrCtl) + RmgrShmemInit(); + + /* + * Check validity + */ + if ((RmgrCtl->RmgrBitmap[wordnum] & ((bitmapword) 1 << bitnum)) != 0) + return true; + + return false; + } + Index: src/backend/access/transam/xlog.c =================================================================== RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/transam/xlog.c,v retrieving revision 1.317 diff -c -r1.317 xlog.c *** src/backend/access/transam/xlog.c 11 Aug 2008 11:05:10 -0000 1.317 --- src/backend/access/transam/xlog.c 31 Aug 2008 08:24:18 -0000 *************** *** 435,441 **** static void pg_start_backup_callback(int code, Datum arg); static bool read_backup_label(XLogRecPtr *checkPointLoc, XLogRecPtr *minRecoveryLoc); - static void rm_redo_error_callback(void *arg); static int get_sync_bit(int method); --- 435,440 ---- *************** *** 496,501 **** --- 495,511 ---- } /* + * Check that we are writing a currently valid RmgrId. If not, then + * we cannot allow this to be written to WAL otherwise we would be + * unable to recover correctly from crash and could lose data written + * after an invalid WAL record. + */ + if (!RmgrIdIsValid(rmid)) + ereport(ERROR, + (errmsg("invalid resource manager ID %u", rmid), + errhint("You must configure resource manager at startup."))); + + /* * Here we scan the rdata chain, determine which buffers must be backed * up, and compute the CRC values for the data. Note that the record * header isn't added into the CRC initially since we don't know the final *************** *** 827,832 **** --- 837,844 ---- { StringInfoData buf; + RmgrInitialize(); + initStringInfo(&buf); appendStringInfo(&buf, "INSERT @ %X/%X: ", RecPtr.xlogid, RecPtr.xrecoff); *************** *** 834,840 **** if (rdata->data != NULL) { appendStringInfo(&buf, " - "); ! RmgrTable[record->xl_rmid].rm_desc(&buf, record->xl_info, rdata->data); } elog(LOG, "%s", buf.data); pfree(buf.data); --- 846,852 ---- if (rdata->data != NULL) { appendStringInfo(&buf, " - "); ! RmgrDesc(&buf, record, rdata->data); } elog(LOG, "%s", buf.data); pfree(buf.data); *************** *** 3144,3153 **** RecPtr->xlogid, RecPtr->xrecoff))); goto next_record_is_invalid; } ! if (record->xl_rmid > RM_MAX_ID) { ereport(emode, ! (errmsg("invalid resource manager ID %u at %X/%X", record->xl_rmid, RecPtr->xlogid, RecPtr->xrecoff))); goto next_record_is_invalid; } --- 3156,3165 ---- RecPtr->xlogid, RecPtr->xrecoff))); goto next_record_is_invalid; } ! if (!RmgrIdIsValid(record->xl_rmid)) { ereport(emode, ! (errmsg("invalid resource manager ID %u" " at %X/%X", record->xl_rmid, RecPtr->xlogid, RecPtr->xrecoff))); goto next_record_is_invalid; } *************** *** 4880,4885 **** --- 4892,4906 ---- */ readRecoveryCommandFile(); + /* + * Initialize RMs. Startup functions are *not* called unless we + * are InRecovery. We do this in two steps so that we know what + * RMs are currently valid for when we call XLogInsert(). We + * specifically do not write the list of RMs to the ControlFile, + * to allow more flexibility in adding/removing RMs. + */ + RmgrInitialize(); + /* Now we can determine the list of expected TLIs */ expectedTLIs = readTimeLineHistory(recoveryTargetTLI); *************** *** 5009,5016 **** /* REDO */ if (InRecovery) { - int rmid; - /* * Update pg_control to show that we are recovering and to show the * selected checkpoint as the place we are starting from. We also mark --- 5030,5035 ---- *************** *** 5056,5067 **** BACKUP_LABEL_FILE, BACKUP_LABEL_OLD))); } ! /* Initialize resource managers */ ! for (rmid = 0; rmid <= RM_MAX_ID; rmid++) ! { ! if (RmgrTable[rmid].rm_startup != NULL) ! RmgrTable[rmid].rm_startup(); ! } /* * Find the first record that logically follows the checkpoint --- it --- 5075,5084 ---- BACKUP_LABEL_FILE, BACKUP_LABEL_OLD))); } ! /* ! * Startup Resource Managers ! */ ! RmgrStartup(); /* * Find the first record that logically follows the checkpoint --- it *************** *** 5105,5113 **** EndRecPtr.xlogid, EndRecPtr.xrecoff); xlog_outrec(&buf, record); appendStringInfo(&buf, " - "); ! RmgrTable[record->xl_rmid].rm_desc(&buf, ! record->xl_info, ! XLogRecGetData(record)); elog(LOG, "%s", buf.data); pfree(buf.data); } --- 5122,5128 ---- EndRecPtr.xlogid, EndRecPtr.xrecoff); xlog_outrec(&buf, record); appendStringInfo(&buf, " - "); ! RmgrDesc(&buf, record, XLogRecGetData(record)); elog(LOG, "%s", buf.data); pfree(buf.data); } *************** *** 5125,5131 **** } /* Setup error traceback support for ereport() */ ! errcontext.callback = rm_redo_error_callback; errcontext.arg = (void *) record; errcontext.previous = error_context_stack; error_context_stack = &errcontext; --- 5140,5146 ---- } /* Setup error traceback support for ereport() */ ! errcontext.callback = rm_redo_error_callback; /* see rmgr.c */ errcontext.arg = (void *) record; errcontext.previous = error_context_stack; error_context_stack = &errcontext; *************** *** 5141,5147 **** if (record->xl_info & XLR_BKP_BLOCK_MASK) RestoreBkpBlocks(record, EndRecPtr); ! RmgrTable[record->xl_rmid].rm_redo(EndRecPtr, record); /* Pop the error context stack */ error_context_stack = errcontext.previous; --- 5156,5162 ---- if (record->xl_info & XLR_BKP_BLOCK_MASK) RestoreBkpBlocks(record, EndRecPtr); ! RmgrRedo(EndRecPtr, record); /* Pop the error context stack */ error_context_stack = errcontext.previous; *************** *** 5288,5303 **** if (InRecovery) { ! int rmid; ! ! /* ! * Allow resource managers to do any required cleanup. ! */ ! for (rmid = 0; rmid <= RM_MAX_ID; rmid++) ! { ! if (RmgrTable[rmid].rm_cleanup != NULL) ! RmgrTable[rmid].rm_cleanup(); ! } /* * Check to see if the XLOG sequence contained any unresolved --- 5303,5309 ---- if (InRecovery) { ! RmgrCleanup(); /* * Check to see if the XLOG sequence contained any unresolved *************** *** 6037,6043 **** RecoveryRestartPoint(const CheckPoint *checkPoint) { int elapsed_secs; - int rmid; /* * Do nothing if the elapsed time since the last restartpoint is less than --- 6043,6048 ---- *************** *** 6053,6075 **** return; /* ! * Is it safe to checkpoint? We must ask each of the resource managers ! * whether they have any partial state information that might prevent a ! * correct restart from this point. If so, we skip this opportunity, but * return at the next checkpoint record for another try. */ ! for (rmid = 0; rmid <= RM_MAX_ID; rmid++) ! { ! if (RmgrTable[rmid].rm_safe_restartpoint != NULL) ! if (!(RmgrTable[rmid].rm_safe_restartpoint())) ! { ! elog(DEBUG2, "RM %d not safe to record restart point at %X/%X", ! rmid, ! checkPoint->redo.xlogid, ! checkPoint->redo.xrecoff); ! return; ! } ! } /* * OK, force data out to disk --- 6058,6068 ---- return; /* ! * Is it safe to restart from here? If not, we skip this opportunity, but * return at the next checkpoint record for another try. */ ! if (!RmgrSafeRestartpoint(checkPoint->redo)) ! return; /* * OK, force data out to disk *************** *** 6304,6310 **** appendStringInfo(buf, "; bkpb%d", i + 1); } ! appendStringInfo(buf, ": %s", RmgrTable[record->xl_rmid].rm_name); } #endif /* WAL_DEBUG */ --- 6297,6303 ---- appendStringInfo(buf, "; bkpb%d", i + 1); } ! appendStringInfo(buf, ": %s", RmgrName(record->xl_rmid)); } #endif /* WAL_DEBUG */ *************** *** 7065,7091 **** } /* - * Error context callback for errors occurring during rm_redo(). - */ - static void - rm_redo_error_callback(void *arg) - { - XLogRecord *record = (XLogRecord *) arg; - StringInfoData buf; - - initStringInfo(&buf); - RmgrTable[record->xl_rmid].rm_desc(&buf, - record->xl_info, - XLogRecGetData(record)); - - /* don't bother emitting empty description */ - if (buf.len > 0) - errcontext("xlog redo %s", buf.data); - - pfree(buf.data); - } - - /* * BackupInProgress: check if online backup mode is active * * This is done by checking for existence of the "backup_label" file. --- 7058,7063 ---- Index: src/backend/storage/ipc/ipci.c =================================================================== RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/storage/ipc/ipci.c,v retrieving revision 1.96 diff -c -r1.96 ipci.c *** src/backend/storage/ipc/ipci.c 12 May 2008 00:00:50 -0000 1.96 --- src/backend/storage/ipc/ipci.c 20 Aug 2008 15:24:26 -0000 *************** *** 18,23 **** --- 18,24 ---- #include "access/heapam.h" #include "access/multixact.h" #include "access/nbtree.h" + #include "access/rmgr.h" #include "access/subtrans.h" #include "access/twophase.h" #include "miscadmin.h" *************** *** 102,107 **** --- 103,109 ---- size = add_size(size, LockShmemSize()); size = add_size(size, ProcGlobalShmemSize()); size = add_size(size, XLOGShmemSize()); + size = add_size(size, RmgrShmemSize()); size = add_size(size, CLOGShmemSize()); size = add_size(size, SUBTRANSShmemSize()); size = add_size(size, TwoPhaseShmemSize()); *************** *** 176,184 **** InitShmemIndex(); /* ! * Set up xlog, clog, and buffers */ XLOGShmemInit(); CLOGShmemInit(); SUBTRANSShmemInit(); TwoPhaseShmemInit(); --- 178,187 ---- InitShmemIndex(); /* ! * Set up xlog, rmgr, clog, and buffers */ XLOGShmemInit(); + RmgrShmemInit(); CLOGShmemInit(); SUBTRANSShmemInit(); TwoPhaseShmemInit(); Index: src/include/access/rmgr.h =================================================================== RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/include/access/rmgr.h,v retrieving revision 1.17 diff -c -r1.17 rmgr.h *** src/include/access/rmgr.h 5 Nov 2006 22:42:10 -0000 1.17 --- src/include/access/rmgr.h 20 Aug 2008 15:25:04 -0000 *************** *** 10,15 **** --- 10,18 ---- typedef uint8 RmgrId; + extern Size RmgrShmemSize(void); + extern void RmgrShmemInit(void); + /* * Built-in resource managers * *************** *** 23,28 **** --- 26,33 ---- #define RM_DBASE_ID 4 #define RM_TBLSPC_ID 5 #define RM_MULTIXACT_ID 6 + /* RmgrId 7 is currently unused */ + /* RmgrId 8 is currently unused */ #define RM_HEAP2_ID 9 #define RM_HEAP_ID 10 #define RM_BTREE_ID 11 *************** *** 30,35 **** #define RM_GIN_ID 13 #define RM_GIST_ID 14 #define RM_SEQ_ID 15 ! #define RM_MAX_ID RM_SEQ_ID #endif /* RMGR_H */ --- 35,49 ---- #define RM_GIN_ID 13 #define RM_GIST_ID 14 #define RM_SEQ_ID 15 ! #define RM_MAX_FIXED_ID RM_SEQ_ID ! ! /* ! * RmgrId Reservation ! * ! * RmgrIds up to 31 are reserved for PostgreSQL Core use only ! * RmgrIds between 32 and 127 are available for registration by ! * PostgreSQL-related projects ! * RmgrIds of 128 and above are always designed for each site ! */ #endif /* RMGR_H */ Index: src/include/access/xlog_internal.h =================================================================== RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/include/access/xlog_internal.h,v retrieving revision 1.24 diff -c -r1.24 xlog_internal.h *** src/include/access/xlog_internal.h 11 Aug 2008 11:05:11 -0000 1.24 --- src/include/access/xlog_internal.h 31 Aug 2008 08:27:31 -0000 *************** *** 229,235 **** */ typedef struct RmgrData { ! const char *rm_name; void (*rm_redo) (XLogRecPtr lsn, XLogRecord *rptr); void (*rm_desc) (StringInfo buf, uint8 xl_info, char *rec); void (*rm_startup) (void); --- 229,235 ---- */ typedef struct RmgrData { ! char *rm_name; void (*rm_redo) (XLogRecPtr lsn, XLogRecord *rptr); void (*rm_desc) (StringInfo buf, uint8 xl_info, char *rec); void (*rm_startup) (void); *************** *** 237,243 **** bool (*rm_safe_restartpoint) (void); } RmgrData; ! extern const RmgrData RmgrTable[]; /* * Exported to support xlog switching from bgwriter --- 237,254 ---- bool (*rm_safe_restartpoint) (void); } RmgrData; ! typedef void (*rmgr_hook_type) (RmgrData *); ! extern PGDLLIMPORT rmgr_hook_type rmgr_hook; ! ! extern void RmgrInitialize(void); ! extern void RmgrStartup(void); ! extern void RmgrCleanup(void); ! extern bool RmgrSafeRestartpoint(XLogRecPtr checkPointRedo); ! extern void RmgrRedo(XLogRecPtr lsn, XLogRecord *rptr); ! extern void RmgrDesc(StringInfo buf, XLogRecord *rptr, char *rec); ! extern void rm_redo_error_callback(void *arg); ! extern const char *RmgrName(RmgrId rmid); ! extern bool RmgrIdIsValid(RmgrId rmid); /* * Exported to support xlog switching from bgwriter --=-1wM3qrVXC7wyle9kqO5m--