Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nF9sC-0000Jv-Gc for pgsql-admin@arkaria.postgresql.org; Wed, 02 Feb 2022 07:16:12 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1nF9sB-0001dC-CT for pgsql-admin@arkaria.postgresql.org; Wed, 02 Feb 2022 07:16:11 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nF9sA-0001d2-TJ for pgsql-admin@lists.postgresql.org; Wed, 02 Feb 2022 07:16:11 +0000 Received: from jakobs.com ([85.214.83.89] helo=rs.plausibolo.de) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nF9s5-0007yE-4Q for pgsql-admin@lists.postgresql.org; Wed, 02 Feb 2022 07:16:09 +0000 Received: from localhost (localhost [127.0.0.1]) by rs.plausibolo.de (Postfix) with ESMTP id 3C1503805B2; Wed, 2 Feb 2022 08:16:02 +0100 (CET) Received: from rs.plausibolo.de ([IPv6:::1]) by localhost (h2367442.stratoserver.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id Mq_GSBUHYSxK; Wed, 2 Feb 2022 08:16:02 +0100 (CET) Received: from [192.168.178.197] (ip-037-201-154-229.um10.pools.vodafone-ip.de [37.201.154.229]) by rs.plausibolo.de (Postfix) with ESMTPSA id B986838027C; Wed, 2 Feb 2022 08:16:01 +0100 (CET) Message-ID: Date: Wed, 2 Feb 2022 08:16:00 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.3.2 From: Holger Jakobs Subject: Re: Determine potential change in table size after a column dropped? To: pgsql-admin@lists.postgresql.org References: <84b61f419a0a5ba1049c7d74a85b6790bfe9e1e9.camel@cybertec.at> <599f850b109d98148ee512890ebe6ec4bfbfe26b.camel@cybertec.at> Content-Language: de-DE In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------OFbXZk8Ie6zMsVG6zhg2ZkO1" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------OFbXZk8Ie6zMsVG6zhg2ZkO1 Content-Type: multipart/mixed; boundary="------------CAFu50Iktzzpqy0IiRK5NCgB"; protected-headers="v1" From: Holger Jakobs To: pgsql-admin@lists.postgresql.org Message-ID: Subject: Re: Determine potential change in table size after a column dropped? References: <84b61f419a0a5ba1049c7d74a85b6790bfe9e1e9.camel@cybertec.at> <599f850b109d98148ee512890ebe6ec4bfbfe26b.camel@cybertec.at> In-Reply-To: --------------CAFu50Iktzzpqy0IiRK5NCgB Content-Type: multipart/alternative; boundary="------------yFfkPk8i6vOAqyBOE1sUAP4z" --------------yFfkPk8i6vOAqyBOE1sUAP4z Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 QW0gMDEuMDIuMjIgdW0gMTQ6NDYgc2NocmllYiBKaWFua2FuZyBKaToNCj4gSGkgQWxsIFN1 cGVybWVuIEV4cGVydHMsDQo+DQo+IEknbSBuZXcgaW4gcGdzcWwgYW5kIGhhdmUgYSBzaW1p bGFyIHByb2JsZW0gZm9yIGEgdGltZXNjYWxlIHBnREIuIEEgREIgDQo+IHRhYmxlIGlzIGZv ciBzdG9yaW5nIHJhdyBzZXNzaW9ucyBkYXRhIHJlY2VpdmVkIHRocm91Z2ggSW9UIG5ldHdv cmsgDQo+IGZyb20gbWFueSByZW1vdGUgbWFjaGluZXMuIFRoZSBkYXRhIGZvcm1hdCBpcyB0 aGUgc2FtZSBmb3IgYWxsIHRoZSANCj4gbWFjaGluZXMgYnV0IHRoZSBzZXNzaW9ucyBsYXN0 aW5nLXBlcmlvZHMgY291bGQgYmUgZGlmZmVyZW50IGZyb20gMSANCj4gbWludXRlIHRvIDEg aG91ciBhbmQgc3VjaC4gRWFjaCBtYWNoaW5lIGNvdWxkIGJlIGFjdGl2YXRlZCBvbmNlIGEg ZGF5IA0KPiBvciBhIGZldyB0aW1lcyBhIGRheSByYW5kb21seS4NCj4NCj4gTXkgcXVlc3Rp b24gaXM6DQo+IDEuIEhvdyB0byBzZXR1cCBhIHdhdGNoLWRvZyB0byBkZXRlY3QgbmV3IGRh dGEgaGFzIGJlZW4gYWRkZWQgaW50byB0aGUgDQo+IERCLCBhbmQNCj4gMi4gSG93IHRvIHBp Y2stdXAgdGhlIG5ld2x5IGNvbXBsZXRlZCBzZXNzaW9ucyBkYXRhIHNpbmNlIGxhc3QgcGlj ay11cCANCj4gYW5kIHB1dCBpdCBpbnRvIGEgYnVmZmVyIHRhYmxlIGRlZGljYXRlZCB0byBu ZXcgZGF0YSBmb3IgZnVydGhlciBFVEwgDQo+IHByb2Nlc3Npbmc/DQo+DQo+IElmIHlvdSBo YXZlIHNvbWUgc2NyaXB0cyBpbiBwZ1NRTCwgUHl0aG9uIG9yIEMsIGl0IHdpbGwgYmUgZ3Jl YXRseSANCj4gYXBwcmVjaWF0ZWQhDQo+DQo+IFRoYW5rIHlvdS4NCj4NCj4gQmVzdCByZWdh cmRzLCBKaQ0KPg0KPg0KPg0KSW4gb3JkZXIgdG8gZ2V0IG5vdGlmaWVkIHNob3VsZCBuZXcg cm93cyBhcnJpdmUgKG9yIGN1cnJlbnQgb25lcyB1cGRhdGVkIA0Kb3IgZGVsZXRlZCksIHlv dSBjYW4gaW5zdGFsbCBhIHRyaWdnZXIgd2hpY2ggZmlyZXMgYSBOT1RJRlkgY29tbWFuZCBv biBhIA0KbmFtZSAoY2hhbm5lbCkuDQoNCkFsbCBvdGhlciBzZXNzaW9ucyB3aGljaCBoYXZl IGlzc3VlZCBhIExJU1RFTiBvbiB0aGUgc2FtZSBuYW1lIChjaGFubmVsKSANCndpbGwgcmVj ZWl2ZSBhIG5vdGlmaWNhdGlvbi4NCg0KVW5mb3J0dW5hdGVseSwgbm90IGFsbCBsYW5ndWFn ZXMgYW5kIGRyaXZlcnMgc3VwcG9ydCB0aGlzLg0KDQpSZWNlbnRseSwgSSB1cGRhdGVkIHRo ZSBjb2RlIGZvciBwZ19saXN0ZW4gaW4gdGhlIHNjcmlwdCBsYW5ndWFnZSBUY2wuIA0KSXQn cyBjb21taXR0ZWQsIGJ1dCBubyBuZXcgdmVyc2lvbiByZWxlYXNlZCB5ZXQuDQoNCg0KLS0g DQpIb2xnZXIgSmFrb2JzLCBCZXJnaXNjaCBHbGFkYmFjaCwgVGVsLiArNDktMTc4LTk3NTkw MTINCg0K --------------yFfkPk8i6vOAqyBOE1sUAP4z Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Am 01.02.22 um 14:46 schrieb Jiankang Ji:
Hi All Supermen Experts,

I'm new in pgsql and have a similar problem for a timescale pgDB. A DB table is for storing raw sessions data received through IoT network from many remote machines. The data format is the same for all the machines but the sessions lasting-periods could be different from 1 minute to 1 hour and such. Each machine could be activated once a day or a few times a day randomly.

My question is:
1. How to setup a watch-dog to detect new data has been added into the DB, and=C2=A0
2. How to pick-up the newly completed sessions data since last pick-up and put it into a buffer table dedicated to new data for further ETL processing?

If you have some scripts in pgSQL, Python or C, it will be greatly appreciated!

Thank you.

Best regards, Ji


=

In order to get notified should new rows arrive (or current ones updated or deleted), you can install a trigger which fires a NOTIFY command on a name (channel).

All other sessions which have issued a LISTEN on the same name (channel) will receive a notification.

Unfortunately, not all languages and drivers support this.

Recently, I updated the code for pg_listen in the script language Tcl. It's committed, but no new version released yet.


--=20
Holger Jakobs, Bergisch Gladbach, Tel. +49-178-9759012
--------------yFfkPk8i6vOAqyBOE1sUAP4z-- --------------CAFu50Iktzzpqy0IiRK5NCgB-- --------------OFbXZk8Ie6zMsVG6zhg2ZkO1 Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wmMEABEIACMWIQScuJCxWJK1p5BOebGKV5yTsxlhugUCYfovsAUDAAAAAAAKCRCKV5yTsxlhuj8s AKCaOqzXt4r3XE7N3yUEN6s1Cw4ySACgv6VQJPFWK7ucMOgeuvOw9Zg7ihs= =aKmT -----END PGP SIGNATURE----- --------------OFbXZk8Ie6zMsVG6zhg2ZkO1--