Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nEtXR-0008NR-Bf for pgsql-admin@arkaria.postgresql.org; Tue, 01 Feb 2022 13:49:41 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1nEtXQ-00077i-6p for pgsql-admin@arkaria.postgresql.org; Tue, 01 Feb 2022 13:49:40 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nEtXP-00077Z-Oi for pgsql-admin@lists.postgresql.org; Tue, 01 Feb 2022 13:49:39 +0000 Received: from jakobs.com ([85.214.83.89] helo=rs.plausibolo.de) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nEtXN-0008FA-3q for pgsql-admin@lists.postgresql.org; Tue, 01 Feb 2022 13:49:38 +0000 Received: from localhost (localhost [127.0.0.1]) by rs.plausibolo.de (Postfix) with ESMTP id C9CCF3805AC for ; Tue, 1 Feb 2022 14:49:34 +0100 (CET) Received: from rs.plausibolo.de ([127.0.0.1]) by localhost (h2367442.stratoserver.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T-UyDZrWG_kP for ; Tue, 1 Feb 2022 14:49:34 +0100 (CET) Received: from [192.168.178.197] (ip-037-201-154-229.um10.pools.vodafone-ip.de [37.201.154.229]) by rs.plausibolo.de (Postfix) with ESMTPSA id 56D3038027C for ; Tue, 1 Feb 2022 14:49:34 +0100 (CET) Message-ID: Date: Tue, 1 Feb 2022 14:49:33 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.3.2 Content-Language: de-DE To: pgsql-admin@lists.postgresql.org References: <84b61f419a0a5ba1049c7d74a85b6790bfe9e1e9.camel@cybertec.at> <599f850b109d98148ee512890ebe6ec4bfbfe26b.camel@cybertec.at> From: Holger Jakobs Subject: Re: Determine potential change in table size after a column dropped? In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------IBmxIaP2CwzPaa81Sv1yYk2u" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------IBmxIaP2CwzPaa81Sv1yYk2u Content-Type: multipart/mixed; boundary="------------X0Ec1tAfMZAIsqUwzIfdUpJa"; protected-headers="v1" From: Holger Jakobs To: pgsql-admin@lists.postgresql.org Message-ID: Subject: Re: Determine potential change in table size after a column dropped? References: <84b61f419a0a5ba1049c7d74a85b6790bfe9e1e9.camel@cybertec.at> <599f850b109d98148ee512890ebe6ec4bfbfe26b.camel@cybertec.at> In-Reply-To: --------------X0Ec1tAfMZAIsqUwzIfdUpJa Content-Type: multipart/alternative; boundary="------------BrYfgdHiGfw6m4L1KZqrtGbo" --------------BrYfgdHiGfw6m4L1KZqrtGbo Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 QW0gMDEuMDIuMjIgdW0gMTQ6NDYgc2NocmllYiBKaWFua2FuZyBKaToNCj4gSGkgQWxsIFN1 cGVybWVuIEV4cGVydHMsDQo+DQo+IEknbSBuZXcgaW4gcGdzcWwgYW5kIGhhdmUgYSBzaW1p bGFyIHByb2JsZW0gZm9yIGEgdGltZXNjYWxlIHBnREIuIEEgREIgDQo+IHRhYmxlIGlzIGZv ciBzdG9yaW5nIHJhdyBzZXNzaW9ucyBkYXRhIHJlY2VpdmVkIHRocm91Z2ggSW9UIG5ldHdv cmsgDQo+IGZyb20gbWFueSByZW1vdGUgbWFjaGluZXMuIFRoZSBkYXRhIGZvcm1hdCBpcyB0 aGUgc2FtZSBmb3IgYWxsIHRoZSANCj4gbWFjaGluZXMgYnV0IHRoZSBzZXNzaW9ucyBsYXN0 aW5nLXBlcmlvZHMgY291bGQgYmUgZGlmZmVyZW50IGZyb20gMSANCj4gbWludXRlIHRvIDEg aG91ciBhbmQgc3VjaC4gRWFjaCBtYWNoaW5lIGNvdWxkIGJlIGFjdGl2YXRlZCBvbmNlIGEg ZGF5IA0KPiBvciBhIGZldyB0aW1lcyBhIGRheSByYW5kb21seS4NCj4NCj4gTXkgcXVlc3Rp b24gaXM6DQo+IDEuIEhvdyB0byBzZXR1cCBhIHdhdGNoLWRvZyB0byBkZXRlY3QgbmV3IGRh dGEgaGFzIGJlZW4gYWRkZWQgaW50byB0aGUgDQo+IERCLCBhbmQNCj4gMi4gSG93IHRvIHBp Y2stdXAgdGhlIG5ld2x5IGNvbXBsZXRlZCBzZXNzaW9ucyBkYXRhIHNpbmNlIGxhc3QgcGlj ay11cCANCj4gYW5kIHB1dCBpdCBpbnRvIGEgYnVmZmVyIHRhYmxlIGRlZGljYXRlZCB0byBu ZXcgZGF0YSBmb3IgZnVydGhlciBFVEwgDQo+IHByb2Nlc3Npbmc/DQo+DQo+IElmIHlvdSBo YXZlIHNvbWUgc2NyaXB0cyBpbiBwZ1NRTCwgUHl0aG9uIG9yIEMsIGl0IHdpbGwgYmUgZ3Jl YXRseSANCj4gYXBwcmVjaWF0ZWQhDQo+DQo+IFRoYW5rIHlvdS4NCj4NCj4gQmVzdCByZWdh cmRzLCBKaQ0KPg0KPg0KPg0KSW4gb3JkZXIgdG8gZ2V0IG5vdGlmaWVkIHNob3VsZCBuZXcg cm93cyBhcnJpdmUgKG9yIGN1cnJlbnQgb25lcyB1cGRhdGVkIA0Kb3IgZGVsZXRlZCksIHlv dSBjYW4gaW5zdGFsbCBhIHRyaWdnZXIgd2hpY2ggZmlyZXMgYSBOT1RJRlkgY29tbWFuZCBv biBhIA0KbmFtZSAoY2hhbm5lbCkuDQoNCkFsbCBvdGhlciBzZXNzaW9ucyB3aGljaCBoYXZl IGlzc3VlZCBhIExJU1RFTiBvbiB0aGUgc2FtZSBuYW1lIChjaGFubmVsKSANCndpbGwgcmVj ZWl2ZSBhIG5vdGlmaWNhdGlvbi4NCg0KVW5mb3J0dW5hdGVseSwgbm90IGFsbCBsYW5ndWFn ZXMgYW5kIGRyaXZlcnMgc3VwcG9ydCB0aGlzLg0KDQpSZWNlbnRseSwgSSB1cGRhdGVkIHRo ZSBjb2RlIGZvciBwZ19saXN0ZW4gaW4gdGhlIHNjcmlwdCBsYW5ndWFnZSBUY2wuIA0KSXQn cyBjb21taXR0ZWQsIGJ1dCBubyBuZXcgdmVyc2lvbiByZWxlYXNlZCB5ZXQuDQoNCg0KLS0g DQpIb2xnZXIgSmFrb2JzLCBCZXJnaXNjaCBHbGFkYmFjaCwgVGVsLiArNDktMTc4LTk3NTkw MTINCg0K --------------BrYfgdHiGfw6m4L1KZqrtGbo Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Am 01.02.22 um 14:46 schrieb Jiankang Ji:
Hi All Supermen Experts,

I'm new in pgsql and have a similar problem for a timescale pgDB. A DB table is for storing raw sessions data received through IoT network from many remote machines. The data format is the same for all the machines but the sessions lasting-periods could be different from 1 minute to 1 hour and such. Each machine could be activated once a day or a few times a day randomly.

My question is:
1. How to setup a watch-dog to detect new data has been added into the DB, and=C2=A0
2. How to pick-up the newly completed sessions data since last pick-up and put it into a buffer table dedicated to new data for further ETL processing?

If you have some scripts in pgSQL, Python or C, it will be greatly appreciated!

Thank you.

Best regards, Ji


=

In order to get notified should new rows arrive (or current ones updated or deleted), you can install a trigger which fires a NOTIFY command on a name (channel).

All other sessions which have issued a LISTEN on the same name (channel) will receive a notification.

Unfortunately, not all languages and drivers support this.

Recently, I updated the code for pg_listen in the script language Tcl. It's committed, but no new version released yet.


--=20
Holger Jakobs, Bergisch Gladbach, Tel. +49-178-9759012
--------------BrYfgdHiGfw6m4L1KZqrtGbo-- --------------X0Ec1tAfMZAIsqUwzIfdUpJa-- --------------IBmxIaP2CwzPaa81Sv1yYk2u Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wmMEABEIACMWIQScuJCxWJK1p5BOebGKV5yTsxlhugUCYfk6bQUDAAAAAAAKCRCKV5yTsxlhusQf AJwLm9alB9HpnFiIwW/4KfZj+gNcugCgznFe9GtW04IAp0d4aknB/LOHwTQ= =QJWG -----END PGP SIGNATURE----- --------------IBmxIaP2CwzPaa81Sv1yYk2u--