Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tXTXn-007kdz-Ih for pgsql-admin@arkaria.postgresql.org; Mon, 13 Jan 2025 23:08:28 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1tXTXl-00DuqR-DS for pgsql-admin@arkaria.postgresql.org; Mon, 13 Jan 2025 23:08:25 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tXTXl-00DuqJ-0B for pgsql-admin@lists.postgresql.org; Mon, 13 Jan 2025 23:08:25 +0000 Received: from mail-lj1-x231.google.com ([2a00:1450:4864:20::231]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1tXTXj-000Fle-0s for pgsql-admin@postgresql.org; Mon, 13 Jan 2025 23:08:25 +0000 Received: by mail-lj1-x231.google.com with SMTP id 38308e7fff4ca-3061513d353so21372781fa.2 for ; Mon, 13 Jan 2025 15:08:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736809702; x=1737414502; darn=postgresql.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=M9pq7+3dZdfrTLhJiRuS3OsvpRJUKZidodKmvcv5whE=; b=la76V0Bt/Or5EegtiR2c5t5PQH2nY7EO7WzEG9Gm/po37Odp1yrSvPOE3VBY+2QUKU LTr46dWbe6Q4nB7eIDP2/Ry0zFuo8mZsrchkNiiKKZZJdSfhPQe5YoB7nCjWX1Obrimu TZOdtoLB+040my+dVrdigMjZbVzvhNcdnJ+ITqZn/rrP33c+8Dh6ms0f1uf2mX8D00PI k+/2dp/Llm/niYYVt+RQdaZbCd/2QsN9+p6wX59eQm+aOk6kM36htsHJEy9nxLqFgRrw 4D9ZWPJeTKW5U22EtM0FfDz2xeqwfxnHoiUd7H0IzkiY4nv5DgC97NAv7C4oj+fC7WzB YywA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736809702; x=1737414502; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=M9pq7+3dZdfrTLhJiRuS3OsvpRJUKZidodKmvcv5whE=; b=O6n7F9qD7phUXwY3SvPPOYpEb02aZku8L40hk3wDAACkK8vHAd/ZkGWMRCSIhidGf9 MbZPU/MMZU1Qt35kWIKkqz5InWLN4RoxbPkB6mOjhkmjumU2u57V1iiFnpRK9Aa25svQ bSQAgEnjQKSD/AH7oeWsJPGIAYGFLnbhA4klGcnIbZfuNZge6WsoA1O9WAuV3g1nJGo3 1293DwQpwCQhcOmSa29qLgeKDNo31B9p27A6CtaVXjPcp8hBDaqFxEkisDq8m3Npmbge hA+9AUA9sZ+m46wBYwL6GZbjZkC3u6L6OgByo6ItEYK038SIWPoeZyBMXY8wdDyjGuBv mYhw== X-Forwarded-Encrypted: i=1; AJvYcCUjsoPF86cpQChEwb1gZzYNvf9anFPLVOssuhdTpoXBf14WlR2YDjd7zFYjTRJ49HE3xZiWU+kEEZKs0Q==@postgresql.org X-Gm-Message-State: AOJu0Yx2GV7nKs/k9CVUROyEj5fN34E4tWEKxu8exKwblRKtmnVn8yzo q7rVNcLagqL0fUMiECX6lj6/e6bnjKk08gTTMPvpBPuf7R8xPw1H6Q2ZdDeqzHd7t0vdtCYu2Zc AfKU8ddxeZ5gWFL/jYT6+NAru/BNKEQ== X-Gm-Gg: ASbGncvakeM7kFUsFr9667xzNbB8SeS1uCUBlPkMpXwDavC7SfKtxPfaHPWvPiZF92R 348Z/yVuNIOEy/hsYWjJPiJ2G+0xafa79cOFpAxM= X-Google-Smtp-Source: AGHT+IFnJ/IrU1vs86y75Jwkv3H+2rw1bbVslvzzKY8ssgb2tOmPmvgRJAGkenEJaeQ1/vInFBhudN3SAOdCh2o5kak= X-Received: by 2002:a05:651c:505:b0:300:3778:4dbb with SMTP id 38308e7fff4ca-305f459ae63mr78652141fa.1.1736809701988; Mon, 13 Jan 2025 15:08:21 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: kasem adel Date: Tue, 14 Jan 2025 01:08:10 +0200 X-Gm-Features: AbW1kvbb3Y5buEbGBAYuleXVKnarjaUTaaYEJkM6R02MLKGNJc6tAekhkhR1baw Message-ID: Subject: Re: Archiving solutions To: Imran Khan , pgsql-admin Content-Type: multipart/alternative; boundary="000000000000545f2a062b9e85fa" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000545f2a062b9e85fa Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Dear imeran , Thanks for your support. Dears, Appreciate your support in simple solution to implement it. Thanks =D9=81=D9=8A =D8=A7=D9=84=D8=A3=D8=AD=D8=AF=D8=8C =D9=A1=D9=A2 =D9=8A=D9=86= =D8=A7=D9=8A=D8=B1 =D9=A2=D9=A0=D9=A2=D9=A5 =D9=A1=D9=A1:=D9=A2=D9=A3 =D9= =85 Imran Khan =D9=83=D8=AA=D8=A8: > Hi, > > How the data can be stored. Do you have historical data? If yes , then > you can keep that data in archived tables or if that data is still requir= ed > by business then store it in partitions monthly wise . You need to design > the data storage with proper planning. Also, if you have dynamic and ever > changing data and you have date records then store with respect to > partitions and retain for last 3 or 6 months then move to archived tables > or purge if not required .. keep track of indexes and capture long runnin= g > SQL statements through pg_stat_statements extension and run explain plan = to > get the proper indexes created based on the output of that plan. Also, ke= ep > track of memory related parameters to check if any memory leak occurs > resulting in EXCESSIVE swap space usage on OS. > > That's my opinion but we have more experts here who can help us to > understand more. > > Thanks, > Imran > > On Sun, Jan 12, 2025, 11:57=E2=80=AFPM kasem adel = wrote: > >> Hi, >> >> What is the regular housekeeping. >> >> Thanks >> >> =D9=81=D9=8A =D8=A7=D9=84=D8=A3=D8=AD=D8=AF=D8=8C =D9=A1=D9=A2 =D9=8A=D9= =86=D8=A7=D9=8A=D8=B1 =D9=A2=D9=A0=D9=A2=D9=A5 =D9=A1=D9=A0:=D9=A5=D9=A5 = =D9=85 Imran Khan =D9=83=D8=AA=D8=A8: >> >>> Hi, >>> >>> Plan to use table partitioning and do regular housekeeping of the >>> cluster. >>> >>> Thanks, >>> Imran >>> >>> On Sun, Jan 12, 2025, 11:42=E2=80=AFPM kasem adel wrote: >>> >>>> Dears , >>>> >>>> Appreciate your support I have 2.4 TB database and I need to implement >>>> archiving solutions to prevent data growthing. >>>> >>>> Thanks >>>> >>> --000000000000545f2a062b9e85fa Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Dear imeran ,

Thanks for your support.

Dears,

Appreciate you= r support in simple solution to implement it.

Thanks



=D9=81=D9=8A =D8=A7=D9=84=D8=A3=D8=AD=D8=AF=D8=8C =D9=A1=D9= =A2 =D9=8A=D9=86=D8=A7=D9=8A=D8=B1 =D9=A2=D9=A0=D9=A2=D9=A5 =D9=A1=D9=A1:= =D9=A2=D9=A3 =D9=85 Imran Khan <imran.k.23@gmail.com> =D9=83=D8= =AA=D8=A8:
Hi,
=C2=A0How the data can be stored= . Do you have historical=C2=A0data? If yes , then you can keep that data in= archived tables or if that data is still required by business then store i= t in partitions monthly wise . You need to design the data storage with pro= per planning. Also, if you have dynamic and ever changing data and you have= date records then store with respect to partitions and retain for last 3 o= r 6 months then move to archived tables or purge if not required .. keep tr= ack of indexes and capture long running SQL statements through pg_stat_stat= ements extension and run explain plan to get the proper indexes created bas= ed on the output of that plan. Also, keep track of memory related parameter= s to check if any memory leak occurs resulting in EXCESSIVE swap space usag= e on OS.=C2=A0

=C2=A0Tha= t's my opinion but we have more experts here who can help us to underst= and more.

Thanks,
<= div dir=3D"auto">Imran=C2=A0

On Sun, Jan 12, 2025, 11:57=E2=80=AFPM ka= sem adel <kasemadel8@gmail.com> wrote:
Hi,

What is the regular housekeeping.=C2=A0

Thanks=C2=A0

=D9=81=D9=8A =D8=A7= =D9=84=D8=A3=D8=AD=D8=AF=D8=8C =D9=A1=D9=A2 =D9=8A=D9=86=D8=A7=D9=8A=D8=B1 = =D9=A2=D9=A0=D9=A2=D9=A5 =D9=A1=D9=A0:=D9=A5=D9=A5 =D9=85 Imran Khan <imran.k.23@gmail.com> =D9=83=D8=AA=D8=A8:
<= /div>
Hi,
=
Plan to use table partitioning and do regular h= ousekeeping of the cluster.=C2=A0

Thanks,
Imran

On Sun, Jan 12, 2025, 1= 1:42=E2=80=AFPM kasem adel <kasemade= l8@gmail.com> wrote:
Dears ,

Appreciate = your support I have 2.4 TB database and I need to implement archiving solut= ions to prevent data growthing.=C2=A0

Thanks=C2=A0
--000000000000545f2a062b9e85fa--