Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tbAUi-008aQ5-TN for pgsql-admin@arkaria.postgresql.org; Fri, 24 Jan 2025 03:36:33 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1tbAUh-008yky-7S for pgsql-admin@arkaria.postgresql.org; Fri, 24 Jan 2025 03:36:31 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tbAUg-008yio-MK for pgsql-admin@lists.postgresql.org; Fri, 24 Jan 2025 03:36:30 +0000 Received: from mail-lj1-x236.google.com ([2a00:1450:4864:20::236]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1tbAUe-001CIl-0O for pgsql-admin@postgresql.org; Fri, 24 Jan 2025 03:36:29 +0000 Received: by mail-lj1-x236.google.com with SMTP id 38308e7fff4ca-30613802a6bso17261051fa.1 for ; Thu, 23 Jan 2025 19:36:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1737689787; x=1738294587; darn=postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=BbVs3gHhzsPOdm8N4N6XVsQByDEeLJyfrHs6UonRrQs=; b=YfVnUY9g3VIpbaWrQp9JVaAIHmRodgsUX6Pu61iymtIJgxaJMh0MdYJ6AX7yqB/mU6 m7b1BSMf88dZpTdoG4cl51WYOPLFCdMaRiq8zhsZK4Rca1hbWlIVLAt1d7xJ5EXrLSJX MeUg6axoaYBxyF2hDxoa4oGtwtZbwHG5Rr5IW2Vz3WQ9EQttinAt+npoB/tiKwQVVEGs 1tNgpSIxADRaamMGnHYVZ/pY7VKTLEgvPSaipeZvxBR+BeFrzguM/kQn9MBxVIZIBcEg rENokd1MJ3R6xXc4pkC/UUOrClu2Kz9pH6D1G/aawIFesgzef6cNF11bccZ6aJ8wAXIt E66Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737689787; x=1738294587; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BbVs3gHhzsPOdm8N4N6XVsQByDEeLJyfrHs6UonRrQs=; b=Gl+6kwFAglGWugu+1HhD3hEIZd+AVlV5/ZdlK0kemBKUFm9g0ELdrAmCh76OGJTuxy W17TThBieZiSL7ilb3kpEJnSoDzBCwOnnb3f1plSA7I7aZe4V5VF/3MdSbfnky49TqWW 0MNThPIywH3TLuksiUn7kEK13DAPf6uirrxJa2YlTMdCpDUhHsSzHxwHklm2MaVtafDE Wy5yEalFgLsWmzny6vgeOLg128KG7Kx9gY3uUzNBsGq3R7gecBsGh8nbq9cY42D69VN0 so7hz9lbBAYoKWmFiWH5NY8fvKlxVglfezSnVDkByZVuArjh4RuVP7JIyctwsPZoHBIk tktQ== X-Forwarded-Encrypted: i=1; AJvYcCVWuS9q8E7+Gc1NxHI/oy+pNCt6Nd7/iEKlpDL5Bmi+T9wHxKt2olRfho/yGb5/flh4yHWEKRoMifnR+g==@postgresql.org X-Gm-Message-State: AOJu0Yx58ssugdubPlcE8V8+dKx3yShK12Fu9SunFW53OQQ+uN3VOa8G laiu6s/fL++71VnbOFqS5LvNoCSeRQPj3IJY5YD9Kah+oPi63nz/LER7BUsyXnNk3OoFaHRn/eS xaPDFBmz9RodzXLkdVYNvP3s+aLE= X-Gm-Gg: ASbGncuOMfk2E1stL8X/VDwVBserZ9rJbF7mfznYe2oABMTOahC1pL6TPlh9sVDsFc/ GDlvlJnkB25YcS0RhgAnJvaJ+oTR85doHjMWF5aJprQyHtMdCrB04FLmL4ZF9rVcHa+bkcXJOtR /icLU= X-Google-Smtp-Source: AGHT+IHMv+zjd4R2vCkg3qqEvAI7gcLktka+v64CVYPRrEDmJ9CQU/7UpBvAN/RrbAAftu2IpxlRj4ARFz/dX/zaIPM= X-Received: by 2002:a05:651c:1602:b0:300:8de:d2a4 with SMTP id 38308e7fff4ca-3072ca9f48fmr108420991fa.17.1737689786386; Thu, 23 Jan 2025 19:36:26 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Rajesh Kumar Date: Fri, 24 Jan 2025 09:06:13 +0530 X-Gm-Features: AWEUYZliyC5wlkQeh8Hj3vcT3knQc7PeNu14cLIkMQUntuGcduF629YnWXoedIw Message-ID: Subject: Re: Archiving solutions To: kasem adel Cc: Imran Khan , pgsql-admin Content-Type: multipart/alternative; boundary="000000000000729005062c6b6e09" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000729005062c6b6e09 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thank you Imran On Tue, 14 Jan 2025, 04:38 kasem adel, wrote: > Dear imeran , > > Thanks for your support. > > Dears, > > Appreciate your support in simple solution to implement it. > > Thanks > > > > =D9=81=D9=8A =D8=A7=D9=84=D8=A3=D8=AD=D8=AF=D8=8C =D9=A1=D9=A2 =D9=8A=D9= =86=D8=A7=D9=8A=D8=B1 =D9=A2=D9=A0=D9=A2=D9=A5 =D9=A1=D9=A1:=D9=A2=D9=A3 = =D9=85 Imran Khan =D9=83=D8=AA=D8=A8: > >> Hi, >> >> How the data can be stored. Do you have historical data? If yes , then >> you can keep that data in archived tables or if that data is still requi= red >> by business then store it in partitions monthly wise . You need to desig= n >> the data storage with proper planning. Also, if you have dynamic and eve= r >> changing data and you have date records then store with respect to >> partitions and retain for last 3 or 6 months then move to archived table= s >> or purge if not required .. keep track of indexes and capture long runni= ng >> SQL statements through pg_stat_statements extension and run explain plan= to >> get the proper indexes created based on the output of that plan. Also, k= eep >> track of memory related parameters to check if any memory leak occurs >> resulting in EXCESSIVE swap space usage on OS. >> >> That's my opinion but we have more experts here who can help us to >> understand more. >> >> Thanks, >> Imran >> >> On Sun, Jan 12, 2025, 11:57=E2=80=AFPM kasem adel = wrote: >> >>> Hi, >>> >>> What is the regular housekeeping. >>> >>> Thanks >>> >>> =D9=81=D9=8A =D8=A7=D9=84=D8=A3=D8=AD=D8=AF=D8=8C =D9=A1=D9=A2 =D9=8A= =D9=86=D8=A7=D9=8A=D8=B1 =D9=A2=D9=A0=D9=A2=D9=A5 =D9=A1=D9=A0:=D9=A5=D9=A5= =D9=85 Imran Khan =D9=83=D8=AA=D8=A8: >>> >>>> Hi, >>>> >>>> Plan to use table partitioning and do regular housekeeping of the >>>> cluster. >>>> >>>> Thanks, >>>> Imran >>>> >>>> On Sun, Jan 12, 2025, 11:42=E2=80=AFPM kasem adel wrote: >>>> >>>>> Dears , >>>>> >>>>> Appreciate your support I have 2.4 TB database and I need to implemen= t >>>>> archiving solutions to prevent data growthing. >>>>> >>>>> Thanks >>>>> >>>> --000000000000729005062c6b6e09 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Thank you Imran


On Tue, 14 Jan 2025, 04:38 kasem adel, <kasemadel8@gmail.com> wrote:
Dear imeran ,
<= br>
Thanks for your support.
=
Dears,

Appreciate your support in simple solution to implement it.

Thanks



=D9=81=D9=8A =D8=A7=D9=84=D8=A3=D8= =AD=D8=AF=D8=8C =D9=A1=D9=A2 =D9=8A=D9=86=D8=A7=D9=8A=D8=B1 =D9=A2=D9=A0=D9= =A2=D9=A5 =D9=A1=D9=A1:=D9=A2=D9=A3 =D9=85 Imran Khan <imra= n.k.23@gmail.com> =D9=83=D8=AA=D8=A8:
Hi,

=C2=A0How the data can be stored. Do you have historical=C2=A0data? If ye= s , then you can keep that data in archived tables or if that data is still= required by business then store it in partitions monthly wise . You need t= o design the data storage with proper planning. Also, if you have dynamic a= nd ever changing data and you have date records then store with respect to = partitions and retain for last 3 or 6 months then move to archived tables o= r purge if not required .. keep track of indexes and capture long running S= QL statements through pg_stat_statements extension and run explain plan to = get the proper indexes created based on the output of that plan. Also, keep= track of memory related parameters to check if any memory leak occurs resu= lting in EXCESSIVE swap space usage on OS.=C2=A0
=C2=A0That's my opinion but we have more exper= ts here who can help us to understand more.

Thanks,
Imran=C2=A0
<= br>
On Sun,= Jan 12, 2025, 11:57=E2=80=AFPM kasem adel <kase= madel8@gmail.com> wrote:
Hi,

What = is the regular housekeeping.=C2=A0

Thanks=C2=A0

=D9=81=D9=8A =D8=A7=D9=84=D8=A3=D8=AD=D8=AF=D8=8C = =D9=A1=D9=A2 =D9=8A=D9=86=D8=A7=D9=8A=D8=B1 =D9=A2=D9=A0=D9=A2=D9=A5 =D9=A1= =D9=A0:=D9=A5=D9=A5 =D9=85 Imran Khan <imran.k.23@gmail.com> =D9=83=D8=AA=D8=A8:
Hi,

Plan to use table partitioning and do regular housekeeping of the= cluster.=C2=A0

Thanks,<= /div>
Imran

On Sun, Jan 12, 2025, 11:42=E2=80=AFPM ka= sem adel <kasemadel8@gmail= .com> wrote:
Dears ,

Appreciate your sup= port I have 2.4 TB database and I need to implement archiving solutions to = prevent data growthing.=C2=A0

Thanks=C2=A0
--000000000000729005062c6b6e09--