X-Original-To: pgsql-sql-postgresql.org@postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 41CD19FA5E6 for ; Sun, 11 Jun 2006 22:39:21 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 80487-01 for ; Sun, 11 Jun 2006 22:39:16 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from weatherbeeta.com.au (mail.weatherbeeta.com.au [202.154.127.242]) by postgresql.org (Postfix) with SMTP id 99AE59F9CAA for ; Sun, 11 Jun 2006 22:39:14 -0300 (ADT) Received: (qmail 10719 invoked from network); 12 Jun 2006 01:39:11 -0000 X-NetworkBox-HamSign: 0101;OUT;wbeetaau1;53e1afad5e5df18eb6e5131e392e5698; Received: from mail.weatherbeeta.com.au (HELO wb-email) (172.20.0.2)by network-box.weatherbeeta.com.au with SMTP; 12 Jun 2006 01:39:11 -0000 Received: from 172.20.0.155 ([172.20.0.155]) by wb-email (602Pro LAN SUITE 2003) id 2d52cf00 for pgsql-sql@postgresql.org; Mon, 12 Jun 2006 11:37:23 +1000 Reply-To: From: "Phillip Smith" To: Subject: COPY to table with array columns (Longish) Date: Mon, 12 Jun 2006 11:38:59 +1000 Organization: Weatherbeeta P/L Message-ID: <004701c68dc0$fc721510$9b0014ac@ITPhil> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0048_01C68E14.CE1E2510" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.6626 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 Sensitivity: Company-Confidential X-Scanned-By-wbeetaau1: Virus scan performed by network-box X-Scanned-By-wbeetaau1: Scanner file id is wbeetaau1-1150076351.778-10714-000 X-Scanned-By-wbeetaau1: No known viruses found in message (received+scanned in 0.05/0.12 secs) X-Scanned-By-wbeetaau1: Spam-Check-Result: No, hits=0 required=7 tests= autolearn=no version=2.0 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200606/120 X-Sequence-Number: 25356 This is a multi-part message in MIME format. ------=_NextPart_000_0048_01C68E14.CE1E2510 Content-Type: text/plain;charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Hi All, =20 Hope someone can help me - our main company system runs on Raining Data PICK/D3 (if anyone familiar with it) which stores records in it's = "tables" as variable length items. Every item has a unique Primary Key (per = table) then each item can have a variable number of fields. These fields are delimited by Char 254, then each field can have sub-values delimited by = Char 253, then sub-sub-values delimited by Char 252. =20 Anyway, we are trying to export everything to Postgres for reporting and querying etc (not to actually run the system.. Yet) and hasn't been a problem so far - everything like stock and purchase orders, sales orders = etc can pretty easily be turned in to a flat file with standard number of columns and consistent data. We truncate every table each night then = import that latest TSV export from D3 using a COPY command. =20 The problem arises with tables like our SYS table which store generic = system data, so one record could have 3 fields, but the next could have 300. = The only way I can work out how to export multi-valued data like this to Postgres is to use an array column. So the table has 2 columns - the = pkey and a data array. =20 How do I get this imported to the truncated table each night? At the = moment I think my best option is to modify the export for the SYS table to call PSQL and use standard SQL INSERT statements to directly insert it = instead of exporting to a flat file, then import to Postgres. =20 Thanks all, -p =20 For those who are interested, or if it might help, here's a rough = comparison of the database structure of D3: Windows =3D PICK/D3 =3D Postgres Drive =3D Account =3D Database Directory =3D File =3D Table File =3D Item =3D Row Line in text file =3D Attribute =3D Field (none) =3D Value =3D Array Element (?) (none) =3D Sub Value =3D (none?) =20 Phillip Smith IT Coordinator Weatherbeeta P/L 8 Moncrief Rd Nunawading, VIC, 3131 AUSTRALIA =20 E. phillips@NO-SPAM.weatherbeeta.com.au =20 *******************Confidentiality and Privilege Notice******************* The material contained in this message is privileged and confidential to the addressee. If you are not the addressee indicated in this message or responsible for delivery of the message to such person, you may not copy or deliver this message to anyone, and you should destroy it and kindly notify the sender by reply email. Information in this message that does not relate to the official business of Weatherbeeta must be treated as neither given nor endorsed by Weatherbeeta. Weatherbeeta, its employees, contractors or associates shall not be liable for direct, indirect or consequential loss arising from transmission of this message or any attachments ------=_NextPart_000_0048_01C68E14.CE1E2510 Content-Type: text/html;charset="US-ASCII" Content-Transfer-Encoding: quoted-printable

Hi All,

 

Hope someone can help me – our main company = system runs on Raining Data PICK/D3 (if anyone familiar with it) which stores = records in it’s “tables” as variable length items. Every item = has a unique Primary Key (per table) then each item can have a variable number = of fields. These fields are delimited by Char 254, then each field can have sub-values delimited by Char 253, then sub-sub-values delimited by Char = 252.

 

Anyway, we are trying to export everything to = Postgres for reporting and querying etc (not to actually run the system…. Yet) = and hasn’t been a problem so far – everything like stock and = purchase orders, sales orders etc can pretty easily be turned in to a flat file = with standard number of columns and consistent data. We truncate every table = each night then import that latest TSV export from D3 using a COPY = command.

 

The problem arises with tables like our SYS table = which store generic system data, so one record could have 3 fields, but the = next could have 300. The only way I can work out how to export multi-valued = data like this to Postgres is to use an array column. So the table has 2 columns = – the pkey and a data array.

 

How do I get this imported to the truncated table = each night? At the moment I think my best option is to modify the export for = the SYS table to call PSQL and use standard SQL INSERT statements to directly = insert it instead of exporting to a flat file, then import to = Postgres.

 

Thanks all,

-p

 

For those who are interested, or if it might help, = here’s a rough comparison of the database structure of D3:

Windows     =       =3D     = PICK/D3     =3D     = Postgres

Drive       =       =3D     = Account     =3D     = Database

Directory   =       =3D     = File        = =3D     Table

File        =       =3D     = Item        = =3D     Row

Line in text file =3D     = Attribute   =3D     = Field

(none)      =       =3D     = Value       =3D     = Array Element (?)

(none)      =       =3D     Sub Value   =3D     (none?)

 

Phillip Smith

IT Coordinator

Weatherbeeta P/L

8 Moncrief Rd

Nunawading, VIC, 3131

AUSTRALIA

 

E. = phillips@NO-SPAM.weatherbeeta.com.au

 


*******************Confidentiality and Privilege Notice*******************

The material contained in this message is privileged and confidential to the addressee. If you are not the addressee indicated in this message or responsible for delivery of the message to such person, you may not copy or deliver this message to anyone, and you should destroy it and kindly notify the sender by reply email.

Information in this message that does not relate to the official business of Weatherbeeta must be treated as neither given nor endorsed by Weatherbeeta. Weatherbeeta, its employees, contractors or associates shall not be liable for direct, indirect or consequential loss arising from transmission of this message or any attachments
------=_NextPart_000_0048_01C68E14.CE1E2510--