Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1v1rxY-00DVb9-1g for pgsql-general@arkaria.postgresql.org; Thu, 25 Sep 2025 19:48:56 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1v1rxW-005Plz-PN for pgsql-general@arkaria.postgresql.org; Thu, 25 Sep 2025 19:48:54 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1v1r2c-004mGL-Cd for pgsql-general@lists.postgresql.org; Thu, 25 Sep 2025 18:50:06 +0000 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1v1r2Z-0004Y1-1s for pgsql-general@lists.postgresql.org; Thu, 25 Sep 2025 18:50:05 +0000 Received: by mail-qt1-x830.google.com with SMTP id d75a77b69052e-4dce9229787so2758531cf.0 for ; Thu, 25 Sep 2025 11:50:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1758826201; x=1759431001; darn=lists.postgresql.org; h=subject:content-language:to:user-agent:mime-version:date:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=RZqaSQHjUo8FzOD9MzbM9wURdlsybxJghUXmZEBPMyQ=; b=FUDUjmFF9PHdgfH1ExwemDcAn/RlElBBLOrKK3iuHDexw+SOwy0Rv8Ns85bYrMpgJM 2vtbVn+//GLjwRuZobo7tRmL8QVdIjA2eoKmrKhf1YXIU2WwV/9iwCKryRqLvaQP3Jbv B8x37MWBAxXZKETUHn4GSVj/BG7ivLsEQ4MfKGnAwrbjQxKUgnQS+lVgTmFHh373EmR4 /KJ4wK2O32Bf8onSOY303jscjoA6fMLt9jUmmADHvzxNaYw3iU5wnpv8jUP5QXZk5i4D EdcEwjA75UblrHJetVH9Jnb8lf+nrBjIhsAv83G5VG0tQCxE6rp5+FSMFKAzan2TkuO1 FnoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758826201; x=1759431001; h=subject:content-language:to:user-agent:mime-version:date:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RZqaSQHjUo8FzOD9MzbM9wURdlsybxJghUXmZEBPMyQ=; b=SUiu6lAIWlVDunyym3xIsCZklbu8yVOzuizI9YSvHwQIU8Z8ttXOmvbCQ4YmonH48N +Koxj3tk0WoR4wp5ZK7Fq74pR4EEgjEjlqZdWc1IMg6GSJz/uupJQqoSpQQSiSS50tyL ojEMhEBPW9+mPTeaVquRCZ6HQ/cZUxgcvoMVz44ZRdAt//EaFkhzkF52wvu/ksD7VwnL sMw7LiCD32ni51PV5vTRC9kqqwKJ389EawejxFpRpCeWW1TwF9C1rzpww9PJCe4EA77f nzTJmX7RpcYypMD/AAibn0zwn1iYHDroX3CccxGDHAvyhqBwdhvnuMbsH8H6WokxQJLe Wl9w== X-Gm-Message-State: AOJu0YwWkTMDV4hJDsIRY4s+mzaZmAF7yGWc+t5p1vy2oFyYZ8NX4l7a NRmcqAOy2VfPoWZCUR49Z4jie31Og6J8tAbc64ZdGDuXHltQ+iBGtrhOC07jrQ== X-Gm-Gg: ASbGncs2AP11eWYOwQUXpxsCzelMyu+WyH4ZVLgDhzzT4uQaQA2rlZnlDLx7Wf1e4g/ bfXVY1bygFfmHAPGSrkxwFzCm1H6gZejZVX4VCGIn9kGxzfM3NLyuDg1Cw08n5lLPABIVhIJFdR pXo0c9eTd+XAZCYJTOjmI06JLmb+iDENW2W5PiN8w9VC8/h3hRINea56DKNOKogVErXe1TUiPZc bvnYnqsjxanbkE36nJIkDMw/Vq4RpMjzRysRvfb0eVP/bsRC7YA/VcEFSCPSpDabI5586+umaw9 f83CEhSIAoo/gV7jFpKzHonlSNTn+NOqKPV+T0CGlw7VMbjoE3XVtpCOiXokZ7UlVIIwO8PFDFq WKydw2mfGhVLScIsWakeDuqQGTPHhXOagwLUDJdmz07b0p+HabNUJblU+ebsV6bfwAzReQaq7wj /r X-Google-Smtp-Source: AGHT+IEO2LVjgQWZTKgKUgj3MkAx6tKhs+daqcHcmjqEoeuENpho0k5rjQ+kt7uYpciSQX3f/cvgJg== X-Received: by 2002:a05:622a:14:b0:4c3:a0ef:9052 with SMTP id d75a77b69052e-4da4cc4d86cmr57846881cf.74.1758826200924; Thu, 25 Sep 2025 11:50:00 -0700 (PDT) Received: from ?IPV6:2600:4040:2d1c:9900:31e6:8deb:e351:f32a? ([2600:4040:2d1c:9900:31e6:8deb:e351:f32a]) by smtp.gmail.com with ESMTPSA id af79cd13be357-85c28a8a7b5sm160203585a.26.2025.09.25.11.50.00 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 25 Sep 2025 11:50:00 -0700 (PDT) From: Bryan Sayer X-Google-Original-From: Bryan Sayer Content-Type: multipart/alternative; boundary="------------yyLsQThhRXh0xt0FP0YKvOQV" Message-ID: <80c44b4d-1185-478b-939c-62e0d4377106@gmail.com> Date: Thu, 25 Sep 2025 14:49:59 -0400 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: "pgsql-general@lists.postgresql.org" Content-Language: en-US Subject: Additional options for COPY from List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------yyLsQThhRXh0xt0FP0YKvOQV Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi, I'm not sure if this is the best list to ask this (and I am very new to PostgreSQL) but I think more options are needed in the COPY from command, in order to better deal with exceptions. By exceptions I mean data not consistent with the format, empty rows, extra delimiters at the end of rows, etc. Is there someone or a party that deals with the details of the COPY command that I could discuss this with? Just for context, I am dealing with delimited data of millions of rows and perhaps 60 columns or so that I wish to read into tables in a Postgres 17 database under Windows 11, with everything on local drives. -- *Bryan Sayer* Retired Demographer/Statistician /In a world in which you can be anything, *be kind*/ --------------yyLsQThhRXh0xt0FP0YKvOQV Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit

Hi,

I'm not sure if this is the best list to ask this (and I am very new to PostgreSQL) but I think more options are needed in the COPY from command, in order to better deal with exceptions. By exceptions I mean data not consistent with the format, empty rows, extra delimiters at the end of rows, etc.

Is there someone or a party that deals with the details of the COPY command that I could discuss this with?

Just for context, I am dealing with delimited data of millions of rows and perhaps 60 columns or so that I wish to read into tables in a Postgres 17 database under Windows 11, with everything on local drives.

--
Bryan Sayer
Retired Demographer/Statistician
In a world in which you can be anything, be kind
--------------yyLsQThhRXh0xt0FP0YKvOQV--