pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: PG Bug reporting form <noreply@postgresql.org>
To: pgsql-bugs@lists.postgresql.org
Cc: mark.lai@integrafec.com
Subject: BUG #15333: pg_dump error on large table -- "pg_dump: could not stat file...Unknown error"
Date: Thu, 16 Aug 2018 12:51:54 +0000
Message-ID: <153442391458.1505.9181095584291689853@wrigleys.postgresql.org> (raw)

The following bug has been logged on the website:

Bug reference:      15333
Logged by:          Mark Lai
Email address:      mark.lai@integrafec.com
PostgreSQL version: 10.4
Operating system:   Windows Server 2016 Server
Description:        

Hi,

When I pg_dump a large table (> 35 GB), I get the following error message:


pg_dump: could not stat file
"O:\postgres-server3\test#test#large_test/2793.dat.gz": Unknown error

The dump however appears to restore correctly.

**************************************
**Script used to create the table**
**************************************
CREATE SCHEMA IF NOT EXISTS test;

DROP TABLE IF EXISTS test.large_test;
CREATE UNLOGGED TABLE test.large_test (num1 bigint, num2 double precision,
num3 double precision);

INSERT INTO test.large_test (num1, num2, num3)
  SELECT round(random()*10), random(), random()*142
  FROM generate_series(1, 40*20000000) s(i);
  --20000000 ~ 1GB

*****************************************************
**power shell script used to launch pg_dump**
*****************************************************
$database = "test"
$schema = "test"
$table = "large_test"

$baseOutputDirectory = "O:\postgres-server3\"
$baseLogDirectory = "O:\postgres-server3\logs\"

############################


New-Item -ItemType Directory -Force -Path $baseOutputDirectory 
New-Item -ItemType Directory -Force -Path $baseLogDirectory


$schemaTable = "$($database)#$($schema)#$($table)"
$outputDirectory = "$($baseOutputDirectory)$($schemaTable)"
$logFile = "$($baseLogDirectory)$($schemaTable).log"

Remove-Item $outputDirectory -Force -Recurse -ErrorAction Ignore
Remove-Item $logFile -Force -Recurse -ErrorAction Ignore

Start-Job -Name $schemaTable -ScriptBlock{& "C:\Program
Files\PostgreSQL\10\bin\pg_dump.exe" --verbose --host=localhost --port=5432
--username=test123123 --no-password --jobs=1 --format=directory
--table=$Using:schema.$Using:table --file=$Using:outputDirectory
$Using:database 1> $Using:logFile 2>&1 }

************
**Log file**
*************
pg_dump.exe : pg_dump: last built-in OID is 16383
At line:1 char:1
+ & "C:\Program Files\PostgreSQL\10\bin\pg_dump.exe" --verbose --host=l
...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (pg_dump: last built-in OID is
16383:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
pg_dump: reading extensions
pg_dump: identifying extension members
pg_dump: reading schemas
pg_dump: reading user-defined tables
pg_dump: reading user-defined functions
pg_dump: reading user-defined types
pg_dump: reading procedural languages
pg_dump: reading user-defined aggregate functions
pg_dump: reading user-defined operators
pg_dump: reading user-defined access methods
pg_dump: reading user-defined operator classes
pg_dump: reading user-defined operator families
pg_dump: reading user-defined text search parsers
pg_dump: reading user-defined text search templates
pg_dump: reading user-defined text search dictionaries
pg_dump: reading user-defined text search configurations
pg_dump: reading user-defined foreign-data wrappers
pg_dump: reading user-defined foreign servers
pg_dump: reading default privileges
pg_dump: reading user-defined collations
pg_dump: reading user-defined conversions
pg_dump: reading type casts
pg_dump: reading transforms
pg_dump: reading table inheritance information
pg_dump: reading event triggers
pg_dump: finding extension tables
pg_dump: finding inheritance relationships
pg_dump: reading column info for interesting tables
pg_dump: finding the columns and types of table "test.large_test"
pg_dump: flagging inherited columns in subtables
pg_dump: reading indexes
pg_dump: reading extended statistics
pg_dump: reading constraints
pg_dump: reading triggers
pg_dump: reading rewrite rules
pg_dump: reading policies
pg_dump: reading row security enabled for table "test.large_test"
pg_dump: reading policies for table "test.large_test"
pg_dump: reading publications
pg_dump: reading publication membership
pg_dump: reading publication membership for table "test.large_test"
pg_dump: reading subscriptions
pg_dump: reading dependency data
pg_dump: saving encoding = UTF8
pg_dump: saving standard_conforming_strings = on
pg_dump: saving search_path = 
pg_dump: dumping contents of table "test.large_test"
pg_dump: could not stat file
"O:\postgres-server3\test#test#large_test/2793.dat.gz": Unknown error



view thread (8+ messages)  latest in thread

Message-ID: <153442391458.1505.9181095584291689853@wrigleys.postgresql.org>
Permalink:  ../153442391458.1505.9181095584291689853@wrigleys.postgresql.org/
Also on:    postgresql.org/message-id/153442391458.1505.9181095584291689853@wrigleys.postgresql.org

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: noreply@postgresql.org, pgsql-bugs@lists.postgresql.org, mark.lai@integrafec.com
  Subject: Re: BUG #15333: pg_dump error on large table -- "pg_dump: could not stat file...Unknown error"
  In-Reply-To: <153442391458.1505.9181095584291689853@wrigleys.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox