Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jQyjg-0007RQ-BF for pgsql-admin@arkaria.postgresql.org; Tue, 21 Apr 2020 19:39:12 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jQyjf-0003jU-7g for pgsql-admin@arkaria.postgresql.org; Tue, 21 Apr 2020 19:39:11 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jQyje-0003jN-RC for pgsql-admin@lists.postgresql.org; Tue, 21 Apr 2020 19:39:10 +0000 Received: from mail-wr1-x435.google.com ([2a00:1450:4864:20::435]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1jQyjb-0005e0-JC for pgsql-admin@lists.postgresql.org; Tue, 21 Apr 2020 19:39:10 +0000 Received: by mail-wr1-x435.google.com with SMTP id s10so8805109wrr.0 for ; Tue, 21 Apr 2020 12:39:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cybertec-at.20150623.gappssmtp.com; s=20150623; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=W5J5o/FrcZycGSu8F+28sQ7nh2biR4QW+fo+BVgZK2Y=; b=H0v8ZYSAgBix/5M7FBtUjOyF3vyeipfcfm1LWBZWS7iayVheGb5/gH/kfVyrtIug2R pLi1teXVcdds8Gy/WZsifRnIlqSQBZZMG5ut7SwYWtDZtYuIvDe/Two9qO/Hs2LV+YZQ hluktcHYFzjMkY6+wPnj4DNnzFsHwaenTAInjMGkIxcsTADk13M9AVTgCyY2YAd6qziD j8zUkf95VZLD8N9DKVoNGoq9YKoOCLaA3vNR0f4C/LEgQbTZ4FtQveFldc5qXZl001ie u7cBD/3Z+pGfAA59x1aO3DNtrVtVzUQ7w2KyUJlBHDSp/E3bOlVJKi6KzXSoiMBvJ58a C3ZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=W5J5o/FrcZycGSu8F+28sQ7nh2biR4QW+fo+BVgZK2Y=; b=I1dNqafM98zu3Z8RvofjhcRPmgEs1FjylGeHAAGSKfbZOxR4HslZHNg3Y5uXVBs9to i5Jw8osAgqOjpowPI+Ob7T6qK4hfl067EAhE9+uHt6UQAHg+a4iQl6DnLyZyC2oeUQ6r ojiy/UeSASb4LYyx+MVj6ltGt/o9QUtfzfWXe6UblRShJin2c8tmnSHTMKSdHFnZaIM7 0Xg2MRwbH2c1s6Ua49kQZo0/2bPZm1a7TUQMhziF44z/209pSHxNUEePHIqzlidOUrUV aN6AJIkyk6AHMk+TG85UPDx5kSTtEcuRpW/aAFiSPEm0bNW9huSmjdm5lnZJzzXT9Z0d r6lQ== X-Gm-Message-State: AGi0PuZiscG3h3dB4NPyNx2ozh/MFGDHSpamn6amsDIKYcxijVfoGbYy Tks18tR6LhZGwkBAyMLII6to9Q== X-Google-Smtp-Source: APiQypLutdMP2WEZycdQY8JNOQrtcTrAy3Np63JCS+RXn+PTpeNi3pr4MLyZgavlUROy7MrXtJWqJA== X-Received: by 2002:adf:f146:: with SMTP id y6mr27289268wro.132.1587497946388; Tue, 21 Apr 2020 12:39:06 -0700 (PDT) Received: from dynamic-enq1j2om7y8y91ixf-pd01.res.v6.highway.a1.net (dynamic-enq1j2om7y8y91ixf-pd01.res.v6.highway.a1.net. [2001:871:60:80f6:1713:58c3:d3cd:8713]) by smtp.gmail.com with ESMTPSA id h6sm4866047wmf.31.2020.04.21.12.39.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2020 12:39:05 -0700 (PDT) Message-ID: <8be53622f7c7d152d550eed59ccc065421df8a0e.camel@cybertec.at> Subject: Re: checkpoint starting: time (insert statement hangs From: Laurenz Albe To: Yannick Collette , pgsql-admin@lists.postgresql.org Date: Tue, 21 Apr 2020 21:39:05 +0200 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.32.5 (3.32.5-1.fc30) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Tue, 2020-04-21 at 12:10 -0400, Yannick Collette wrote: > I'm having an issue with Insert statement (through oracle_fdw) that hangs after checkpoint starting is "time" instead of "xlog". see end of log below. > Migrating data from Oracle (Windows 2008 R2) to Postgresql (Windows 2012 R2). Both systems are virtual machines residing on same host. > Over 200 migrations have been done without any issues. Trying to understand what's happening here... > > 2020-04-20 14:02:36.103 EDT,5e9dbe0e.990,126,,,,00000,LOG: checkpoint complete: wrote 112333 buffers (28.6%); 0 trans > 2020-04-20 14:07:37.793 EDT,5e9dbe0e.990,131,,,,00000,LOG: checkpoint starting: xlog > 2020-04-20 14:09:50.222 EDT,5e9dbe0e.990,132,,,,00000,LOG: checkpoint complete: wrote 112903 buffers (28.7%); 0 transaction log file(s) added, 0 removed, 44 recycled; write=132.240 s, sync=0.078 s, > total=132.428 s; sync files=8, longest=0.034 s, average=0.010 s; distance=720594 kB, estimate=722102 kB > 2020-04-20 14:10:03.574 EDT,5e9dbe0e.990,133,,,,00000,LOG: checkpoint starting: xlog > 2020-04-20 14:19:03.160 EDT,5e9dbe0e.990,134,,,,00000,LOG: checkpoint complete: wrote 113066 buffers (28.8%); 0 transaction log file(s) added, 0 removed, 44 recycled; write=539.492 s, sync=0.000 s, > total=539.586 s; sync files=8, longest=0.004 s, average=0.001 s; distance=721542 kB, estimate=722046 kB > 2020-04-20 14:20:03.172 EDT,5e9dbe0e.990,135,,,,00000,LOG: checkpoint starting: time > 2020-04-20 14:29:03.255 EDT,5e9dbe0e.990,136,,,,00000,LOG: checkpoint complete: wrote 97752 buffers (24.9%); 0 transaction log file(s) added, 0 removed, 44 recycled; write=539.911 s, sync=0.000 s, > total=540.083 s; sync files=7, longest=0.004 s, average=0.001 s; distance=623883 kB, estimate=712230 kB > > The transaction still active in pg_stat_activity but not writing on disk since last checkpoint complete. Non errors after last checkpoint. Also no errors in logs on Oracle side. > Any help or hints would be greatly appreciated. How does pg_stat_activity look? Is there an entry with "granted = FALSE" in pg_locks? What is the PostgreSQL query and the Oracle query? What is the state of the statement in Oracle? Is it running, blocked or waiting for the client? Yours, Laurenz Albe -- Cybertec | https://www.cybertec-postgresql.com