Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ncs2t-0006cX-6h for pgsql-sql@arkaria.postgresql.org; Fri, 08 Apr 2022 17:05:15 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1ncs2s-0005uu-2k for pgsql-sql@arkaria.postgresql.org; Fri, 08 Apr 2022 17:05:14 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ncs2r-0005ul-Oh for pgsql-sql@lists.postgresql.org; Fri, 08 Apr 2022 17:05:13 +0000 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1ncs2p-0004Aa-K6 for pgsql-sql@lists.postgresql.org; Fri, 08 Apr 2022 17:05:12 +0000 Received: by mail-pj1-x102b.google.com with SMTP id nt14-20020a17090b248e00b001ca601046a4so12495139pjb.0 for ; Fri, 08 Apr 2022 10:05:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :references:from:in-reply-to; bh=0PDlYCQfAhpjt07qrQD3KALmjXsgA5QskHbStSEaBnU=; b=RlFzbPmNH5a5L9TZgqe13hDH7WqWmF9yU16+mFJvYiwXPCK9KpMt+eQ6HNu/cdwxh7 p0Syt4o9H89RLD6Hrhe39WaedFQHj46X2MzQZfP5kqIHnYDWBpNgFy4ajWGB0eng2i3s F1Uln++2wfliyJLreMBwSwULpNSHsILd3oth13tOzEyM8Kgvy4ixpbprtFga9K8NSDmz l9y6ELNjeeTaCsHtZFy5nJVYNhehI7aU2lMMBnH1rqiZGgqoozmgE/7SmOl+Amul3+8j GJ1xpK7voy77N+eEgqvLm8dYI49cw1hJtulkp9LeM/gZmumU64Xs2oEwIsOwhQ8Jw6TX YlAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:references:from:in-reply-to; bh=0PDlYCQfAhpjt07qrQD3KALmjXsgA5QskHbStSEaBnU=; b=l6ybzVjjRIK/kTGvJsJBqSZ1LlYT7HMswGSeBmXRi7Zs8s07t9CyXCSaB4ACmBy5VF /LRicBvrkRFKSGf3pwS+DXuxSFEPNzoTjb9jFaLEUxHOkmdQiP5mYiD1fBIfED6SNeTi p3lkh0mlcBiBF0yegHqt5fsffFLM+7W070/eufCxuWOZ4xdPIdgDlwGQhz7+cphIjt4w T42OHg/577v0lEJH3m1+Fl22Qyz8kpgRseNuYmkawWoDHorBYD2mJwo5bojRceg+v8wA Cf3A9jcTtcys0hElj/DpgqSFuC4WPrcMwccMefiYeeHjW0hhwhe7tw5doRxZwFIFUYqY fLoA== X-Gm-Message-State: AOAM532lDvZombXw0ysefe9jL9vg7qbNer9fyWstMg3WNjEvdDKpdoGd QjuwzjuirhSulRMv8mMFM/FgRVegscM= X-Google-Smtp-Source: ABdhPJxeCnGwI+OYs7ZaP67z+sPohXGOEfxxkLf4oQg3NJMg4fIrhUT6A92CvF3BTcVZQEqInUUTUw== X-Received: by 2002:a17:902:ea07:b0:157:f83:5e08 with SMTP id s7-20020a170902ea0700b001570f835e08mr8250787plg.1.1649437510065; Fri, 08 Apr 2022 10:05:10 -0700 (PDT) Received: from [10.128.71.194] ([155.98.131.2]) by smtp.gmail.com with ESMTPSA id z7-20020a056a00240700b004e1cde37bc1sm27287842pfh.84.2022.04.08.10.05.09 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 08 Apr 2022 10:05:09 -0700 (PDT) Content-Type: multipart/alternative; boundary="------------X4xJ0rNW2WbcODmgCXLnHZ80" Message-ID: <17a3b7ea-5334-c3f6-577b-a582a64d7548@gmail.com> Date: Fri, 8 Apr 2022 11:05:08 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: Long running processes and commit writing to disk Content-Language: en-CA To: pgsql-sql@lists.postgresql.org References: <02f801d84b69$06336370$129a2a50$@runbox.com> From: Rob Sargent In-Reply-To: <02f801d84b69$06336370$129a2a50$@runbox.com> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------X4xJ0rNW2WbcODmgCXLnHZ80 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 4/8/22 10:52, Mike Sofen wrote: > > *From:* Shaozhong SHI *Sent:* Friday, April > 08, 2022 9:39 AM > > When long running processes got disrupted, one may not see any > expected result. > > How to make sure that the result of each operation is saved to disk in > a loop? > > Regards, > > David > > Simple:  don’t use a database – you’re asking to violate ACID. > > A database’s power comes from set-based operations.  You’re asking to > do single row operations.  Use a spreadsheet...or a plain text file > and treat it like a log. > > Mike > Or commit every loop (perhaps every N loops, depending on how much you're prepared to re-do.) --------------X4xJ0rNW2WbcODmgCXLnHZ80 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
On 4/8/22 10:52, Mike Sofen wrote:

From: Shaozhong SHI <shishaozhong@gmail.com>   Sent: Friday, April 08, 2022 9:39 AM

When long running processes got disrupted, one may not see any expected result.

How to make sure that the result of each operation is saved to disk in a loop?

Regards,

David

 

 

Simple:  don’t use a database – you’re asking to violate ACID. 

 

A database’s power comes from set-based operations.  You’re asking to do single row operations.  Use a spreadsheet...or a plain text file and treat it like a log.

 

Mike

Or commit every loop (perhaps every N loops, depending on how much you're prepared to re-do.)
--------------X4xJ0rNW2WbcODmgCXLnHZ80--