Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lw674-0006Bh-2G for pgsql-hackers@arkaria.postgresql.org; Wed, 23 Jun 2021 16:52:30 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lw672-0007UL-Sm for pgsql-hackers@arkaria.postgresql.org; Wed, 23 Jun 2021 16:52:28 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lw5eP-0007wl-FW for pgsql-hackers@lists.postgresql.org; Wed, 23 Jun 2021 16:22:53 +0000 Received: from mail-qk1-x72a.google.com ([2607:f8b0:4864:20::72a]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lw5eM-0001Gu-Id for pgsql-hackers@lists.postgresql.org; Wed, 23 Jun 2021 16:22:52 +0000 Received: by mail-qk1-x72a.google.com with SMTP id bl4so6614414qkb.8 for ; Wed, 23 Jun 2021 09:22:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:mime-version:content-disposition :content-transfer-encoding:in-reply-to; bh=ebLhSLqu6GcneFy8ixOoWPxcQeU1jsY6OexcxjmTEKM=; b=MbZjF0y4gPaZgsAOW5hmwio4qz4bx2sZ9VfnKlZeqzP2cyCCVVE0zfh47P1AOhhAEX Msx7qhVQuVUBf4sf1wzefCjCFKUcaRPVs4yrRXAQmW34AqpmP0Klh6ij68lpHaz5hrrD dS9K3wC+Gv1HpK/TsJinnkZ7NecKSYFkyIaHdwUpdfp92Y2Yy2uUigQkX9peI66Gxd0G vfgLGLuPjUgCk7zHVeg6bCA67naiw5/awrEom+txW8QgG8r24o3EcRlQyMmApRFsVnEz Czha6h607kJTuee4SRDeL1TPCU83rLFCbWxp97Une0jnvJIRDkVxUJIlY8HWo19AKP4y PrVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=ebLhSLqu6GcneFy8ixOoWPxcQeU1jsY6OexcxjmTEKM=; b=iF5FiiPZlgXE2zsilcTpTULhzxQbulLW7JO9tmWkUO3CTEvNBoBNTrAv2VjSgEAlfE GlunJ2BvJKwnLA8p5mpg3C2lgckOYckGjdeRV9zbuoorcRCOjygddfQ4yy7HJY6CYz2D CMqP6gwyJ9vFl1uOhgr9cpUF0L7i5GKbzVPEfm3y9E/HS6eVhtr5S0FgoxqXzfm+TXZQ TNrg8KcrjziZHwuR4bAA1i1mA4Dl1asoVYnCaYyKWy9eRP0BZyaAEvPzoBseyoVyGQti 5aILLM9KglUEu7Y7zKCpIu1iN+d5B/qqRO4nBK5xwX6scy53usZCls4eq7m3KHaAYZSj bhZw== X-Gm-Message-State: AOAM530E3Ryd9P5i5JBmkctFw+D5B5wzkVTPgws+to5J+ePC5Lkoq/Tq lGoTwt2ZsSCrWU9vuRlE1IwdNqunnYFHzg== X-Google-Smtp-Source: ABdhPJw5yoJKdR/pyHMwqSGuqSZwsu8LXHujjmcvBBS3wGx7W/NI9s9h2briY59xvvokxoPODDA9nw== X-Received: by 2002:a37:642:: with SMTP id 63mr843268qkg.400.1624465369052; Wed, 23 Jun 2021 09:22:49 -0700 (PDT) Received: from perhan.alvh.no-ip.org ([190.95.19.95]) by smtp.gmail.com with ESMTPSA id w2sm259508qto.50.2021.06.23.09.22.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jun 2021 09:22:48 -0700 (PDT) Received: by perhan.alvh.no-ip.org (Postfix, from userid 1000) id 6706E2A0ADA; Wed, 23 Jun 2021 12:22:46 -0400 (-04) Date: Wed, 23 Jun 2021 12:22:46 -0400 From: Alvaro Herrera To: Boris Kolpackov Cc: pgsql-hackers@lists.postgresql.org Subject: Re: Pipeline mode and PQpipelineSync() Message-ID: <202106231622.ppuztv2vegix@alvherre.pgsql> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 2021-Jun-23, Boris Kolpackov wrote: > I think one change that is definitely needed is to make it clear that > the PQpipelineSync() call is not optional. > > I would also add a note saying that while the server starts processing > the pipeline immediately, it may buffer the results and the only way > to flush them out is to call PQpipelineSync(). Curious -- I just noticed that the server understands a message 'H' that requests a flush of the server buffer. However, libpq has no way to generate that message as far as I can see. I think you could use that to request results from the pipeline, without the sync point. I wonder if it's worth adding an entry point to libpq to allow access to this. PQrequestFlush() or something like that ... Prior to pipeline mode this has no use (since everything ends with ReadyForQuery which involves a flush) but it does seem to have use in pipeline mode. -- Álvaro Herrera 39°49'30"S 73°17'W