Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oTbzD-0006o1-1T for pgsql-bugs@arkaria.postgresql.org; Thu, 01 Sep 2022 04:39:27 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1oTbzB-0008Dj-Qg for pgsql-bugs@arkaria.postgresql.org; Thu, 01 Sep 2022 04:39:25 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oTbzB-0008Da-IJ for pgsql-bugs@lists.postgresql.org; Thu, 01 Sep 2022 04:39:25 +0000 Received: from mail-oa1-x29.google.com ([2001:4860:4864:20::29]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1oTbz4-0003dz-O5 for pgsql-bugs@postgresql.org; Thu, 01 Sep 2022 04:39:24 +0000 Received: by mail-oa1-x29.google.com with SMTP id 586e51a60fabf-11f0fa892aeso20722696fac.7 for ; Wed, 31 Aug 2022 21:39:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leadboat.com; s=google; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date; bh=HheXFRGg8iQoed1NexPFNP1ri2Hxc37+DtXQ40xlYhg=; b=PL9tcF7MMaKMFc9fw1LQR5JVq8J8j7e89oE+6g7S2ZjFFzIbS3qmdgZPDGJKAAYj/J SOO4bSeX712EsO/rlhMVs9945AXsd6asrUx5HwnGh1wQUbx/5/cdPX5hqzqF/EGQCRuJ iqOi+4mzJHUpuW5SZF+1yzhoMlUUKRvSghK4g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date; bh=HheXFRGg8iQoed1NexPFNP1ri2Hxc37+DtXQ40xlYhg=; b=D2ihQbM+FVu7JiriAuR+aKpxCnryCDVvaA7Fi209C8V8xvWcbupMaekkNFqXcA3hyN a5knVamPQCW+QlDUpKGtYEJYHgr20mPB3DGh9bKgkmdfTn5aoFXO1t9muhfBbm//9tTh 4GT0d9kyeWnsTmq1Byw3otiZzWFjgj5enJTVQzL7BkRo2GoY6rmA/UxcuhBNDZmdGFrQ gRDW4jSmGM2rprdDFtsX4cP5WwcsasDPAHWEOubOCqSu+z5cXFEtqN4C3Xcq4I6dJX9g X4JMqhRK4EnvwMKuw/Bb9O4jVlyfz1y6OoBpis9yWayvMzkdHrUYAuvVrSDEZKtQ+i7u q61w== X-Gm-Message-State: ACgBeo1gxCiM+yIYdJLdY/qE05P0CYMTIx7hTfxFMEVAUyg4exackby1 4nrI3Jt+dvd2q+MZi82rUjckPA== X-Google-Smtp-Source: AA6agR7ozj10qGyIKy7vuKYj9CrPlJxOHy0JICYfBusGjziL+90ky5WE2rQXpJgVU9haTJrXINNg+A== X-Received: by 2002:a05:6870:961f:b0:10b:ad08:8909 with SMTP id d31-20020a056870961f00b0010bad088909mr3025373oaq.269.1662007156811; Wed, 31 Aug 2022 21:39:16 -0700 (PDT) Received: from rfd.leadboat.com ([2600:1702:a20:5750::2e]) by smtp.gmail.com with ESMTPSA id q16-20020a05683033d000b00638dd127f54sm9512439ott.1.2022.08.31.21.39.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 31 Aug 2022 21:39:15 -0700 (PDT) Date: Wed, 31 Aug 2022 21:39:13 -0700 From: Noah Misch To: jitesh tiwari , John Naylor Cc: pgsql-bugs@postgresql.org Subject: Re: fetching bytea (blob) data of 850 MB from psql client failed Message-ID: <20220901043913.GA1001056@rfd.leadboat.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Mon, Aug 22, 2022 at 05:52:29PM +0700, John Naylor wrote: > If the data is binary and not printable ASCII, then there is no easy > workaround. I would say the chief solution is to request binary transmission in your application. This cuts your network traffic in half for retrieving a bytea, so it's a good change even if the server improves someday. If you're using libpq, search for show_binary_results in https://www.postgresql.org/docs/devel/libpq-example.html to see an example of doing this. I don't know of an equivalent option in ODBC; if there is none, one could modify the ODBC driver to use the same protocol feature that libpq uses.