Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kFzvB-0006E4-9Q for pgsql-bugs@arkaria.postgresql.org; Wed, 09 Sep 2020 13:13:57 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kFzv9-0001nk-9Y for pgsql-bugs@arkaria.postgresql.org; Wed, 09 Sep 2020 13:13:55 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kFzLr-0006jU-Hv for pgsql-bugs@lists.postgresql.org; Wed, 09 Sep 2020 12:37:27 +0000 Received: from mail-ej1-x644.google.com ([2a00:1450:4864:20::644]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1kFzLk-0006sj-EQ for pgsql-bugs@postgresql.org; Wed, 09 Sep 2020 12:37:26 +0000 Received: by mail-ej1-x644.google.com with SMTP id lo4so3286138ejb.8 for ; Wed, 09 Sep 2020 05:37:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language; bh=2nkiKm+hUyVF3mocWq4Y3c9XrVzDKTWCGJtcSEh4lis=; b=lD3thh9JXeMaNSAXEcPtPFHOPXwJXbOQyGaTygjUpmYGWORioW3CpMCjIU8Wy5MK4u kAjTc1eRJzKF89hfOFrocje52+645ftyTInNuPeRmozLXpDhEfwiyr/ZFXclWU0rWvJu OWczL4epYDr+97gS/f2sK0tkaFnV+0E8fTGviVtlF2rd1D3hrpCU+a8D4RW4GW4W/tqi IMRPQfjcM/qVIJqYGnfgFS225BHixJmZqHoRqCGdNwQwPJXmrNlVpo+NP1ifiqZQmXoF v+ThL+Il7nUsCB6JjO3a4hiQw2CY5wOwz4C9TWMiwWZcfx1GTHfrwqiuEWKUVWGhsCoS 2Hcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=2nkiKm+hUyVF3mocWq4Y3c9XrVzDKTWCGJtcSEh4lis=; b=JGMmSOK9GRxRjSgh/Cdx7QpouRSDL0nH/J+9+nkP0HG7SZjcZy77kelrfhBIjnI0EL fti062phZtab5t9GPqSZB3o5g41FIv3R4eqY88s8fgTYOkdIPi6B2g4FzZmsrlyk0pIL bXTWyyaBErMzBai8NBMKNmKA5kwXWXcd1Of5EuIWy41M3k/HhF+ndQklReneZi/tbpzY uKVcCmwMAHgfBg0/8MRCdwgQuIrkggme7sH262jTvRsKFREciFm0YoMHlrUTq52L1YP8 GjJKiOe7xk354d/PYSK03mofd4X1947tb/iEfXTZ5Hn4SDWIT737JcepDN9Sd3DWGvWE +TdA== X-Gm-Message-State: AOAM530TrSsMBfvsO6yrh5zCTh1vbyY69m1zrymS1w9ttO40kvBLHaW0 GxV0Ws8IQknElZTNJiCr/qO13Q== X-Google-Smtp-Source: ABdhPJxLxRI9Cls6LXFt+Bm5nnDDaq+vgbCmGvuNaIgDkaj85pdS5MURk5onSqVwn2xKOm0SA9/SpA== X-Received: by 2002:a17:906:1ed4:: with SMTP id m20mr3490339ejj.460.1599655039550; Wed, 09 Sep 2020 05:37:19 -0700 (PDT) Received: from [10.80.18.219] (pool1-42.cgn.as8758.net. [212.25.6.42]) by smtp.gmail.com with ESMTPSA id l26sm2139561ejr.78.2020.09.09.05.37.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Sep 2020 05:37:18 -0700 (PDT) Subject: Re: invalid alloc size error possible in shm_mq To: Peter Eisentraut , Markus Wanner , pgsql-bugs@postgresql.org, Robert Haas References: <3bb363e7-ac04-0ac4-9fe8-db1148755bfa@2ndquadrant.com> <3e428657-26ce-ed05-a7cf-7f2a546c2672@2ndquadrant.com> From: Markus Wanner Message-ID: Date: Wed, 9 Sep 2020 14:37:17 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <3e428657-26ce-ed05-a7cf-7f2a546c2672@2ndquadrant.com> Content-Type: multipart/mixed; boundary="------------92BF1B45804A44597EDB8D1C" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------92BF1B45804A44597EDB8D1C Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 8/25/20 12:00 PM, Peter Eisentraut wrote: > I wonder if the assertion is appropriate or whether it should be a full > error check. Good point. Originally, it used to be an error. With the patch (but w/o assertions enabled) it could result in a buffer overrun. Not good. I changed the patch to add an error (instead of just an assert) when asked to read a message larger than MaxAllocSize. So this patch essentially corrects handling of messages in size between MaxAllocSize/2 and MaxAllocSize. > Is anything on the sending side ensuring that the maximum > size is kept?  All the size variables are Size/size_t so could be much > larger than MaxAllocSize. In this v2 of the patch, I added a check that errors out on the sender side as well (for trying to send a message larger than MaxAllocSize). I'd still recommend to back-patch this. -- Markus Wanner Senior PostgreSQL Developer 2ndQuadrant - PostgreSQL Solutions for the Enterprise https://www.2ndQuadrant.com/ --------------92BF1B45804A44597EDB8D1C Content-Type: text/x-patch; charset=UTF-8; name="shm_mq_inv_allocation_fix_v2.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="shm_mq_inv_allocation_fix_v2.diff" diff --git a/src/backend/storage/ipc/shm_mq.c b/src/backend/storage/ipc/shm_mq.c index 6f40fb165d..262faecf13 100644 --- a/src/backend/storage/ipc/shm_mq.c +++ b/src/backend/storage/ipc/shm_mq.c @@ -24,6 +24,7 @@ #include "storage/procsignal.h" #include "storage/shm_mq.h" #include "storage/spin.h" +#include "utils/memutils.h" /* * This structure represents the actual queue, stored in shared memory. @@ -360,6 +361,13 @@ shm_mq_sendv(shm_mq_handle *mqh, shm_mq_iovec *iov, int iovcnt, bool nowait) for (i = 0; i < iovcnt; ++i) nbytes += iov[i].len; + /* Prevent writing messages overwhelming the receiver. */ + if (nbytes > MaxAllocSize) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("cannot send a message of size %zu via shared memory queue", + nbytes))); + /* Try to write, or finish writing, the length word into the buffer. */ while (!mqh->mqh_length_word_complete) { @@ -675,6 +683,17 @@ shm_mq_receive(shm_mq_handle *mqh, Size *nbytesp, void **datap, bool nowait) } nbytes = mqh->mqh_expected_bytes; + /* + * Should be disallowed on the sending side already, but better check + * and error out on the receiver side as well, rather than trying to + * read a prohibitively large message. + */ + if (nbytes > MaxAllocSize) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg("invalid message size %zu in shared memory queue", + nbytes))); + if (mqh->mqh_partial_bytes == 0) { /* @@ -703,8 +722,13 @@ shm_mq_receive(shm_mq_handle *mqh, Size *nbytesp, void **datap, bool nowait) { Size newbuflen = Max(mqh->mqh_buflen, MQH_INITIAL_BUFSIZE); + /* + * Double the buffer size until the payload fits, but limit to + * MaxAllocSize. + */ while (newbuflen < nbytes) newbuflen *= 2; + newbuflen = Min(newbuflen, MaxAllocSize); if (mqh->mqh_buffer != NULL) { --------------92BF1B45804A44597EDB8D1C--