Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kplTs-0007uL-98 for pgsql-hackers@arkaria.postgresql.org; Thu, 17 Dec 2020 05:05:36 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kplTq-00042g-0G for pgsql-hackers@arkaria.postgresql.org; Thu, 17 Dec 2020 05:05:34 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kplTp-00042U-LS for pgsql-hackers@lists.postgresql.org; Thu, 17 Dec 2020 05:05:33 +0000 Received: from mail-ot1-x333.google.com ([2607:f8b0:4864:20::333]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1kplTh-00073C-IG for pgsql-hackers@postgresql.org; Thu, 17 Dec 2020 05:05:31 +0000 Received: by mail-ot1-x333.google.com with SMTP id h18so26099712otq.12 for ; Wed, 16 Dec 2020 21:05:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telsasoft-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=JUChtu0XBWiNctj2WwnRpKodGF3rcVIVkGmMcn/cFtc=; b=nxuyrgZgfAbiU6eibNnDjuNOftNsUGFaprghXYeprT/t3ZHKdTpBwgE+25/3BMVqOY Yc0RUtJy9SvoEEL+kkh/d6qHVwLOJ0FmMQzAgLUuJdLZtLLYGNVQQRayaoqpKlJLTyCP 2ITfcImdPixio1RwLjVtoISVug0R8lTCmUwYQCrmlbwT+GkOGxYuU5Hhc875nI7hjlBx MsB4nW2bhHYQRhL86DzeNx0xJZtNgoaLtsmS+/TAjsVd0QN8QtDNO9PhnWWrJkpiP/C+ ob9JFMvHVkALjHsmnijNG0DhMlM65XTnV/RP74CfyhPfeuGGaMrM0rQSqt7MPST1z68u ZCpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=JUChtu0XBWiNctj2WwnRpKodGF3rcVIVkGmMcn/cFtc=; b=lMXR1aWq/WWVOOyZn5valvX8Z2RPAUGvrdGC/33oehCR5HZTD7OcmK8rP8ge/vWUT/ DL9j0twpLRSUIHVK/8XphFjKbSIOEy+XBVgTcDM0T2/L8fOMdX6WgHSDjXgBwev0Sy/n ujnhHYSRunUTRbqs8gP0xH/T5wx/DnMjjF4DLEAbJdq66SFe/JcAiXH8vXAdhNSKWjVp rFs9lLME6rzUoIICOsnGGzfRNxBfktlUDehY00vSt9eK+S9dzNKZdwY/0IkXGtu4MED4 OUt0OMZ0f+nw+73SCT6eKbz+j1o2KEdf1mu6MsqJqDjl5p5PIlJVOjDW0H2zsVrm52Lt uZbQ== X-Gm-Message-State: AOAM531Y3MirRQ+w93bFoyjgpz1DlTDNBOOIGdJixVzdICHvAWNdv+Pk RUfOx4WD4Vv/rsMWTcpYrvITag== X-Google-Smtp-Source: ABdhPJzR83GHekIx/ZWKfTnFMBM3GqFDAPJyLfqx328AZBbnHlJ+4VyLwr7mVb4riKhccOzVgTFHgg== X-Received: by 2002:a9d:6f82:: with SMTP id h2mr6221159otq.276.1608181524542; Wed, 16 Dec 2020 21:05:24 -0800 (PST) Received: from pryzbyj.telsasoft (charmander.telsasoft.com. [50.244.222.1]) by smtp.gmail.com with ESMTPSA id j2sm987360otq.78.2020.12.16.21.05.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Dec 2020 21:05:23 -0800 (PST) Received: by pryzbyj.telsasoft (Postfix, from userid 1000) id 9BD708008E3; Wed, 16 Dec 2020 23:05:22 -0600 (CST) Date: Wed, 16 Dec 2020 23:05:22 -0600 From: Justin Pryzby To: Bharath Rupireddy Cc: pgsql-hackers@postgresql.org, Andres Freund , Luc Vlaming , Paul Guo Subject: Re: New Table Access Methods for Multi and Single Inserts Message-ID: <20201217050522.GU30237@telsasoft.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk Typos: + * 1) Specify is_multi as true, then multi insert state is allcoated. => allocated + * dropped, short-lived memory context is delted and mistate is freed up. => deleted + * 2) Currently, GetTupleSize() handles the existing heap, buffer, minmal and => minimal + /* Mulit insert state if requested, otherwise NULL. */ => multi + * Buffer the input slots and insert the tuples from the buffered slots at a => *one* at a time ? + * Compute the size of the tuple only if mi_max_size i.e. the total tuple size => I guess you mean max_size This variable could use a better name: +CopyMulitInsertFlushBuffers(List **mirri, .. mirri is fine for a local variable like an element of a struture/array, or a loop variable, but not for a function parameter which is an "List" of arbitrary pointers. I think this comment needs to be updated (again) for the removal of the Info structure. - * CopyMultiInsertBuffer items stored in CopyMultiInsertInfo's + * multi insert buffer items stored in CopyMultiInsertInfo's I think the COPY patch should be 0002 (or maybe merged into 0001). There's some superfluous whitespace (and other) changes there which make the patch unnecessarily long. You made the v2 insert interface a requirement for all table AMs. Should it be optional, and fall back to simple inserts if not implemented ? For CTAS, I think we need to consider Paul's idea here. https://www.postgresql.org/message-id/26C14A63-CCE5-4B46-975A-57C1784B3690%40vmware.com Conceivably, tableam should support something like that for arbitrary AMs ("insert into a new table for which we have exclusive lock"). I think that AM method should also be optional. It should be possible to implement a minimal AM without implementing every available optimization, which may not apply to all AMs, anyway. -- Justin