Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hDt21-0004Qq-M4 for pgsql-hackers@arkaria.postgresql.org; Tue, 09 Apr 2019 15:51:29 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1hDt20-0006Sm-Ae for pgsql-hackers@arkaria.postgresql.org; Tue, 09 Apr 2019 15:51:28 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hDt1z-0006Ek-Qs for pgsql-hackers@lists.postgresql.org; Tue, 09 Apr 2019 15:51:28 +0000 Received: from mail-qt1-x841.google.com ([2607:f8b0:4864:20::841]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hDt1w-00078S-PX for pgsql-hackers@postgresql.org; Tue, 09 Apr 2019 15:51:26 +0000 Received: by mail-qt1-x841.google.com with SMTP id w30so20347784qta.8 for ; Tue, 09 Apr 2019 08:51:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=5TlnnNyC18B+lcupNE8rxjHXebzAnDqZg0pyZFMZMi4=; b=MaeM+YV682phu/7uyIBwsMTLXcs6RxrLLypUt8oIqjNXrPx+8QjAqCYJ4x6oeIQHaA VPJRNQoFADDXSK29A49i9JTre9nN2w4et8eXSs1XJiZZ5A8mzD3BgHWWUb/G/dFgOEBy VqJohgiiwXGoBfP4SIKB/Rd8EUqtHLrkYsDmrKgKZf/Jjtxh4Pj6KToNfCISvPstd2tN PxLleqkmrNyqT97tk3dM7SqTRjNoA51R8sxv9ZX3/5Rebh70wjaHeicy93Y1U91C0xs0 9Q1W0dpsBF3svzK/gJdHKtvcZzKUJue8+4eDf6cIUPl0SfbsX+WEBj1jK6s/YoQF9JVr BbMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=5TlnnNyC18B+lcupNE8rxjHXebzAnDqZg0pyZFMZMi4=; b=pIoy+jmeUYt+flaaGj7VQqIbh3jNqyWGCZLs+fpa0WILYd2Zs/2Br7/HQhm9YVGyUb MkdHqhXPGZR2TIzwHKh7o/AEA54L9aTYBZIwmI8gyVUDiQrTY2Swp3bCEqHeO53dYux0 EhuR46z8bPaS1Oh/m4Aeq+HS57pqsy2a805Vg5cKoXygHAcIMv+K0ned2JVBMrO8WX0s O666jgGfJF6Dff4/HZnL+pxx1VUAxexBe/tXtpqSLcEtA6v6J6pf4kypnVvE8UcVbBFs dIv8go7ucp8xBHPUIRax1Ixp4V41Kcc9ottvrRZO+HeAIvm7Bmzf+nwOw/fCzCozcJVZ JPnA== X-Gm-Message-State: APjAAAWDuspSxaFVqQQPw2xVMpewqOj7kXYdK0byo8ESBDo1LVXtWYs7 ksej7KML+nH4DtzkejaA9F2h3Q== X-Google-Smtp-Source: APXvYqywROAL0fmQMLTNV/qGbi8dmVk8dW7T0OQxFf9dbjGBWwbeqcyzHSVwLLZxP9jeOWIfCtGX9A== X-Received: by 2002:ac8:441a:: with SMTP id j26mr30063176qtn.117.1554825083830; Tue, 09 Apr 2019 08:51:23 -0700 (PDT) Received: from nimloth.alvh.no-ip.org ([190.121.29.3]) by smtp.gmail.com with ESMTPSA id s189sm19389331qkh.19.2019.04.09.08.51.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Apr 2019 08:51:23 -0700 (PDT) Received: by nimloth.alvh.no-ip.org (Postfix, from userid 1000) id 9FACB121023; Tue, 9 Apr 2019 11:51:21 -0400 (-04) Date: Tue, 9 Apr 2019 11:51:21 -0400 From: Alvaro Herrera To: Konstantin Knizhnik Cc: Ashwin Agrawal , PostgreSQL mailing lists Subject: Re: Zedstore - compressed in-core columnar storage Message-ID: <20190409155121.GA4320@alvherre.pgsql> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 2019-Apr-09, Konstantin Knizhnik wrote: > On 09.04.2019 3:27, Ashwin Agrawal wrote: > > Heikki and I have been hacking recently for few weeks to implement > > in-core columnar storage for PostgreSQL. Here's the design and initial > > implementation of Zedstore, compressed in-core columnar storage (table > > access method). Attaching the patch and link to github branch [1] to > > follow along. > > Thank you for publishing this patch. IMHO Postgres is really missing normal > support of columnar store Yep. > and table access method API is the best way of integrating it. This is not surprising, considering that columnar store is precisely the reason for starting the work on table AMs. We should certainly look into integrating some sort of columnar storage in mainline. Not sure which of zedstore or VOPS is the best candidate, or maybe we'll have some other proposal. My feeling is that having more than one is not useful; if there are optimizations to one that can be borrowed from the other, let's do that instead of duplicating effort. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services