Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1UcXaz-0004Tp-Su for pgsql-hackers@arkaria.postgresql.org; Wed, 15 May 2013 09:02:02 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.72) (envelope-from ) id 1UcXaz-0008N0-C4 for pgsql-hackers@arkaria.postgresql.org; Wed, 15 May 2013 09:02:01 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1UcXay-0008MU-4Q for pgsql-hackers@postgresql.org; Wed, 15 May 2013 09:02:00 +0000 Received: from mail-ea0-x22e.google.com ([2a00:1450:4013:c01::22e]) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1UcXaq-0000eI-OL for pgsql-hackers@postgresql.org; Wed, 15 May 2013 09:01:59 +0000 Received: by mail-ea0-f174.google.com with SMTP id z7so866999eaf.5 for ; Wed, 15 May 2013 02:01:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:sender:date:from:reply-to:organization:x-priority :message-id:to:cc:subject:in-reply-to:references:mime-version :content-type:content-transfer-encoding:x-gm-message-state; bh=OHa+F68r8LkbqRr17/Hqm3AOjuDNaro3G5ViWj0pf9g=; b=fdFZ2DhxSV15FbFbK5VhpOMxZ+DRew06ZTIquZL+SbxBT/9O6JWfvOyyzol7eh72AT ie04Q9XexELNDjlsvMbGkCRVFLEozoBRkHxD30UbUlDM1r8Vm188UiDXixcK6wXFBxA0 C2I792wkj+wM0u3c6Ny32uEveW1n7M0nut+LC7dMrR92mK46YDeEZDmNV9gqIMguVxyY DrBDC6Y2qJRTDEOtmNt/j85LfyjUOoPydDq11d/xlWF0odADp/UxlTcF++y2WnYTDy5B avE7Ve84CRsTfgN0UyM3i+f2//5naiX8H1gX4Yv0iqbgX206JH38+wCpChLlouXX0XAx y//g== X-Received: by 10.15.108.6 with SMTP id cc6mr40657680eeb.28.1368608510308; Wed, 15 May 2013 02:01:50 -0700 (PDT) Received: from [192.168.1.102] (99-255-133-95.pool.ukrtel.net. [95.133.255.99]) by mx.google.com with ESMTPSA id i2sm2559992eeg.2.2013.05.15.02.01.48 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 15 May 2013 02:01:49 -0700 (PDT) Date: Wed, 15 May 2013 12:01:51 +0300 From: Pavel Golub Reply-To: Pavel Golub Organization: Microolap X-Priority: 3 (Normal) Message-ID: <1886757050.20130515120151@gf.microolap.com> To: Heikki Linnakangas CC: Josh Berkus , pgsql-hackers@postgresql.org, Pavel Stehule Subject: Re: Slicing TOAST In-Reply-To: <519286BB.6080901@vmware.com> References: <51928433.1040700@agliodbs.com> <519286BB.6080901@vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Gm-Message-State: ALoCoQlrv/cxwXO6MD/iuzGBrZCmE5KJhSfKGRLgKvyfTNnPyy0Ye7bngDFkfdVz+fJU/KMl474a X-Pg-Spam-Score: -1.2 (-) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-hackers Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Hello, Heikki. You wrote: HL> On 14.05.2013 21:36, Josh Berkus wrote: >> >>> I'm proposing this now as a possible GSoC project; I don't propose to >>> actively work on it myself. >> >> The deadline for submitting GSOC projects (by students) was a week ago. >> So is this a project suggestion for next year ...? HL> I've been thinking, we should already start collecting ideas for next HL> year, and collect them throughout the year. I know I come up with some HL> ideas every now and then, but when it's time for another GSoC, I can't HL> remember any of them. HL> I just created a GSoC2014 ideas pages on the wiki, for collecting these: HL> https://wiki.postgresql.org/wiki/GSoC_2014. Let's keep the ideas coming, HL> throughout the year. Good idea! It reminds about feature proposed by Pavel Stehule while ago here: http://www.postgresql.org/message-id/BANLkTini+ChGKfnyjkF1rsHSQ2kMktSDjg@mail.gmail.com It's about streaming functionality for BYTEA type. But I think streaming must be added to BYTEA, TEXT and VARCHAR without length specifier too. As Pavel stated: "A very large bytea are limited by query size - processing long query needs too RAM". This is the holy true, which came up suddenly in the project of one of my client. Becuase he used bytea for images storing and text format in PQexec, which as you know doubles-triples size of the data. Some more details from Pavel: There is a few disadvantages LO against bytea, so there are requests for "smarter" API for bytea. Significant problem is different implementation of LO for people who have to port application to PostgreSQL from Oracle, DB2. There are some JDBC issues too. For me - main disadvantage of LO in one space for all. Bytea removes this disadvantage, but it is slower for lengths > 20 MB. It could be really very practical have a possibility insert some large fields in second NON SQL stream. Same situation is when large bytea is read. I'm not sure if the whole project is simple enough for GSOC, but I suppose it may be splitted somehow. PS Should we start separate thread for proposals, because I've spent an hour since I found wiki for GSOC14 mention. HL> - Heikki -- With best wishes, Pavel mailto:pavel@gf.microolap.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers