pg.ddx.io pljava-dev@postgresql.org mailing list archive
help / color / mirror / Atom feedFrom: Dave Cramer <pg@fastcrypt.com>
To: Thomas Hallgren <thomas@tada.se>
Cc: Simon Riggs <simon@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Martijn van Oosterhout <kleptog@svana.org>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Cc: PL/Java Development <Pljava-dev@gborg.postgresql.org>
Subject: Re: Shared memory
Date: Tue, 28 Mar 2006 13:25:36 -0500
Message-ID: <2606EB97-424C-4C15-A3F6-AED510E5B37F@fastcrypt.com> (raw)
In-Reply-To: <44295AB0.5060203@tada.se>
References: <4423CF32.3060406@tada.se>
<20060324125851.GC8718@svana.org>
<4427A8F1.3020009@tada.se>
<14559.1143473507@sss.pgh.pa.us>
<4428125D.7000707@tada.se>
<1143541439.3839.323.camel@localhost.localdomain>
<44295AB0.5060203@tada.se>
On 28-Mar-06, at 10:48 AM, Thomas Hallgren wrote:
> Hi Simon,
> Thanks for your input. All good points. I actually did some work
> using Java stored procedures on DB2 a while back but I had managed
> to forget (or repress :-) ) all about the FENCED/NOT FENCED stuff.
> The current discussion definitely puts it in a different
> perspective. I think PL/Java has a pretty good 'NOT FENCED'
> implementation, as does many other PL's, but no PL has yet come up
> with a FENCED solution.
What exactly is a FENCED solution ? If it is simply a remote
connection to a single JVM then pl-j already does that.
>
> This FENCED/NOT FENCED terminology would be a good way to
> differentiate between the two approaches. Any chance of that syntax
> making it into the PostgreSQL grammar, should the need arise?
>
> Some more comments inline:
>
> Simon Riggs wrote:
>> Just some thoughts from afar: DB2 supports in-process and out-of-
>> process
>> external function calls (UDFs) that it refers to as UNFENCED and
>> FENCED
>> procedures. For Java only, IBM have moved to supporting *only* FENCED
>> procedures for Java functions, i.e. having a single JVM for all
>> connections.
> >
> Are you sure about this? As I recall it a FENCED stored procedure
> executed in a remote JVM of it's own. A parameter could be used
> that either caused a new JVM to be instantiated for each stored
> procedure call or to be kept for the duration of the session. The
> former would yield really horrible performance but keep memory
> utilization at a minimum. The latter would get a more acceptable
> performance but waste more memory (in par with PL/Java today).
>
>
>> Each connection's Java function runs as a thread on a
>> single dedicated JVM-only process.
> If that was true, then different threads could share dirty session
> data. I wanted to do that using DB2 but found it impossible. That
> was a while back though.
>
>> That approach definitely does increase the invocation time, but it
>> significantly reduces the resources associated with the JVM, as
>> well as
>> allowing memory management to be more controllable (bliss...). So the
>> overall picture could be more CPU and memory resources for each
>> connection in the connection pool.
> My very crude measurements indicate that the overhead of using a
> separate JVM is between 6-15MB of real memory per connection.
> Today, you get about 10MB/$ and servers configured with 4GB RAM or
> more are not uncommon.
>
> I'm not saying that the overhead doesn't matter. Of course it does.
> But the time when you needed to be extremely conservative with
> memory usage has passed. It might be far less expensive to buy some
> extra memory then to invest in SMP architectures to minimize IPC
> overhead.
>
> My point is, even fairly large app-servers (using connection pools
> with up to 200 simultaneous connections) can run using relatively
> inexpensive boxes such as an AMD64 based server with 4GB RAM and
> show very good throughput with the current implementation.
>
>
>> If you have a few small Java functions centralisation would not be
>> good,
>> but if you have a whole application architecture with many
>> connections
>> executing reasonable chunks of code then this can be a win.
> One thing to remembered is that a 'chunk of code' that executes in
> a remote JVM and uses JDBC will be hit by the IPC overhead on each
> interaction over the JDBC connection. I.e. the overhead is not just
> limited to the actual call of the UDF, it's also imposed on all
> database accesses that the UDF makes in turn.
>
>
>> In that environment we used Java for major database functions,
>> with SQL
>> functions for small extensions.
> My guess is that those major database functions did a fair amount
> of JDBC. Am I right?
>
>
>> Also the Java invocation time we should be celebrating is that by
>> having
>> Java in the database the Java<->DB time is much less than it would
>> be if
>> we had a Java stack sitting on another server.
>
> I think the cases when you have a Tomcat or JBoss sitting on the
> same physical server as the actual database are very common. One
> major reason being that you don't want network overhead between the
> middle tier and the backend. Moving logic into the database instead
> of keeping it in the middle tier is often done to get rid of the
> last hurdle, the overhead of IPC.
>
>
> Regards,
> Thomas Hallgren
>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
>
view thread (22+ messages) latest in thread
Message-ID: <2606EB97-424C-4C15-A3F6-AED510E5B37F@fastcrypt.com>
Permalink: ../2606EB97-424C-4C15-A3F6-AED510E5B37F@fastcrypt.com/
Also on: postgresql.org/message-id/2606EB97-424C-4C15-A3F6-AED510E5B37F@fastcrypt.com
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pljava-dev@postgresql.org
Cc: pg@fastcrypt.com, thomas@tada.se, simon@2ndquadrant.com, tgl@sss.pgh.pa.us, kleptog@svana.org, pgsql-hackers@postgresql.org, Pljava-dev@gborg.postgresql.org
Subject: Re: Shared memory
In-Reply-To: <2606EB97-424C-4C15-A3F6-AED510E5B37F@fastcrypt.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox