X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id D941A5656B for ; Wed, 16 Mar 2005 11:40:58 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 42326-01 for ; Wed, 16 Mar 2005 11:40:57 +0000 (GMT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by svr1.postgresql.org (Postfix) with ESMTP id 71232564EF for ; Wed, 16 Mar 2005 11:40:57 +0000 (GMT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by news.hub.org (8.13.1/8.13.1) with ESMTP id j2GBeudA042341 for ; Wed, 16 Mar 2005 11:40:56 GMT (envelope-from news@news.hub.org) Received: (from news@localhost) by news.hub.org (8.13.1/8.13.1/Submit) id j2GBFlnw034863 for pgsql-performance@postgresql.org; Wed, 16 Mar 2005 11:15:47 GMT (envelope-from news) From: "Rosny" X-Newsgroups: pgsql.performance Subject: Which one is faster: one way reading ="single pass reading" Date: Wed, 16 Mar 2005 03:15:48 -0800 Organization: Hub.Org Networking Services Lines: 35 Message-ID: Reply-To: "Rosny" X-Complaints-To: usenet@news.hub.org X-Priority: 3 X-MSMail-Priority: Normal X-Newsreader: Microsoft Outlook Express 6.00.3790.181 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.181 To: pgsql-performance@postgresql.org X-Virus-Scanned: by amavisd-new at hub.org X-Archive-Number: 200503/404 X-Sequence-Number: 11109 Hi, Which one is faster: one way reading ="single pass reading" Assumption : a. Need to have 3 millions records b. Need to have call 10 or 20 records repeatly (so for database it will be 10 times connection, each connection with one record. or can be fancy 1 connection call return 10 sets of records) 1. Reading from Flat file Assume already give file name and just need to read the file (since it is flat file, each record represent a filename, with multiple directory category) 2. Reading from XML file Assume schema already given just need to read the file (since it is xml file, each record represent an xml filename, with multiple directory category) 3. Reading from Postgresql Assume primary key has been done with indexing just need to search the number and grap the text content (assume 3 millions of records, search the number, read the content file) trying to recreate WebDBReader (from nutch) using C# http://nutch.sourceforge.net/docs/api/net/nutch/db/WebDBReader.html Thank you in advances, Rosny