Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1umLFS-00DzAp-2l for pgsql-hackers@arkaria.postgresql.org; Wed, 13 Aug 2025 23:51:14 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1umLFP-001GDW-Bd for pgsql-hackers@arkaria.postgresql.org; Wed, 13 Aug 2025 23:51:11 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1umLFO-001GDN-LL for pgsql-hackers@lists.postgresql.org; Wed, 13 Aug 2025 23:51:11 +0000 Received: from mail-qv1-xf2a.google.com ([2607:f8b0:4864:20::f2a]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1umLFI-000RjK-29 for pgsql-hackers@lists.postgresql.org; Wed, 13 Aug 2025 23:51:09 +0000 Received: by mail-qv1-xf2a.google.com with SMTP id 6a1803df08f44-70a9f5b60b9so4438146d6.3 for ; Wed, 13 Aug 2025 16:51:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bowt-ie.20230601.gappssmtp.com; s=20230601; t=1755129064; x=1755733864; darn=lists.postgresql.org; h=in-reply-to:references:to:from:subject:cc:message-id:date :content-transfer-encoding:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=slort9PIrOUczn9KAZZ1u7uLVHt6jrPKit+Tix07snk=; b=LGO/s0O415UfD1WdlN1CHi4xwSg7o0qHnjt3Fqb2zML3MYdTVWkdHFmPtw+CvXI8yP 5wZy/Irc44ZNDWsViHri9suTRO+T4tAe8rjujsjnCaOQBaHQHBPbQC6pO7pLbbgfc9X5 duSfzFxu9yPj9Prbmi11HkOnvwx1N/+AAABKKj3sJFIZ9ZX6hFZDy4V2fu5Qg2mHIT3Z VGSXOSw2VvK2N4LGl0ixrsvQgmuyGpT2chzFduKu0kvwhu5PoSru5/ZGoEL2G1wvU75L EKkUX2/fDlPB85jMFYl/NGIXcFB4kLkAgWKJd8Vmf8lm/cNwzU1aKIkm1NYL9ExQX56D 2D0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755129064; x=1755733864; h=in-reply-to:references:to:from:subject:cc:message-id:date :content-transfer-encoding:mime-version:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=slort9PIrOUczn9KAZZ1u7uLVHt6jrPKit+Tix07snk=; b=VbVPA0abrODLHiZXCo9Q6/omWWK7LHPM0o8NKpgY8OrV+KiKZOqLtR8U/c22506rt5 BhE3DB4NhOlvUzMSSdr1QrSd8LbKixVNvzCyFBdp4ZMioepkqn8umoz8qglCwdf0Hh9q 9plQSpN0twtnVRjKafU7RyOCjUKXLG7luhk4HSgyc/TAaHhxIJv9rxk1Kpf0GfBQ0gpM jX8NmFwqqszEWPqdlDW8pMzj0bsnhceFfRpuB9WCpuJOfxITrlWnQhWgZo+WvLu7Tk/3 XgtdF51SjE86V8dzKA7F9oTwFbWeQsiQZqxdOlCWW4vffUbMGrcTgS28Dtm/JRs0FLNY AIRA== X-Forwarded-Encrypted: i=1; AJvYcCXgRt2kUfTWib4eynj13XHX/9Sq4ZfQ2gzTnNf+iGo5FigbPLWhXU2j1YiRqifmkgTTGnpifgcPq2ZA9mZw@lists.postgresql.org X-Gm-Message-State: AOJu0Yw3Xi7PWfhP+pk7faYLE7sU3+MnnXRBlgMyGRSmRL6CX7GwezyD 1AIK0ViLPdFVfwFpf5KCKBPjImM5+3/4A1HtkxlGcGpJPqjtba868jnIDOkKPtfbirQ= X-Gm-Gg: ASbGnctmxiy0csUBtvaOLht372WSp9kO9ngm5Vb6oqPACMKNfsChqBfI8tDKkhIN/4C hjD3zxLkGt1LUCaNQZSSWtmmqElL6+s5uy/c4EZ2aXBWd0pY9dmQhDmTzkwgjDiklfNdIZrW9v9 3CE4IsTJZD1bE4IGDiNhRcV3cWAgqNQo7g3sicH//GBJIo7FcvWv+OhxjTD3929U7z34fbnXhXh fPpHt8e8p92+yhSDu24/gzY2bCpgOycxcb/wASljCuP/2R4wYRfA6KSio2AQ/9umumI4pgaarfY e90E9o5MIxgYt4gHcb3tm4i+DsFFDF24fX5Y1b6aQBEBS8Lx9z7uzcXzoA8INJeuRhePz5YtgPi IHlllLFKYWBVSfZSEkVmFkpnVWzKZJIUep8pmBGQr3/AdTQ== X-Google-Smtp-Source: AGHT+IF5bsJE3wbJCwXmerhyVFjecsFSB3kZZuRWeOpUOFori0WzppnNUCXHfRGUbPlmodego4uByw== X-Received: by 2002:a05:6214:ca8:b0:709:76b4:5933 with SMTP id 6a1803df08f44-70b98566813mr6831506d6.52.1755129063677; Wed, 13 Aug 2025 16:51:03 -0700 (PDT) Received: from localhost (syn-024-090-110-103.res.spectrum.com. [24.90.110.103]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-70af5b4eca3sm5029786d6.57.2025.08.13.16.51.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 13 Aug 2025 16:51:02 -0700 (PDT) Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Wed, 13 Aug 2025 19:50:59 -0400 Message-Id: Cc: "Thomas Munro" , "Nazir Bilal Yavuz" , "Robert Haas" , "Melanie Plageman" , "PostgreSQL Hackers" , "Georgios" , "Konstantin Knizhnik" , "Dilip Kumar" Subject: Re: index prefetching From: "Peter Geoghegan" To: "Tomas Vondra" , "Andres Freund" X-Mailer: aerc 0.20.0 References: <51b5f71b-5f19-4453-91ff-2b9f2a840c58@vondra.me> <6cb6109d-71d6-490c-8056-d8885081b008@vondra.me> In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Wed Aug 13, 2025 at 5:19 PM EDT, Tomas Vondra wrote: > I did investigate this, and I don't think there's anything broken in > read_stream. It happens because ReadStream has a concept of "ungetting" > a block, which can happen after hitting some I/O limits. > > In that case we "remember" the last block (in read_stream_look_ahead > calls read_stream_unget_block), and we return it again. It may seem as > if read_stream_get_block() produced the same block twice, but it's > really just the block from the last round. I instrumented this for myself, and I agree: backwards and forwards scan ca= ses are being fed the same block numbers, as expected (it's just that the order= is precisely backwards, as expected). The only real difference is that the for= wards scan case seems to be passed InvalidBlockNumber quite a bit more often. You= were right: I was confused about the read_stream_unget_block thing. However, the magnitude of the difference that I see between the forwards an= d backwards scan cases just doesn't pass the smell test -- I stand by that pa= rt. I was able to confirm this intuition by performing a simple experiment. I asked myself a fairly obvious question: if the backwards scan in question takes about 2.5x as long, just because each group of TIDs for each index va= lue appears in descending order, then what happens if the order is made random? Where does that leave the forwards scan case, and where does it leave the backwards scan case? I first made the order of the table random, except among groups of index tu= ples that have exactly the same value. Those will still point to the same 1 or 2= heap blocks in virtually all cases, so we have "heap clustering without any heap correlation" in the newly rewritten table. To set things up this way, I fi= rst made another index, and then clustered the table using that new index: pg@regression:5432 [2476413]=3D# create index on t (hashint8(a)); CREATE INDEX pg@regression:5432 [2476413]=3D# cluster t using t_hashint8_idx ; CLUSTER Next, I reran the queries in the obvious way (same procedure as yesterday, though with a very different result): pg@regression:5432 [2476413]=3D# select pg_buffercache_evict_relation('t');= select pg_prewarm('idx'); ***SNIP*** pg@regression:5432 [2476413]=3D# EXPLAIN (ANALYZE ,costs off, timing off) S= ELECT * FROM t WHERE a BETWEEN 16336 AND 49103 ORDER BY a; =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=90 =E2=94=82 QUERY PLAN =E2=94= =82 =E2=94=9C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=A4 =E2=94=82 Index Scan using idx on t (actual rows=3D1048576.00 loops=3D1) = =E2=94=82 =E2=94=82 Index Cond: ((a >=3D 16336) AND (a <=3D 49103)) = =E2=94=82 =E2=94=82 Index Searches: 1 =E2=94= =82 =E2=94=82 Buffers: shared hit=3D6082 read=3D77813 = =E2=94=82 =E2=94=82 I/O Timings: shared read=3D153.672 =E2= =94=82 =E2=94=82 Planning Time: 0.057 ms =E2=94= =82 =E2=94=82 Execution Time: 402.735 ms =E2=94= =82 =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=98 (7 rows) pg@regression:5432 [2476413]=3D# select pg_buffercache_evict_relation('t');= select pg_prewarm('idx'); ***SNIP*** pg@regression:5432 [2476413]=3D# EXPLAIN (ANALYZE ,costs off, timing off) S= ELECT * FROM t WHERE a BETWEEN 16336 AND 49103 ORDER BY a desc; =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=90 =E2=94=82 QUERY PLAN = =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=A4 =E2=94=82 Index Scan Backward using idx on t (actual rows=3D1048576.00 loop= s=3D1) =E2=94=82 =E2=94=82 Index Cond: ((a >=3D 16336) AND (a <=3D 49103)) = =E2=94=82 =E2=94=82 Index Searches: 1 = =E2=94=82 =E2=94=82 Buffers: shared hit=3D6082 read=3D77813 = =E2=94=82 =E2=94=82 I/O Timings: shared read=3D324.305 = =E2=94=82 =E2=94=82 Planning Time: 0.071 ms = =E2=94=82 =E2=94=82 Execution Time: 616.268 ms = =E2=94=82 =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=98 (7 rows) Apparently random I/O is twice as fast as sequential I/O in descending orde= r! In fact, this test case creates the appearance of random I/O being at least slightly faster than sequential I/O for pages read in _ascending_ order! Obviously something doesn't add up here. I'm no closer to explaining what = the underlying problem is than I was yesterday, but I find it _very_ hard to be= lieve that the inconsistency in performance has anything to do with SSD firmware/= OS implementation details. It just looks wonky to me. Also possibly worth noting: I'm pretty sure that "shared hit=3D6082" is wro= ng. Though now it's wrong in the same way with both variants. Just for context, I'll show what the TIDs for 3 randomly chosen adjacent-in-index values look like after CLUSTER runs (in case it was uncle= ar what I meant about "heap clustering without any heap correlation" earlier): pg@regression:5432 [2476413]=3D# SELECT ctid, a FROM t WHERE a BETWEEN 20_0= 00 AND 20_002 ORDER BY a; =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=AC=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=90 =E2=94=82 ctid =E2=94=82 a =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=BC=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=A4 =E2=94=82 (142534,3) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,4) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,5) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,6) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,7) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,8) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,9) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,10) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,11) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,12) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,13) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,14) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,15) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,16) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,17) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,18) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,19) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,20) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142534,21) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,1) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,2) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,3) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,4) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,5) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,6) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,7) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,8) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,9) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,10) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,11) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,12) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (142535,13) =E2=94=82 20,000 =E2=94=82 =E2=94=82 (216406,19) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216406,20) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216406,21) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,1) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,2) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,3) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,4) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,5) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,6) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,7) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,8) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,9) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,10) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,11) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,12) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,13) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,14) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,15) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,16) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,17) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,18) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,19) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,20) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216407,21) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216408,1) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216408,2) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216408,3) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216408,4) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216408,5) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216408,6) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216408,7) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (216408,8) =E2=94=82 20,001 =E2=94=82 =E2=94=82 (260993,12) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260993,13) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260993,14) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260993,15) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260993,16) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260993,17) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260993,18) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260993,19) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260993,20) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260993,21) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,1) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,2) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,3) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,4) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,5) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,6) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,7) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,8) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,9) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,10) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,11) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,12) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,13) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,14) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,15) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,16) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,17) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,18) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,19) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,20) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260994,21) =E2=94=82 20,002 =E2=94=82 =E2=94=82 (260995,1) =E2=94=82 20,002 =E2=94=82 =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=B4=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=98 (96 rows) --=20 Peter Geoghegan