pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru>
Cc: pgsql-hackers@postgresql.org, Ildar Musin <i.musin@postgrespro.ru>
Subject: Re: [HACKERS] Custom compression methods
Date: Mon, 27 Nov 2017 18:20:12 +0100
Message-ID: <0158242e-af45-c0f0-4fbb-4d54ca219cfd@2ndquadrant.com> (raw)
In-Reply-To: <20171127185218.5cddad41@wp.localdomain>
References: <20170907194236.4cefce96@wp.localdomain>
	<20170912175505.4afa11fd@wp.localdomain>
	<20171102152836.60c041e4@wp.localdomain>
	<20171114162356.52e3d388@wp.localdomain>
	<62e46a47-08a8-6e06-5e4f-6f52e0a13202@2ndquadrant.com>
	<20171121174717.69ecd8f4@wp.localdomain>
	<bfb3d34e-ba74-02cb-ca2a-43344866d9fd@2ndquadrant.com>
	<20171123123849.5a686d7b@wp.localdomain>
	<57daf28d-ed76-c364-a9ca-65d0ff71a36f@2ndquadrant.com>
	<20171124123800.034c9208@wp.localdomain>
	<e3378c06-d805-a3f5-6ea7-af2a2417361e@2ndquadrant.com>
	<b969fa5a-bc2a-c220-c992-e6e0de34df14@2ndquadrant.com>
	<20171127185218.5cddad41@wp.localdomain>

Hi,

On 11/27/2017 04:52 PM, Ildus Kurbangaliev wrote:
> ...
>
> Hi. This looks like a serious bug, but I couldn't reproduce it yet.
> Did you upgrade some old database or this bug happened after 
> insertion of all data to new database? I tried using your 'archie' 
> tool to download mailing lists and insert them to database, but
> couldn't catch any errors.
> 

I can trigger it pretty reliably with these steps:

    git checkout f65d21b258085bdc8ef2cc282ab1ff12da9c595c
    patch -p1 < ~/custom_compression_methods_v6.patch
    ./configure --enable-debug --enable-cassert \
     CFLAGS="-fno-omit-frame-pointer -O0 -DRANDOMIZE_ALLOCATED_MEMORY" \
     --prefix=/home/postgres/pg-compress
    make -s clean && make -s -j4 install
    cd contrib/
    make -s clean && make -s -j4 install

    export PATH=/home/postgres/pg-compress/bin:$PATH
    pg_ctl -D /mnt/raid/pg-compress init
    pg_ctl -D /mnt/raid/pg-compress -l compress.log start
    createdb archie
    cd ~/archie/sql/
    psql archie < create.sql

    ~/archie/bin/load.py --workers 4 --db archie */* > load.log 2>&1


I guess the trick might be -DRANDOMIZE_ALLOCATED_MEMORY (I first tried
without it, and it seemed working fine). If that's the case, I bet there
is a palloc that should have been palloc0, or something like that.

If you still can't reproduce that, I may give you access to this machine
so that you can debug it there.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services




view thread (430+ messages)  latest in thread

Message-ID: <0158242e-af45-c0f0-4fbb-4d54ca219cfd@2ndquadrant.com>
Permalink:  ../0158242e-af45-c0f0-4fbb-4d54ca219cfd@2ndquadrant.com/
Also on:    postgresql.org/message-id/0158242e-af45-c0f0-4fbb-4d54ca219cfd@2ndquadrant.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: tomas.vondra@2ndquadrant.com, i.kurbangaliev@postgrespro.ru, i.musin@postgrespro.ru
  Subject: Re: [HACKERS] Custom compression methods
  In-Reply-To: <0158242e-af45-c0f0-4fbb-4d54ca219cfd@2ndquadrant.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox