Subject: Re: proposal: possibility to read dumped table's name from file
To: Pavel Stehule <pavel.stehule@gmail.com>,
 Justin Pryzby <pryzby@telsasoft.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Dean Rasheed <dean.a.rasheed@gmail.com>,
 Stephen Frost <sfrost@snowman.net>, Surafel Temesgen
 <surafel3000@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>,
 vignesh C <vignesh21@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@postgresql.org>,
 Daniel Gustafsson <daniel@yesql.se>
References: <20201110200904.GU16415@tamriel.snowman.net>
 <CAFj8pRAZLSk+6cXtWwjSnPB9S7aE6LrgeqUpHEZKMRd7Ve9DwA@mail.gmail.com>
 <CAFj8pRDTmY0A1RzrS9csuzsq7+PHqcAui09ZGO_48vOBOfZt1Q@mail.gmail.com>
 <20201117215321.GF24784@telsasoft.com>
 <CAFj8pRBuEOCTGR8VwhQi3_dNg6v=g5zBB4kmACRsnAFiWJKWVA@mail.gmail.com>
 <CAFj8pRBoZ1yY7Q_UBCiRFicKvtpxdzkuyus701A4wKe9z9d-9Q@mail.gmail.com>
 <CAEZATCVKMG7+b+_5tNwrNZ-aNDBy3=FMRNea2bO9O4qGcEvSTg@mail.gmail.com>
 <CAFj8pRDKQiiuHuzRwj6aca=ZkKpu7UdqBzgLmPmg_=BYYzD9zw@mail.gmail.com>
 <502641.1606334432@sss.pgh.pa.us>
 <CAFj8pRCtfaoD0QOAxrcQCqvJBbTr4ryhBkk2MJQhUJK-dLjdPw@mail.gmail.com>
 <20201128234945.GM24052@telsasoft.com>
 <CAFj8pRCrbDPAWyHjwZauEX78jyGBowOoiaWySA0ibRYoWzZTMw@mail.gmail.com>
 <CAFj8pRDcEjL-pEWmZj-3p5W6Xg29xbd6D2hdyPxZ5Znw0OXqJA@mail.gmail.com>
 <CAFj8pRBmOm9yxvOLbvYfZvAdVJ4jc084=-c3MG5_kZDk0NL5UA@mail.gmail.com>
 <CAFj8pRBkHSSq4epfVBfkabBtf_ZZMcRb8uC-YgxdbOzwCnhKug@mail.gmail.com>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
Message-ID: <cb545d78-2dae-8d27-f062-822a07ca56cf@enterprisedb.com>
Date: Sat, 10 Jul 2021 17:47:12 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: 
 <CAFj8pRBkHSSq4epfVBfkabBtf_ZZMcRb8uC-YgxdbOzwCnhKug@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/cb545d78-2dae-8d27-f062-822a07ca56cf%40enterprisedb.com>
Precedence: bulk

Hi,

I started looking at the patch allowing to export just functions [1], 
and I got pointed to this patch as an alternative approach (to adding a 
separate filtering option for every possible object type).

I'm familiar with the customer that inspired Pavel to start working on 
this, so I understand the use case he's trying to address - a flexible 
way to filter (include/exclude) large number of objects.

IMHO it's a mistake to try to broaden the scope of the patch and require 
implementing some universal pg_dump config file, particularly if it 
requires "complex" structure or formats like JSON, TOML or whatever. 
Maybe that's worth doing, but in my mind it's orthogonal to what this 
patch aims (or aimed) to do - filtering objects using rules in a file, 
not on the command line.

I believe it's much closer to .gitignore or rsync --filter than to a 
full config file. Even if we end up implementing the pg_dump config 
file, it'd be nice to keep the filter rules in a separate file and just 
reference that file from the config file.

That also means I find it pointless to use an "advanced" format like 
JSON or TOML - I think the format should be as simple as possible. Yes, 
it has to support all valid identifiers, comments and so on. But I don't 
quite see a point in using JSON or similar "full" format. If a simple 
format is good enough for rsync or gitignore, why should we insist on 
using something more complex?

OTOH I don't quite like the current approach of simply reading options 
from a file, because that requires adding new command-line options for 
each type of object we want to support. Which seems to contradict the 
idea of "general filter" method as mentioned in [1].

So if it was up to me, I'd go back to the original format or something 
close it. So something like this:

[+-] OBJECT_TYPE_PATTERN OBJECT_NAME_PATTERN


regards


[1] https://commitfest.postgresql.org/33/3051/

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company