public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 1/7] All grouping sets do their own sorting
142+ messages / 22 participants
[nested] [flat]
* [PATCH 1/7] All grouping sets do their own sorting
@ 2020-03-12 03:07 Pengzhou Tang <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Pengzhou Tang @ 2020-03-12 03:07 UTC (permalink / raw)
PG used to add a SORT path explicitly beneath�the AGG for sort aggregate,
grouping sets path also add a SORT path for the first sort aggregate phase,
but the following sort aggregate phases do their own sorting using a tuplesort.
This commit unified the way how grouping sets path doing sort, all sort aggregate
phases now do their own sorting using tuplesort.
This commit is mainly a preparing step to support parallel grouping sets, the
main idea of parallel grouping sets is: like parallel aggregate,� we separate
grouping sets into two stages:
The initial stage: this stage has almost the same plan and execution routines
with the current implementation of grouping sets, the differenceis are 1) it
only produces partial aggregate results 2) the output is attached with an extra
grouping set id. We know partial aggregate results will be combined in the final
stage and we have multiple�grouping sets, so only partial aggregate results
belong to the same grouping set can be combined, that is why grouping set id is
introduced to identify the sets. We keep all the optimizations of multiple
grouping sets in the initial stage, eg, 1) the grouping sets (that can be
grouped by one single sort) are put into the one rollup structure so those sets
arecomputed in one aggregate phase. 2) do hash aggregate concurrently when a
sort aggregate is performed. 3) do all hash transitions in one expression state.
The�final stage: this stage combine the partial aggregate results according to
the grouping set id. Obviously, all the optimizations�in the initial stage
cannot be used, so all rollups are extracted, each rollup contains only one
grouping set, then each aggregate phase only processes one set. We do a filter
in the final stage, redirect the tuples to each aggregate phase.
Obviously, adding a SORT path underneath the AGG in the final stage is not
right. This commit can avoid it and all non-hashed aggregate phases can do
their own sorting after thetuples are redirected.
---
src/backend/commands/explain.c | 5 +-
src/backend/executor/nodeAgg.c | 79 +++++++++++--
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/nodes/readfuncs.c | 1 +
src/backend/optimizer/plan/createplan.c | 65 ++++++++---
src/backend/optimizer/plan/planner.c | 66 +++++++----
src/backend/optimizer/util/pathnode.c | 30 ++++-
src/include/executor/nodeAgg.h | 2 -
src/include/nodes/execnodes.h | 5 +-
src/include/nodes/pathnodes.h | 1 +
src/include/nodes/plannodes.h | 1 +
src/include/optimizer/pathnode.h | 3 +-
src/include/optimizer/planmain.h | 2 +-
src/test/regress/expected/groupingsets.out | 130 ++++++++++-----------
15 files changed, 260 insertions(+), 132 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index d901dc4a50..b1609b339a 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2289,15 +2289,14 @@ show_grouping_sets(PlanState *planstate, Agg *agg,
ExplainOpenGroup("Grouping Sets", "Grouping Sets", false, es);
- show_grouping_set_keys(planstate, agg, NULL,
+ show_grouping_set_keys(planstate, agg, (Sort *) agg->sortnode,
context, useprefix, ancestors, es);
foreach(lc, agg->chain)
{
Agg *aggnode = lfirst(lc);
- Sort *sortnode = (Sort *) aggnode->plan.lefttree;
- show_grouping_set_keys(planstate, aggnode, sortnode,
+ show_grouping_set_keys(planstate, aggnode, (Sort *) aggnode->sortnode,
context, useprefix, ancestors, es);
}
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 7aebb247d8..b4f53bf77a 100644
--- a/src/backend/executor/nodeAgg.c
+++ b/src/backend/executor/nodeAgg.c
@@ -278,6 +278,7 @@ static void build_hash_table(AggState *aggstate, int setno, long nbuckets);
static AggStatePerGroup lookup_hash_entry(AggState *aggstate, uint32 hash);
static void lookup_hash_entries(AggState *aggstate);
static TupleTableSlot *agg_retrieve_direct(AggState *aggstate);
+static void agg_sort_input(AggState *aggstate);
static void agg_fill_hash_table(AggState *aggstate);
static TupleTableSlot *agg_retrieve_hash_table(AggState *aggstate);
static Datum GetAggInitVal(Datum textInitVal, Oid transtype);
@@ -367,7 +368,7 @@ initialize_phase(AggState *aggstate, int newphase)
*/
if (newphase > 0 && newphase < aggstate->numphases - 1)
{
- Sort *sortnode = aggstate->phases[newphase + 1].sortnode;
+ Sort *sortnode = (Sort *)aggstate->phases[newphase + 1].aggnode->sortnode;
PlanState *outerNode = outerPlanState(aggstate);
TupleDesc tupDesc = ExecGetResultType(outerNode);
@@ -1594,6 +1595,8 @@ ExecAgg(PlanState *pstate)
break;
case AGG_PLAIN:
case AGG_SORTED:
+ if (!node->input_sorted)
+ agg_sort_input(node);
result = agg_retrieve_direct(node);
break;
}
@@ -1945,6 +1948,45 @@ agg_retrieve_direct(AggState *aggstate)
return NULL;
}
+static void
+agg_sort_input(AggState *aggstate)
+{
+ AggStatePerPhase phase = &aggstate->phases[1];
+ TupleDesc tupDesc;
+ Sort *sortnode;
+
+ Assert(!aggstate->input_sorted);
+ Assert(phase->aggnode->sortnode);
+
+ sortnode = (Sort *) phase->aggnode->sortnode;
+ tupDesc = ExecGetResultType(outerPlanState(aggstate));
+
+ aggstate->sort_in = tuplesort_begin_heap(tupDesc,
+ sortnode->numCols,
+ sortnode->sortColIdx,
+ sortnode->sortOperators,
+ sortnode->collations,
+ sortnode->nullsFirst,
+ work_mem,
+ NULL, false);
+ for (;;)
+ {
+ TupleTableSlot *outerslot;
+
+ outerslot = ExecProcNode(outerPlanState(aggstate));
+ if (TupIsNull(outerslot))
+ break;
+
+ tuplesort_puttupleslot(aggstate->sort_in, outerslot);
+ }
+
+ /* Sort the first phase */
+ tuplesort_performsort(aggstate->sort_in);
+
+ /* Mark the input to be sorted */
+ aggstate->input_sorted = true;
+}
+
/*
* ExecAgg for hashed case: read input and build hash table
*/
@@ -2127,6 +2169,7 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
Plan *outerPlan;
ExprContext *econtext;
TupleDesc scanDesc;
+ Agg *firstSortAgg;
int numaggs,
transno,
aggno;
@@ -2171,6 +2214,7 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
aggstate->grp_firstTuple = NULL;
aggstate->sort_in = NULL;
aggstate->sort_out = NULL;
+ aggstate->input_sorted = true;
/*
* phases[0] always exists, but is dummy in sorted/plain mode
@@ -2178,6 +2222,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
numPhases = (use_hashing ? 1 : 2);
numHashes = (use_hashing ? 1 : 0);
+ firstSortAgg = node->aggstrategy == AGG_SORTED ? node : NULL;
+
/*
* Calculate the maximum number of grouping sets in any phase; this
* determines the size of some allocations. Also calculate the number of
@@ -2199,7 +2245,13 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
* others add an extra phase.
*/
if (agg->aggstrategy != AGG_HASHED)
+ {
++numPhases;
+
+ if (!firstSortAgg)
+ firstSortAgg = agg;
+
+ }
else
++numHashes;
}
@@ -2208,6 +2260,13 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
aggstate->maxsets = numGroupingSets;
aggstate->numphases = numPhases;
+ /*
+ * The first SORTED phase is not sorted, agg need to do its own sort. See
+ * agg_sort_input(), this can only happen in groupingsets case.
+ */
+ if (firstSortAgg && firstSortAgg->sortnode)
+ aggstate->input_sorted = false;
+
aggstate->aggcontexts = (ExprContext **)
palloc0(sizeof(ExprContext *) * numGroupingSets);
@@ -2269,7 +2328,8 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
* If there are more than two phases (including a potential dummy phase
* 0), input will be resorted using tuplesort. Need a slot for that.
*/
- if (numPhases > 2)
+ if (numPhases > 2 ||
+ !aggstate->input_sorted)
{
aggstate->sort_slot = ExecInitExtraTupleSlot(estate, scanDesc,
&TTSOpsMinimalTuple);
@@ -2340,20 +2400,11 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
for (phaseidx = 0; phaseidx <= list_length(node->chain); ++phaseidx)
{
Agg *aggnode;
- Sort *sortnode;
if (phaseidx > 0)
- {
aggnode = list_nth_node(Agg, node->chain, phaseidx - 1);
- sortnode = castNode(Sort, aggnode->plan.lefttree);
- }
else
- {
aggnode = node;
- sortnode = NULL;
- }
-
- Assert(phase <= 1 || sortnode);
if (aggnode->aggstrategy == AGG_HASHED
|| aggnode->aggstrategy == AGG_MIXED)
@@ -2470,7 +2521,6 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
phasedata->aggnode = aggnode;
phasedata->aggstrategy = aggnode->aggstrategy;
- phasedata->sortnode = sortnode;
}
}
@@ -3559,6 +3609,10 @@ ExecReScanAgg(AggState *node)
sizeof(AggStatePerGroupData) * node->numaggs);
}
+ /* Reset input_sorted */
+ if (aggnode->sortnode)
+ node->input_sorted = false;
+
/* reset to phase 1 */
initialize_phase(node, 1);
@@ -3566,6 +3620,7 @@ ExecReScanAgg(AggState *node)
node->projected_set = -1;
}
+
if (outerPlan->chgParam == NULL)
ExecReScan(outerPlan);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..04b4c65858 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -992,6 +992,7 @@ _copyAgg(const Agg *from)
COPY_BITMAPSET_FIELD(aggParams);
COPY_NODE_FIELD(groupingSets);
COPY_NODE_FIELD(chain);
+ COPY_NODE_FIELD(sortnode);
return newnode;
}
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e084c3f069..5816d122c1 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -787,6 +787,7 @@ _outAgg(StringInfo str, const Agg *node)
WRITE_BITMAPSET_FIELD(aggParams);
WRITE_NODE_FIELD(groupingSets);
WRITE_NODE_FIELD(chain);
+ WRITE_NODE_FIELD(sortnode);
}
static void
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index d5b23a3479..af4fcfe1ee 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2207,6 +2207,7 @@ _readAgg(void)
READ_BITMAPSET_FIELD(aggParams);
READ_NODE_FIELD(groupingSets);
READ_NODE_FIELD(chain);
+ READ_NODE_FIELD(sortnode);
READ_DONE();
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index fc25908dc6..d5b34089aa 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -1645,6 +1645,7 @@ create_unique_plan(PlannerInfo *root, UniquePath *best_path, int flags)
NIL,
best_path->path.rows,
0,
+ NULL,
subplan);
}
else
@@ -2098,6 +2099,7 @@ create_agg_plan(PlannerInfo *root, AggPath *best_path)
NIL,
best_path->numGroups,
best_path->transitionSpace,
+ NULL,
subplan);
copy_generic_path_info(&plan->plan, (Path *) best_path);
@@ -2159,6 +2161,7 @@ create_groupingsets_plan(PlannerInfo *root, GroupingSetsPath *best_path)
List *rollups = best_path->rollups;
AttrNumber *grouping_map;
int maxref;
+ int flags = CP_LABEL_TLIST;
List *chain;
ListCell *lc;
@@ -2168,9 +2171,15 @@ create_groupingsets_plan(PlannerInfo *root, GroupingSetsPath *best_path)
/*
* Agg can project, so no need to be terribly picky about child tlist, but
- * we do need grouping columns to be available
+ * we do need grouping columns to be available; If the groupingsets need
+ * to sort the input, the agg will store the input rows in a tuplesort,
+ * it therefore behooves us to request a small tlist to avoid wasting
+ * spaces.
*/
- subplan = create_plan_recurse(root, best_path->subpath, CP_LABEL_TLIST);
+ if (!best_path->is_sorted)
+ flags = flags | CP_SMALL_TLIST;
+
+ subplan = create_plan_recurse(root, best_path->subpath, flags);
/*
* Compute the mapping from tleSortGroupRef to column index in the child's
@@ -2230,12 +2239,22 @@ create_groupingsets_plan(PlannerInfo *root, GroupingSetsPath *best_path)
new_grpColIdx = remap_groupColIdx(root, rollup->groupClause);
- if (!rollup->is_hashed && !is_first_sort)
+ if (!rollup->is_hashed)
{
- sort_plan = (Plan *)
- make_sort_from_groupcols(rollup->groupClause,
- new_grpColIdx,
- subplan);
+ if (!is_first_sort ||
+ (is_first_sort && !best_path->is_sorted))
+ {
+ sort_plan = (Plan *)
+ make_sort_from_groupcols(rollup->groupClause,
+ new_grpColIdx,
+ subplan);
+
+ /*
+ * Remove stuff we don't need to avoid bloating debug output.
+ */
+ sort_plan->targetlist = NIL;
+ sort_plan->lefttree = NULL;
+ }
}
if (!rollup->is_hashed)
@@ -2260,16 +2279,8 @@ create_groupingsets_plan(PlannerInfo *root, GroupingSetsPath *best_path)
NIL,
rollup->numGroups,
best_path->transitionSpace,
- sort_plan);
-
- /*
- * Remove stuff we don't need to avoid bloating debug output.
- */
- if (sort_plan)
- {
- sort_plan->targetlist = NIL;
- sort_plan->lefttree = NULL;
- }
+ sort_plan,
+ NULL);
chain = lappend(chain, agg_plan);
}
@@ -2281,10 +2292,26 @@ create_groupingsets_plan(PlannerInfo *root, GroupingSetsPath *best_path)
{
RollupData *rollup = linitial(rollups);
AttrNumber *top_grpColIdx;
+ Plan *sort_plan = NULL;
int numGroupCols;
top_grpColIdx = remap_groupColIdx(root, rollup->groupClause);
+ /* the input is not sorted yet */
+ if (!rollup->is_hashed &&
+ !best_path->is_sorted)
+ {
+ sort_plan = (Plan *)
+ make_sort_from_groupcols(rollup->groupClause,
+ top_grpColIdx,
+ subplan);
+ /*
+ * Remove stuff we don't need to avoid bloating debug output.
+ */
+ sort_plan->targetlist = NIL;
+ sort_plan->lefttree = NULL;
+ }
+
numGroupCols = list_length((List *) linitial(rollup->gsets));
plan = make_agg(build_path_tlist(root, &best_path->path),
@@ -2299,6 +2326,7 @@ create_groupingsets_plan(PlannerInfo *root, GroupingSetsPath *best_path)
chain,
rollup->numGroups,
best_path->transitionSpace,
+ sort_plan,
subplan);
/* Copy cost data from Path to Plan */
@@ -6197,7 +6225,7 @@ make_agg(List *tlist, List *qual,
AggStrategy aggstrategy, AggSplit aggsplit,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
List *groupingSets, List *chain, double dNumGroups,
- Size transitionSpace, Plan *lefttree)
+ Size transitionSpace, Plan *sortnode, Plan *lefttree)
{
Agg *node = makeNode(Agg);
Plan *plan = &node->plan;
@@ -6217,6 +6245,7 @@ make_agg(List *tlist, List *qual,
node->aggParams = NULL; /* SS_finalize_plan() will fill this */
node->groupingSets = groupingSets;
node->chain = chain;
+ node->sortnode = sortnode;
plan->qual = qual;
plan->targetlist = tlist;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index b44efd6314..82a15761b4 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -175,7 +175,8 @@ static void consider_groupingsets_paths(PlannerInfo *root,
bool can_hash,
grouping_sets_data *gd,
const AggClauseCosts *agg_costs,
- double dNumGroups);
+ double dNumGroups,
+ AggStrategy strat);
static RelOptInfo *create_window_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *input_target,
@@ -4186,6 +4187,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
* it, by combinations of hashing and sorting. This can be called multiple
* times, so it's important that it not scribble on input. No result is
* returned, but any generated paths are added to grouped_rel.
+ *
+ * - strat:
+ * preferred aggregate strategy to use.
+ *
+ * - is_sorted:
+ * Is the input sorted on the groupCols of the first rollup. Caller
+ * must set it correctly if strat is set to AGG_SORTED, the planner
+ * uses it to generate a sortnode.
*/
static void
consider_groupingsets_paths(PlannerInfo *root,
@@ -4195,13 +4204,15 @@ consider_groupingsets_paths(PlannerInfo *root,
bool can_hash,
grouping_sets_data *gd,
const AggClauseCosts *agg_costs,
- double dNumGroups)
+ double dNumGroups,
+ AggStrategy strat)
{
Query *parse = root->parse;
+ Assert(strat == AGG_HASHED || strat == AGG_SORTED);
/*
- * If we're not being offered sorted input, then only consider plans that
- * can be done entirely by hashing.
+ * If strat is AGG_HASHED, then only consider plans that can be done
+ * entirely by hashing.
*
* We can hash everything if it looks like it'll fit in work_mem. But if
* the input is actually sorted despite not being advertised as such, we
@@ -4210,7 +4221,7 @@ consider_groupingsets_paths(PlannerInfo *root,
* If none of the grouping sets are sortable, then ignore the work_mem
* limit and generate a path anyway, since otherwise we'll just fail.
*/
- if (!is_sorted)
+ if (strat == AGG_HASHED)
{
List *new_rollups = NIL;
RollupData *unhashed_rollup = NULL;
@@ -4251,6 +4262,8 @@ consider_groupingsets_paths(PlannerInfo *root,
unhashed_rollup = lfirst_node(RollupData, l_start);
exclude_groups = unhashed_rollup->numGroups;
l_start = lnext(gd->rollups, l_start);
+ /* update is_sorted to true */
+ is_sorted = true;
}
hashsize = estimate_hashagg_tablesize(path,
@@ -4348,6 +4361,8 @@ consider_groupingsets_paths(PlannerInfo *root,
rollup->hashable = false;
rollup->is_hashed = false;
new_rollups = lappend(new_rollups, rollup);
+ /* update is_sorted to true */
+ is_sorted = true;
strat = AGG_MIXED;
}
@@ -4359,18 +4374,23 @@ consider_groupingsets_paths(PlannerInfo *root,
strat,
new_rollups,
agg_costs,
- dNumGroups));
+ dNumGroups,
+ is_sorted));
return;
}
/*
- * If we have sorted input but nothing we can do with it, bail.
+ * Strategy is AGG_SORTED but nothing we can do with it, bail.
*/
if (list_length(gd->rollups) == 0)
return;
/*
- * Given sorted input, we try and make two paths: one sorted and one mixed
+ * Callers consider AGG_SORTED strategy, the first rollup must
+ * use non-hashed aggregate, 'is_sorted' tells whether the first
+ * rollup need to do its own sort.
+ *
+ * we try and make two paths: one sorted and one mixed
* sort/hash. (We need to try both because hashagg might be disabled, or
* some columns might not be sortable.)
*
@@ -4427,7 +4447,7 @@ consider_groupingsets_paths(PlannerInfo *root,
/*
* We leave the first rollup out of consideration since it's the
- * one that matches the input sort order. We assign indexes "i"
+ * one that need to be sorted. We assign indexes "i"
* to only those entries considered for hashing; the second loop,
* below, must use the same condition.
*/
@@ -4516,7 +4536,8 @@ consider_groupingsets_paths(PlannerInfo *root,
AGG_MIXED,
rollups,
agg_costs,
- dNumGroups));
+ dNumGroups,
+ is_sorted));
}
}
@@ -4532,7 +4553,8 @@ consider_groupingsets_paths(PlannerInfo *root,
AGG_SORTED,
gd->rollups,
agg_costs,
- dNumGroups));
+ dNumGroups,
+ is_sorted));
}
/*
@@ -6399,6 +6421,16 @@ add_paths_to_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
path->pathkeys);
if (path == cheapest_path || is_sorted)
{
+ if (parse->groupingSets)
+ {
+ /* consider AGG_SORTED strategy */
+ consider_groupingsets_paths(root, grouped_rel,
+ path, is_sorted, can_hash,
+ gd, agg_costs, dNumGroups,
+ AGG_SORTED);
+ continue;
+ }
+
/* Sort the cheapest-total path if it isn't already sorted */
if (!is_sorted)
path = (Path *) create_sort_path(root,
@@ -6407,14 +6439,7 @@ add_paths_to_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
root->group_pathkeys,
-1.0);
- /* Now decide what to stick atop it */
- if (parse->groupingSets)
- {
- consider_groupingsets_paths(root, grouped_rel,
- path, true, can_hash,
- gd, agg_costs, dNumGroups);
- }
- else if (parse->hasAggs)
+ if (parse->hasAggs)
{
/*
* We have aggregation, possibly with plain GROUP BY. Make
@@ -6514,7 +6539,8 @@ add_paths_to_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
*/
consider_groupingsets_paths(root, grouped_rel,
cheapest_path, false, true,
- gd, agg_costs, dNumGroups);
+ gd, agg_costs, dNumGroups,
+ AGG_HASHED);
}
else
{
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d9ce516211..0feb3363d3 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2983,6 +2983,7 @@ create_agg_path(PlannerInfo *root,
* 'rollups' is a list of RollupData nodes
* 'agg_costs' contains cost info about the aggregate functions to be computed
* 'numGroups' is the estimated total number of groups
+ * 'is_sorted' is the input sorted in the group cols of first rollup
*/
GroupingSetsPath *
create_groupingsets_path(PlannerInfo *root,
@@ -2992,7 +2993,8 @@ create_groupingsets_path(PlannerInfo *root,
AggStrategy aggstrategy,
List *rollups,
const AggClauseCosts *agg_costs,
- double numGroups)
+ double numGroups,
+ bool is_sorted)
{
GroupingSetsPath *pathnode = makeNode(GroupingSetsPath);
PathTarget *target = rel->reltarget;
@@ -3010,6 +3012,7 @@ create_groupingsets_path(PlannerInfo *root,
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->subpath = subpath;
+ pathnode->is_sorted = is_sorted;
/*
* Simplify callers by downgrading AGG_SORTED to AGG_PLAIN, and AGG_MIXED
@@ -3061,14 +3064,33 @@ create_groupingsets_path(PlannerInfo *root,
*/
if (is_first)
{
+ Cost input_startup_cost = subpath->startup_cost;
+ Cost input_total_cost = subpath->total_cost;
+
+ if (!rollup->is_hashed && !is_sorted && numGroupCols)
+ {
+ Path sort_path; /* dummy for result of cost_sort */
+
+ cost_sort(&sort_path, root, NIL,
+ input_total_cost,
+ subpath->rows,
+ subpath->pathtarget->width,
+ 0.0,
+ work_mem,
+ -1.0);
+
+ input_startup_cost = sort_path.startup_cost;
+ input_total_cost = sort_path.total_cost;
+ }
+
cost_agg(&pathnode->path, root,
aggstrategy,
agg_costs,
numGroupCols,
rollup->numGroups,
having_qual,
- subpath->startup_cost,
- subpath->total_cost,
+ input_startup_cost,
+ input_total_cost,
subpath->rows);
is_first = false;
if (!rollup->is_hashed)
@@ -3079,7 +3101,7 @@ create_groupingsets_path(PlannerInfo *root,
Path sort_path; /* dummy for result of cost_sort */
Path agg_path; /* dummy for result of cost_agg */
- if (rollup->is_hashed || is_first_sort)
+ if (rollup->is_hashed || (is_first_sort && is_sorted))
{
/*
* Account for cost of aggregation, but don't charge input
diff --git a/src/include/executor/nodeAgg.h b/src/include/executor/nodeAgg.h
index 264916f9a9..66a83b9ac9 100644
--- a/src/include/executor/nodeAgg.h
+++ b/src/include/executor/nodeAgg.h
@@ -277,8 +277,6 @@ typedef struct AggStatePerPhaseData
ExprState **eqfunctions; /* expression returning equality, indexed by
* nr of cols to compare */
Agg *aggnode; /* Agg node for phase data */
- Sort *sortnode; /* Sort node for input ordering for phase */
-
ExprState *evaltrans; /* evaluation of transition functions */
} AggStatePerPhaseData;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cd3ddf781f..5e33a368f5 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -2083,8 +2083,11 @@ typedef struct AggState
AggStatePerGroup *hash_pergroup; /* grouping set indexed array of
* per-group pointers */
+ /* these fields are used in AGG_SORTED and AGG_MIXED */
+ bool input_sorted; /* hash table filled yet? */
+
/* support for evaluation of agg input expressions: */
-#define FIELDNO_AGGSTATE_ALL_PERGROUPS 34
+#define FIELDNO_AGGSTATE_ALL_PERGROUPS 35
AggStatePerGroup *all_pergroups; /* array of first ->pergroups, than
* ->hash_pergroup */
ProjectionInfo *combinedproj; /* projection machinery */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 0ceb809644..c1e69c808f 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1702,6 +1702,7 @@ typedef struct GroupingSetsPath
List *rollups; /* list of RollupData */
List *qual; /* quals (HAVING quals), if any */
uint64 transitionSpace; /* for pass-by-ref transition data */
+ bool is_sorted; /* input sorted in groupcols of first rollup */
} GroupingSetsPath;
/*
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 4869fe7b6d..3cd2537e9e 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -818,6 +818,7 @@ typedef struct Agg
/* Note: planner provides numGroups & aggParams only in HASHED/MIXED case */
List *groupingSets; /* grouping sets to use */
List *chain; /* chained Agg/Sort nodes */
+ Plan *sortnode; /* agg does its own sort, only used by grouping sets now */
} Agg;
/* ----------------
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index e450fe112a..f9f388ba06 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -217,7 +217,8 @@ extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
AggStrategy aggstrategy,
List *rollups,
const AggClauseCosts *agg_costs,
- double numGroups);
+ double numGroups,
+ bool is_sorted);
extern MinMaxAggPath *create_minmaxagg_path(PlannerInfo *root,
RelOptInfo *rel,
PathTarget *target,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 4781201001..5954ff3997 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -55,7 +55,7 @@ extern Agg *make_agg(List *tlist, List *qual,
AggStrategy aggstrategy, AggSplit aggsplit,
int numGroupCols, AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
List *groupingSets, List *chain, double dNumGroups,
- Size transitionSpace, Plan *lefttree);
+ Size transitionSpace, Plan *sortnode, Plan *lefttree);
extern Limit *make_limit(Plan *lefttree, Node *limitOffset, Node *limitCount);
/*
diff --git a/src/test/regress/expected/groupingsets.out b/src/test/regress/expected/groupingsets.out
index c1f802c88a..12425f46ca 100644
--- a/src/test/regress/expected/groupingsets.out
+++ b/src/test/regress/expected/groupingsets.out
@@ -366,15 +366,14 @@ explain (costs off)
select g as alias1, g as alias2
from generate_series(1,3) g
group by alias1, rollup(alias2);
- QUERY PLAN
-------------------------------------------------
+ QUERY PLAN
+------------------------------------------
GroupAggregate
- Group Key: g, g
- Group Key: g
- -> Sort
- Sort Key: g
- -> Function Scan on generate_series g
-(6 rows)
+ Sort Key: g, g
+ Group Key: g, g
+ Group Key: g
+ -> Function Scan on generate_series g
+(5 rows)
select g as alias1, g as alias2
from generate_series(1,3) g
@@ -640,15 +639,14 @@ select a, b, sum(v.x)
-- Test reordering of grouping sets
explain (costs off)
select * from gstest1 group by grouping sets((a,b,v),(v)) order by v,b,a;
- QUERY PLAN
-------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------------------
GroupAggregate
- Group Key: "*VALUES*".column3, "*VALUES*".column2, "*VALUES*".column1
- Group Key: "*VALUES*".column3
- -> Sort
- Sort Key: "*VALUES*".column3, "*VALUES*".column2, "*VALUES*".column1
- -> Values Scan on "*VALUES*"
-(6 rows)
+ Sort Key: "*VALUES*".column3, "*VALUES*".column2, "*VALUES*".column1
+ Group Key: "*VALUES*".column3, "*VALUES*".column2, "*VALUES*".column1
+ Group Key: "*VALUES*".column3
+ -> Values Scan on "*VALUES*"
+(5 rows)
-- Agg level check. This query should error out.
select (select grouping(a,b) from gstest2) from gstest2 group by a,b;
@@ -723,13 +721,12 @@ explain (costs off)
QUERY PLAN
----------------------------------
GroupAggregate
- Group Key: a
- Group Key: ()
+ Sort Key: a
+ Group Key: a
+ Group Key: ()
Filter: (a IS DISTINCT FROM 1)
- -> Sort
- Sort Key: a
- -> Seq Scan on gstest2
-(7 rows)
+ -> Seq Scan on gstest2
+(6 rows)
select v.c, (select count(*) from gstest2 group by () having v.c)
from (values (false),(true)) v(c) order by v.c;
@@ -1018,18 +1015,17 @@ explain (costs off) select a, b, grouping(a,b), sum(v), count(*), max(v)
explain (costs off)
select a, b, grouping(a,b), array_agg(v order by v)
from gstest1 group by cube(a,b);
- QUERY PLAN
-----------------------------------------------------------
+ QUERY PLAN
+-------------------------------------------------------
GroupAggregate
- Group Key: "*VALUES*".column1, "*VALUES*".column2
- Group Key: "*VALUES*".column1
- Group Key: ()
+ Sort Key: "*VALUES*".column1, "*VALUES*".column2
+ Group Key: "*VALUES*".column1, "*VALUES*".column2
+ Group Key: "*VALUES*".column1
+ Group Key: ()
Sort Key: "*VALUES*".column2
Group Key: "*VALUES*".column2
- -> Sort
- Sort Key: "*VALUES*".column1, "*VALUES*".column2
- -> Values Scan on "*VALUES*"
-(9 rows)
+ -> Values Scan on "*VALUES*"
+(8 rows)
-- unsortable cases
select unsortable_col, count(*)
@@ -1071,11 +1067,10 @@ explain (costs off)
Sort Key: (GROUPING(unhashable_col, unsortable_col)), (sum(v))
-> MixedAggregate
Hash Key: unsortable_col
- Group Key: unhashable_col
- -> Sort
- Sort Key: unhashable_col
- -> Seq Scan on gstest4
-(8 rows)
+ Sort Key: unhashable_col
+ Group Key: unhashable_col
+ -> Seq Scan on gstest4
+(7 rows)
select unhashable_col, unsortable_col,
grouping(unhashable_col, unsortable_col),
@@ -1114,11 +1109,10 @@ explain (costs off)
Sort Key: (GROUPING(unhashable_col, unsortable_col)), (sum(v))
-> MixedAggregate
Hash Key: v, unsortable_col
- Group Key: v, unhashable_col
- -> Sort
- Sort Key: v, unhashable_col
- -> Seq Scan on gstest4
-(8 rows)
+ Sort Key: v, unhashable_col
+ Group Key: v, unhashable_col
+ -> Seq Scan on gstest4
+(7 rows)
-- empty input: first is 0 rows, second 1, third 3 etc.
select a, b, sum(v), count(*) from gstest_empty group by grouping sets ((a,b),a);
@@ -1366,19 +1360,18 @@ explain (costs off)
BEGIN;
SET LOCAL enable_hashagg = false;
EXPLAIN (COSTS OFF) SELECT a, b, count(*), max(a), max(b) FROM gstest3 GROUP BY GROUPING SETS(a, b,()) ORDER BY a, b;
- QUERY PLAN
----------------------------------------
+ QUERY PLAN
+---------------------------------
Sort
Sort Key: a, b
-> GroupAggregate
- Group Key: a
- Group Key: ()
+ Sort Key: a
+ Group Key: a
+ Group Key: ()
Sort Key: b
Group Key: b
- -> Sort
- Sort Key: a
- -> Seq Scan on gstest3
-(10 rows)
+ -> Seq Scan on gstest3
+(9 rows)
SELECT a, b, count(*), max(a), max(b) FROM gstest3 GROUP BY GROUPING SETS(a, b,()) ORDER BY a, b;
a | b | count | max | max
@@ -1549,22 +1542,21 @@ explain (costs off)
count(hundred), count(thousand), count(twothousand),
count(*)
from tenk1 group by grouping sets (unique1,twothousand,thousand,hundred,ten,four,two);
- QUERY PLAN
--------------------------------
+ QUERY PLAN
+----------------------------
MixedAggregate
Hash Key: two
Hash Key: four
Hash Key: ten
Hash Key: hundred
- Group Key: unique1
+ Sort Key: unique1
+ Group Key: unique1
Sort Key: twothousand
Group Key: twothousand
Sort Key: thousand
Group Key: thousand
- -> Sort
- Sort Key: unique1
- -> Seq Scan on tenk1
-(13 rows)
+ -> Seq Scan on tenk1
+(12 rows)
explain (costs off)
select unique1,
@@ -1572,18 +1564,17 @@ explain (costs off)
count(hundred), count(thousand), count(twothousand),
count(*)
from tenk1 group by grouping sets (unique1,hundred,ten,four,two);
- QUERY PLAN
--------------------------------
+ QUERY PLAN
+-------------------------
MixedAggregate
Hash Key: two
Hash Key: four
Hash Key: ten
Hash Key: hundred
- Group Key: unique1
- -> Sort
- Sort Key: unique1
- -> Seq Scan on tenk1
-(9 rows)
+ Sort Key: unique1
+ Group Key: unique1
+ -> Seq Scan on tenk1
+(8 rows)
set work_mem = '384kB';
explain (costs off)
@@ -1592,21 +1583,20 @@ explain (costs off)
count(hundred), count(thousand), count(twothousand),
count(*)
from tenk1 group by grouping sets (unique1,twothousand,thousand,hundred,ten,four,two);
- QUERY PLAN
--------------------------------
+ QUERY PLAN
+----------------------------
MixedAggregate
Hash Key: two
Hash Key: four
Hash Key: ten
Hash Key: hundred
Hash Key: thousand
- Group Key: unique1
+ Sort Key: unique1
+ Group Key: unique1
Sort Key: twothousand
Group Key: twothousand
- -> Sort
- Sort Key: unique1
- -> Seq Scan on tenk1
-(12 rows)
+ -> Seq Scan on tenk1
+(11 rows)
-- check collation-sensitive matching between grouping expressions
-- (similar to a check for aggregates, but there are additional code
--
2.21.1
--4ms6w442s2ji2wqe
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="0002-fixes.patch"
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-04 11:34 Andres Freund <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Andres Freund @ 2023-02-04 11:34 UTC (permalink / raw)
To: Noah Misch <[email protected]>; +Cc: Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Hi,
On 2023-02-03 20:18:48 -0800, Noah Misch wrote:
> On Fri, Feb 03, 2023 at 12:52:50PM -0500, Tom Lane wrote:
> > Andrew Dunstan <[email protected]> writes:
> > > On 2023-01-22 Su 17:47, Tom Lane wrote:
> > >> Yeah. That's one of my biggest gripes about pgperltidy: if you insert
> > >> another assignment in a series of assignments, it is very likely to
> > >> reformat all the adjacent assignments because it thinks it's cool to
> > >> make all the equal signs line up. That's just awful.
> >
> > > Modern versions of perltidy give you much more control over this, so
> > > maybe we need to investigate the possibility of updating.
> >
> > I have no objection to updating perltidy from time to time. I think the
> > idea is just to make sure that we have an agreed-on version for everyone
> > to use.
>
> Agreed. If we're changing the indentation of assignments, that's a
> considerable diff already. It would be a good time to absorb other diffs
> we'll want eventually, including diffs from a perltidy version upgrade.
ISTM that we're closer to being able to enforce pgindent than
perltidy. At the same time, I think the issue of C code in HEAD not
being indented is more pressing - IME it's much more common to have to
touch a lot of C code than to have to touch a lot fo perl files. So
perhaps we should just start with being more stringent with C code, and
once we made perltidy less noisy, add that?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-04 14:20 Andrew Dunstan <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-04 14:20 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Noah Misch <[email protected]>; +Cc: Tom Lane <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-04 Sa 06:34, Andres Freund wrote:
>
> ISTM that we're closer to being able to enforce pgindent than
> perltidy. At the same time, I think the issue of C code in HEAD not
> being indented is more pressing - IME it's much more common to have to
> touch a lot of C code than to have to touch a lot fo perl files. So
> perhaps we should just start with being more stringent with C code, and
> once we made perltidy less noisy, add that?
>
Sure, we don't have to tie them together.
I'm currently experimenting with settings on the buildfarm code, trying
to get it both stable and looking nice. Then I'll try on the Postgres
core code and post some results.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-04 16:07 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-02-04 16:07 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Noah Misch <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Andres Freund <[email protected]> writes:
> ISTM that we're closer to being able to enforce pgindent than
> perltidy. At the same time, I think the issue of C code in HEAD not
> being indented is more pressing - IME it's much more common to have to
> touch a lot of C code than to have to touch a lot fo perl files. So
> perhaps we should just start with being more stringent with C code, and
> once we made perltidy less noisy, add that?
Agreed, we should move more slowly with perltidy. Aside from the
points you raise, I bet fewer committers have it installed at all.
(I haven't forgotten that I'm on the hook to import pg_bsd_indent
into our tree. Will get to that soon.)
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-04 17:11 Justin Pryzby <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Justin Pryzby @ 2023-02-04 17:11 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Noah Misch <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Sat, Feb 04, 2023 at 11:07:59AM -0500, Tom Lane wrote:
> (I haven't forgotten that I'm on the hook to import pg_bsd_indent
> into our tree. Will get to that soon.)
+1 for that - it's no surprise that you have trouble convincing people
to follow the current process:
1) requires using a hacked copy of BSD indent; 2) which is stored
outside the main repo; 3) is run via a perl script that itself mungles
the source code (because the only indent tool that can support the
project's style doesn't actually support what's needed); 4) and wants to
retrieve a remote copy of typedefs.list (?).
The only thing that makes this scheme even remotely viable is that
apt.postgresql.org includes a package for pg-bsd-indent. I've used it
only a handful of times by running:
pg_bsd_indent -bad -bap -bbb -bc -bl -cli1 -cp33 -cdb -nce -d0 -di12 -nfc1 -i4 -l79 -lp -lpl -nip -npro -sac -tpg -ts4 -U .../typedefs.list
The perl wrapper is still a step too far for me (maybe it'd be tolerable
if available as a build target).
Would you want to make those the default options of the in-tree indent ?
Or provide a shortcut like --postgresql ?
--
Justin
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-04 17:37 Tom Lane <[email protected]>
parent: Justin Pryzby <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Tom Lane @ 2023-02-04 17:37 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: Andres Freund <[email protected]>; Noah Misch <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Justin Pryzby <[email protected]> writes:
> Would you want to make those the default options of the in-tree indent ?
> Or provide a shortcut like --postgresql ?
Hmmm ... inserting all of those as the default options would likely
make it impossible to update pg_bsd_indent itself with anything like
its current indent style (not that it's terribly consistent about
that). I could see inventing a --postgresql shortcut switch perhaps.
But it's not clear to me why you're allergic to the perl wrapper?
It's not like that's the only perl infrastructure in our build process.
Also, whether or not we could push some of what it does into pg_bsd_indent
proper, I can't see pushing all of it (for instance, the very PG-specific
list of typedef exclusions).
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-05 14:29 Andrew Dunstan <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-05 14:29 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Noah Misch <[email protected]>; +Cc: Tom Lane <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-04 Sa 09:20, Andrew Dunstan wrote:
>
>
> On 2023-02-04 Sa 06:34, Andres Freund wrote:
>>
>> ISTM that we're closer to being able to enforce pgindent than
>> perltidy. At the same time, I think the issue of C code in HEAD not
>> being indented is more pressing - IME it's much more common to have to
>> touch a lot of C code than to have to touch a lot fo perl files. So
>> perhaps we should just start with being more stringent with C code, and
>> once we made perltidy less noisy, add that?
>>
>
> Sure, we don't have to tie them together.
>
> I'm currently experimenting with settings on the buildfarm code,
> trying to get it both stable and looking nice. Then I'll try on the
> Postgres core code and post some results.
>
So here's a diff made from running perltidy v20221112 with the
additional setting --valign-exclusion-list=", = => || && if unless"
Essentially this abandons those bits of vertical alignment that tend to
catch us out when additions are made to the code.
I think this will make the code much more maintainable and result in
much less perltidy churn. It would also mean that it's far more likely
that what we would naturally code can be undisturbed by perltidy.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
Attachments:
[application/gzip] pgperltidy-no-valign.patch.gz (102.4K, ../../[email protected]/3-pgperltidy-no-valign.patch.gz)
download
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-06 14:40 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 3 replies; 142+ messages in thread
From: Robert Haas @ 2023-02-06 14:40 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Sat, Feb 4, 2023 at 12:37 PM Tom Lane <[email protected]> wrote:
> But it's not clear to me why you're allergic to the perl wrapper?
> It's not like that's the only perl infrastructure in our build process.
> Also, whether or not we could push some of what it does into pg_bsd_indent
> proper, I can't see pushing all of it (for instance, the very PG-specific
> list of typedef exclusions).
I don't mind that there is a script. I do mind that it's not that good
of a script. There have been some improvements for which I am
grateful, like removing the thing where the first argument was taken
as a typedefs file under some circumstances. But there are still some
things that I would like:
1. I'd like to be able to run pgindent src/include and have it indent
everything relevant under src/include. Right now that silently does
nothing.
2. I'd like an easy way to indent the unstaged files in the current
directory (e.g. pgindent --dirty) or the files that have been queued
up for commit (e.g. pgindent --cached).
3. I'd also like an easy way to indent every file touched by a recent
commit, e.g. pgindent --commit HEAD, pgindent --commit HEAD~2,
pgindent --commit 62e1e28bf7.
It'd be much less annoying to include this in my workflow with these
kinds of options. For instance:
patch -p1 < ~/Downloads/some_stuff_v94.patch
# committer adjustments as desired
git add -u
pgindent --cached
git diff # did pgindent change anything? does it look ok?
git commit -a
For a while I, like some others here, was trying to be religious about
pgindenting at least the bigger commits that I pushed. But I fear I've
grown slack. I don't mind if we tighten up the process, but the better
we make the tools, the less friction it will cause.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-06 15:16 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-02-06 15:16 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Robert Haas <[email protected]> writes:
> I don't mind that there is a script. I do mind that it's not that good
> of a script. There have been some improvements for which I am
> grateful, like removing the thing where the first argument was taken
> as a typedefs file under some circumstances. But there are still some
> things that I would like:
I have no objection to someone coding those things up ;-).
I'll just note that adding features like those to a Perl script
is going to be a ton easier than doing it inside pg_bsd_indent.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-06 15:21 Andrew Dunstan <[email protected]>
parent: Robert Haas <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-06 15:21 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Tom Lane <[email protected]>; +Cc: Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-06 Mo 09:40, Robert Haas wrote:
> On Sat, Feb 4, 2023 at 12:37 PM Tom Lane<[email protected]> wrote:
>> But it's not clear to me why you're allergic to the perl wrapper?
>> It's not like that's the only perl infrastructure in our build process.
>> Also, whether or not we could push some of what it does into pg_bsd_indent
>> proper, I can't see pushing all of it (for instance, the very PG-specific
>> list of typedef exclusions).
> I don't mind that there is a script. I do mind that it's not that good
> of a script. There have been some improvements for which I am
> grateful, like removing the thing where the first argument was taken
> as a typedefs file under some circumstances. But there are still some
> things that I would like:
>
> 1. I'd like to be able to run pgindent src/include and have it indent
> everything relevant under src/include. Right now that silently does
> nothing.
>
> 2. I'd like an easy way to indent the unstaged files in the current
> directory (e.g. pgindent --dirty) or the files that have been queued
> up for commit (e.g. pgindent --cached).
>
> 3. I'd also like an easy way to indent every file touched by a recent
> commit, e.g. pgindent --commit HEAD, pgindent --commit HEAD~2,
> pgindent --commit 62e1e28bf7.
Good suggestions. 1 and 3 seem fairly straightforward. I'll start on
those, and look into 2.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-06 15:35 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Robert Haas @ 2023-02-06 15:35 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Mon, Feb 6, 2023 at 10:16 AM Tom Lane <[email protected]> wrote:
> I'll just note that adding features like those to a Perl script
> is going to be a ton easier than doing it inside pg_bsd_indent.
+1.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-06 15:36 Robert Haas <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Robert Haas @ 2023-02-06 15:36 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Mon, Feb 6, 2023 at 10:21 AM Andrew Dunstan <[email protected]> wrote:
> Good suggestions. 1 and 3 seem fairly straightforward. I'll start on those, and look into 2.
Thanks!
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-06 17:03 Andrew Dunstan <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-06 17:03 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-06 Mo 10:36, Robert Haas wrote:
> On Mon, Feb 6, 2023 at 10:21 AM Andrew Dunstan<[email protected]> wrote:
>> Good suggestions. 1 and 3 seem fairly straightforward. I'll start on those, and look into 2.
> Thanks!
>
Here's a quick patch for 1 and 3. Would also need to adjust the docco.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-06 17:53 Andrew Dunstan <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-06 17:53 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-06 Mo 12:03, Andrew Dunstan wrote:
>
>
> On 2023-02-06 Mo 10:36, Robert Haas wrote:
>> On Mon, Feb 6, 2023 at 10:21 AM Andrew Dunstan<[email protected]> wrote:
>>> Good suggestions. 1 and 3 seem fairly straightforward. I'll start on those, and look into 2.
>> Thanks!
>>
>
> Here's a quick patch for 1 and 3. Would also need to adjust the docco.
>
>
This time with patch.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
Attachments:
[text/x-patch] pgindent-directory+git-enhancements.patch (3.2K, ../../[email protected]/3-pgindent-directory+git-enhancements.patch)
download | inline diff:
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index 56640e576a..ca329747a4 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -23,12 +23,14 @@ my $devnull = File::Spec->devnull;
my ($typedefs_file, $typedef_str, $code_base,
@excludes, $indent, $build,
- $show_diff, $silent_diff, $help);
+ $show_diff, $silent_diff, $help,
+ @commits,);
$help = 0;
my %options = (
"help" => \$help,
+ "commit=s" => \@commits,
"typedefs=s" => \$typedefs_file,
"list-of-typedefs=s" => \$typedef_str,
"code-base=s" => \$code_base,
@@ -44,6 +46,9 @@ usage() if $help;
usage("Cannot have both --silent-diff and --show-diff")
if $silent_diff && $show_diff;
+usage("Cannot use --commit with --code-base or command line file list")
+ if (@commits && ($code_base || @ARGV));
+
run_build($code_base) if ($build);
# command line option wins, then environment (which is how --build sets it) ,
@@ -388,6 +393,7 @@ Usage:
pgindent [OPTION]... [FILE]...
Options:
--help show this message and quit
+ --commit=gitref use files modified by the named commit
--typedefs=FILE file containing a list of typedefs
--list-of-typedefs=STR string containing typedefs, space separated
--code-base=DIR path to the base of PostgreSQL source code
@@ -396,7 +402,7 @@ Options:
--build build the pg_bsd_indent program
--show-diff show the changes that would be made
--silent-diff exit with status 2 if any changes would be made
-The --excludes option can be given more than once.
+The --excludes and --commit options can be given more than once.
EOF
if ($help)
{
@@ -412,27 +418,38 @@ EOF
# main
-# get the list of files under code base, if it's set
-File::Find::find(
- {
- wanted => sub {
- my ($dev, $ino, $mode, $nlink, $uid, $gid);
- (($dev, $ino, $mode, $nlink, $uid, $gid) = lstat($_))
- && -f _
- && /^.*\.[ch]\z/s
- && push(@files, $File::Find::name);
- }
- },
- $code_base) if $code_base;
-
$filtered_typedefs_fh = load_typedefs();
check_indent();
-# any non-option arguments are files to be processed
-push(@files, @ARGV);
+build_clean($code_base) if $build;
-# the exclude list applies to command line arguments as well as found files
+my $wanted = sub
+{
+ my ($dev, $ino, $mode, $nlink, $uid, $gid);
+ (($dev, $ino, $mode, $nlink, $uid, $gid) = lstat($_))
+ && -f _
+ && /^.*\.[ch]\z/s
+ && push(@files, $File::Find::name);
+};
+
+# get the list of files under code base, if it's set
+File::Find::find({wanted => $wanted }, $code_base) if $code_base;
+
+# any non-option arguments are files or directories to be processed
+File::Find::find({wanted => $wanted}, @ARGV) if @ARGV;
+
+# process named commits by comparing each with their immediate ancestor
+foreach my $commit (@commits)
+{
+ my $prev="$commit~";
+ my @affected=`git diff-tree --no-commit-id --name-only -r $commit $prev`;
+ die "git error" if $?;
+ chomp(@affected);
+ push(@files,@affected);
+}
+
+# remove excluded files from the file list
process_exclude();
foreach my $source_filename (@files)
@@ -481,6 +498,4 @@ foreach my $source_filename (@files)
}
}
-build_clean($code_base) if $build;
-
exit 0;
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-07 13:17 Andrew Dunstan <[email protected]>
parent: Robert Haas <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-07 13:17 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Tom Lane <[email protected]>; +Cc: Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-06 Mo 09:40, Robert Haas wrote:
> 2. I'd like an easy way to indent the unstaged files in the current
> directory (e.g. pgindent --dirty) or the files that have been queued
> up for commit (e.g. pgindent --cached).
>
My git-fu is probably not all that it should be. I think we could
possibly get at this list of files by running
git status --porcelain --untracked-files=no --ignored=no -- .
And then your --dirty list would be lines beginning with ' M' while your
--cached list would be lines beginning with 'A[ M]'
Does that seem plausible?
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-07 15:25 Justin Pryzby <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 2 replies; 142+ messages in thread
From: Justin Pryzby @ 2023-02-07 15:25 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Noah Misch <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Sat, Feb 04, 2023 at 12:37:11PM -0500, Tom Lane wrote:
> Justin Pryzby <[email protected]> writes:
> Hmmm ... inserting all of those as the default options would likely
> make it impossible to update pg_bsd_indent itself with anything like
> its current indent style (not that it's terribly consistent about
> that). I could see inventing a --postgresql shortcut switch perhaps.
Or you could add ./.indent.pro, or ./src/tools/indent.profile for it to
read.
> > Would you want to make those the default options of the in-tree indent ?
> > Or provide a shortcut like --postgresql ?
>
> But it's not clear to me why you're allergic to the perl wrapper?
My allergy is to the totality of the process, not to the perl component.
It's a bit weird to enforce a coding style that no upstream indent tool
supports. But what's weirder is that, *having forked the indent tool*,
it still doesn't implement the desired style, and the perl wrapper tries
to work around that.
It would be more reasonable if the forked C program knew how to handle
the stuff for which the perl script currently has kludges to munge the
source code before indenting and then un-munging afterwards.
Or if the indentation were handled by the (or a) perl script itself.
Or if the perl script handled everything that an unpatched "ident"
didn't handle, rather than some things but not others, demanding use of
a patched indent as well as a perl wrapper (not just for looping around
files and fancy high-level shortcuts like indenting staged files).
On the one hand, "indent" ought to handle all the source-munging stuff.
On the other hand, it'd be better to use an unpatched indent tool. The
current way is the worst of both worlds.
Currently, the perl wrapper supports the "/* -"-style comments that
postgres wants to use (why?) by munging the source code. That could be
supported in pg-bsd-indent with a one line change. I think an even
better option would be to change postgres' C files to use "/*-" without
a space, which requires neither perl munging nor patching indent.
On a less critical note, I wonder if it's a good idea to import
pgbsdindent as a git "submodule".
--
Justin
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-07 15:46 Tom Lane <[email protected]>
parent: Justin Pryzby <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Tom Lane @ 2023-02-07 15:46 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: Andres Freund <[email protected]>; Noah Misch <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Justin Pryzby <[email protected]> writes:
> On Sat, Feb 04, 2023 at 12:37:11PM -0500, Tom Lane wrote:
>> But it's not clear to me why you're allergic to the perl wrapper?
> My allergy is to the totality of the process, not to the perl component.
> It's a bit weird to enforce a coding style that no upstream indent tool
> supports. But what's weirder is that, *having forked the indent tool*,
> it still doesn't implement the desired style, and the perl wrapper tries
> to work around that.
> It would be more reasonable if the forked C program knew how to handle
> the stuff for which the perl script currently has kludges to munge the
> source code before indenting and then un-munging afterwards.
[ shrug... ] If you want to put cycles into that, nobody is stopping
you. For me, it sounds like make-work.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-07 15:46 Andrew Dunstan <[email protected]>
parent: Justin Pryzby <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-07 15:46 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Noah Misch <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-07 Tu 10:25, Justin Pryzby wrote:
> On Sat, Feb 04, 2023 at 12:37:11PM -0500, Tom Lane wrote:
>> Justin Pryzby<[email protected]> writes:
>> Hmmm ... inserting all of those as the default options would likely
>> make it impossible to update pg_bsd_indent itself with anything like
>> its current indent style (not that it's terribly consistent about
>> that). I could see inventing a --postgresql shortcut switch perhaps.
> Or you could add ./.indent.pro, or ./src/tools/indent.profile for it to
> read.
>
>>> Would you want to make those the default options of the in-tree indent ?
>>> Or provide a shortcut like --postgresql ?
>> But it's not clear to me why you're allergic to the perl wrapper?
> My allergy is to the totality of the process, not to the perl component.
> It's a bit weird to enforce a coding style that no upstream indent tool
> supports. But what's weirder is that, *having forked the indent tool*,
> it still doesn't implement the desired style, and the perl wrapper tries
> to work around that.
>
> It would be more reasonable if the forked C program knew how to handle
> the stuff for which the perl script currently has kludges to munge the
> source code before indenting and then un-munging afterwards.
>
> Or if the indentation were handled by the (or a) perl script itself.
>
> Or if the perl script handled everything that an unpatched "ident"
> didn't handle, rather than some things but not others, demanding use of
> a patched indent as well as a perl wrapper (not just for looping around
> files and fancy high-level shortcuts like indenting staged files).
>
> On the one hand, "indent" ought to handle all the source-munging stuff.
> On the other hand, it'd be better to use an unpatched indent tool. The
> current way is the worst of both worlds.
>
> Currently, the perl wrapper supports the "/* -"-style comments that
> postgres wants to use (why?) by munging the source code. That could be
> supported in pg-bsd-indent with a one line change. I think an even
> better option would be to change postgres' C files to use "/*-" without
> a space, which requires neither perl munging nor patching indent.
Historically we used to do a heck of a lot more in pgindent that is
currently done in the pre_indent and post_indent functions. If you want
to spend time implementing that logic in pg_bsd_indent so we can remove
the remaining bits of that processing then go for it.
> On a less critical note, I wonder if it's a good idea to import
> pgbsdindent as a git "submodule".
Meh, git submodules can be a pain in the neck in my limited experience.
I'd rather steer clear.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-07 16:10 Robert Haas <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Robert Haas @ 2023-02-07 16:10 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Tue, Feb 7, 2023 at 8:17 AM Andrew Dunstan <[email protected]> wrote:
> My git-fu is probably not all that it should be. I think we could possibly get at this list of files by running
>
> git status --porcelain --untracked-files=no --ignored=no -- .
>
> And then your --dirty list would be lines beginning with ' M' while your --cached list would be lines beginning with 'A[ M]'
>
> Does that seem plausible?
I don't know if that works or not, but it does seem plausible, at
least. My idea would have been to use the --name-status option, which
works for both git diff and git show. You just look and see which
lines in the output start with M or A and then take the file names
from those lines.
So to indent files that are dirty, you would look at:
git diff --name-status
For what's cached:
git diff --name-status --cached
For the combination of the two:
git diff --name-status HEAD
For a prior commit:
git show --name-status $COMMITID
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-07 16:32 Jelte Fennema <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Jelte Fennema @ 2023-02-07 16:32 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Tue, 7 Feb 2023 at 17:11, Robert Haas <[email protected]> wrote:
> I don't know if that works or not, but it does seem plausible, at
> least. My idea would have been to use the --name-status option, which
> works for both git diff and git show. You just look and see which
> lines in the output start with M or A and then take the file names
> from those lines.
If you add `--diff-filter=ACMR`, then git diff/show will only show
Added, Copied, Modified, and Renamed files.
The pre-commit hook that Andrew added to the wiki uses that in
combination with --name-only to get the list of files that you want to
check on commit:
https://wiki.postgresql.org/wiki/Working_with_Git#Using_git_hooks
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-07 16:57 Robert Haas <[email protected]>
parent: Jelte Fennema <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Robert Haas @ 2023-02-07 16:57 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Tue, Feb 7, 2023 at 11:32 AM Jelte Fennema <[email protected]> wrote:
> On Tue, 7 Feb 2023 at 17:11, Robert Haas <[email protected]> wrote:
> > I don't know if that works or not, but it does seem plausible, at
> > least. My idea would have been to use the --name-status option, which
> > works for both git diff and git show. You just look and see which
> > lines in the output start with M or A and then take the file names
> > from those lines.
>
> If you add `--diff-filter=ACMR`, then git diff/show will only show
> Added, Copied, Modified, and Renamed files.
>
> The pre-commit hook that Andrew added to the wiki uses that in
> combination with --name-only to get the list of files that you want to
> check on commit:
> https://wiki.postgresql.org/wiki/Working_with_Git#Using_git_hooks
Thanks, that sounds nicer than what I suggested.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-07 17:21 Jelte Fennema <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Jelte Fennema @ 2023-02-07 17:21 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
> On Mon, Feb 6, 2023 at 10:21 AM Andrew Dunstan <[email protected]> wrote:
>
> Here's a quick patch for 1 and 3. Would also need to adjust the docco.
>
>
>
> This time with patch.
When supplying the --commit flag it still formats all files for me. I
was able to fix that by replacing:
# no non-option arguments given. so do everything in the current directory
$code_base ||= '.' unless @ARGV;
with:
# no files, dirs or commits given. so do everything in the current directory
$code_base ||= '.' unless @ARGV || @commits;
Does the code-base flag still make sense if you can simply pass a
directory as regular args now?
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-08 12:41 Andrew Dunstan <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-08 12:41 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-07 Tu 12:21, Jelte Fennema wrote:
>> On Mon, Feb 6, 2023 at 10:21 AM Andrew Dunstan<[email protected]> wrote:
>>
>> Here's a quick patch for 1 and 3. Would also need to adjust the docco.
>>
>>
>>
>> This time with patch.
> When supplying the --commit flag it still formats all files for me. I
> was able to fix that by replacing:
> # no non-option arguments given. so do everything in the current directory
> $code_base ||= '.' unless @ARGV;
>
> with:
> # no files, dirs or commits given. so do everything in the current directory
> $code_base ||= '.' unless @ARGV || @commits;
Yeah, thanks for testing. Here's a new patch with that change and the
comment adjusted.
>
> Does the code-base flag still make sense if you can simply pass a
> directory as regular args now?
Probably not. I'll look into removing it.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
Attachments:
[text/x-patch] pgindent-directory+git-enhancements-v2.patch (3.7K, ../../[email protected]/3-pgindent-directory+git-enhancements-v2.patch)
download | inline diff:
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index 56640e576a..34fb7d604d 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -23,12 +23,14 @@ my $devnull = File::Spec->devnull;
my ($typedefs_file, $typedef_str, $code_base,
@excludes, $indent, $build,
- $show_diff, $silent_diff, $help);
+ $show_diff, $silent_diff, $help,
+ @commits,);
$help = 0;
my %options = (
"help" => \$help,
+ "commit=s" => \@commits,
"typedefs=s" => \$typedefs_file,
"list-of-typedefs=s" => \$typedef_str,
"code-base=s" => \$code_base,
@@ -44,6 +46,9 @@ usage() if $help;
usage("Cannot have both --silent-diff and --show-diff")
if $silent_diff && $show_diff;
+usage("Cannot use --commit with --code-base or command line file list")
+ if (@commits && ($code_base || @ARGV));
+
run_build($code_base) if ($build);
# command line option wins, then environment (which is how --build sets it) ,
@@ -53,8 +58,9 @@ $typedefs_file ||= $ENV{PGTYPEDEFS};
# build mode sets PGINDENT
$indent ||= $ENV{PGINDENT} || $ENV{INDENT} || "pg_bsd_indent";
-# no non-option arguments given. so do everything in the current directory
-$code_base ||= '.' unless @ARGV;
+# if no non-option arguments or commits are given, default to looking in the
+# current directory
+$code_base ||= '.' unless (@ARGV || @commits);
my $sourcedir = locate_sourcedir();
@@ -388,6 +394,7 @@ Usage:
pgindent [OPTION]... [FILE]...
Options:
--help show this message and quit
+ --commit=gitref use files modified by the named commit
--typedefs=FILE file containing a list of typedefs
--list-of-typedefs=STR string containing typedefs, space separated
--code-base=DIR path to the base of PostgreSQL source code
@@ -396,7 +403,7 @@ Options:
--build build the pg_bsd_indent program
--show-diff show the changes that would be made
--silent-diff exit with status 2 if any changes would be made
-The --excludes option can be given more than once.
+The --excludes and --commit options can be given more than once.
EOF
if ($help)
{
@@ -412,27 +419,38 @@ EOF
# main
-# get the list of files under code base, if it's set
-File::Find::find(
- {
- wanted => sub {
- my ($dev, $ino, $mode, $nlink, $uid, $gid);
- (($dev, $ino, $mode, $nlink, $uid, $gid) = lstat($_))
- && -f _
- && /^.*\.[ch]\z/s
- && push(@files, $File::Find::name);
- }
- },
- $code_base) if $code_base;
-
$filtered_typedefs_fh = load_typedefs();
check_indent();
-# any non-option arguments are files to be processed
-push(@files, @ARGV);
+build_clean($code_base) if $build;
-# the exclude list applies to command line arguments as well as found files
+my $wanted = sub
+{
+ my ($dev, $ino, $mode, $nlink, $uid, $gid);
+ (($dev, $ino, $mode, $nlink, $uid, $gid) = lstat($_))
+ && -f _
+ && /^.*\.[ch]\z/s
+ && push(@files, $File::Find::name);
+};
+
+# get the list of files under code base, if it's set
+File::Find::find({wanted => $wanted }, $code_base) if $code_base;
+
+# any non-option arguments are files or directories to be processed
+File::Find::find({wanted => $wanted}, @ARGV) if @ARGV;
+
+# process named commits by comparing each with their immediate ancestor
+foreach my $commit (@commits)
+{
+ my $prev="$commit~";
+ my @affected=`git diff-tree --no-commit-id --name-only -r $commit $prev`;
+ die "git error" if $?;
+ chomp(@affected);
+ push(@files,@affected);
+}
+
+# remove excluded files from the file list
process_exclude();
foreach my $source_filename (@files)
@@ -481,6 +499,4 @@ foreach my $source_filename (@files)
}
}
-build_clean($code_base) if $build;
-
exit 0;
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-08 13:27 Andrew Dunstan <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-08 13:27 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-08 We 07:41, Andrew Dunstan wrote:
>
>
> On 2023-02-07 Tu 12:21, Jelte Fennema wrote:
>
>
>> Does the code-base flag still make sense if you can simply pass a
>> directory as regular args now?
>
>
> Probably not. I'll look into removing it.
>
>
>
What we should probably do is remove all the build stuff along with
$code_base. It dates back to the time when I developed this as an out of
tree replacement for the old pgindent, and is just basically wasted
space now. After I get done with the current round of enhancements I'll
reorganize the script and get rid of things we don't need any more.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-08 17:06 Jelte Fennema <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Jelte Fennema @ 2023-02-08 17:06 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
With the new patch --commit works as expected for me now. And sounds
good to up the script a bit afterwards.
On Wed, 8 Feb 2023 at 14:27, Andrew Dunstan <[email protected]> wrote:
>
>
> On 2023-02-08 We 07:41, Andrew Dunstan wrote:
>
>
> On 2023-02-07 Tu 12:21, Jelte Fennema wrote:
>
>
> Does the code-base flag still make sense if you can simply pass a
> directory as regular args now?
>
>
> Probably not. I'll look into removing it.
>
>
>
>
> What we should probably do is remove all the build stuff along with $code_base. It dates back to the time when I developed this as an out of tree replacement for the old pgindent, and is just basically wasted space now. After I get done with the current round of enhancements I'll reorganize the script and get rid of things we don't need any more.
>
>
> cheers
>
>
> andrew
>
> --
> Andrew Dunstan
> EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-08 22:09 Andrew Dunstan <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 3 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-08 22:09 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-08 We 12:06, Jelte Fennema wrote:
> With the new patch --commit works as expected for me now. And sounds
> good to up the script a bit afterwards.
>
>
Thanks, I have committed this. Still looking at Robert's other request.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* RE: run pgindent on a regular basis / scripted manner
@ 2023-02-09 02:29 Shinoda, Noriyoshi (PN Japan FSIP) <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: Shinoda, Noriyoshi (PN Japan FSIP) @ 2023-02-09 02:29 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Hi,
I tried the committed pgindent.
The attached small patch changes spaces in the usage message to tabs.
Options other than --commit start with a tab.
Regards,
Noriyoshi Shinoda
From: Andrew Dunstan <[email protected]>
Sent: Thursday, February 9, 2023 7:10 AM
To: Jelte Fennema <[email protected]>
Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; [email protected]
Subject: Re: run pgindent on a regular basis / scripted manner
On 2023-02-08 We 12:06, Jelte Fennema wrote:
With the new patch --commit works as expected for me now. And sounds
good to up the script a bit afterwards.
Thanks, I have committed this. Still looking at Robert's other request.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com<https://www.enterprisedb.com;
Attachments:
[application/octet-stream] pgindent_usage_v1.diff (630B, ../../DM4PR84MB1734DE48B1800024DABE9A57EED99@DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM/3-pgindent_usage_v1.diff)
download | inline diff:
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index 34fb7d604d..07970f758c 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -394,7 +394,7 @@ Usage:
pgindent [OPTION]... [FILE]...
Options:
--help show this message and quit
- --commit=gitref use files modified by the named commit
+ --commit=gitref use files modified by the named commit
--typedefs=FILE file containing a list of typedefs
--list-of-typedefs=STR string containing typedefs, space separated
--code-base=DIR path to the base of PostgreSQL source code
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-09 18:34 Andrew Dunstan <[email protected]>
parent: Shinoda, Noriyoshi (PN Japan FSIP) <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-09 18:34 UTC (permalink / raw)
To: Shinoda, Noriyoshi (PN Japan FSIP) <[email protected]>; Jelte Fennema <[email protected]>; Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-08 We 21:29, Shinoda, Noriyoshi (PN Japan FSIP) wrote:
>
> Hi,
> I tried the committed pgindent.
> The attached small patch changes spaces in the usage message to tabs.
> Options other than --commit start with a tab.
>
Thanks, pushed.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* RE: run pgindent on a regular basis / scripted manner
@ 2023-02-10 02:37 [email protected] <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: [email protected] @ 2023-02-10 02:37 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Thu, Feb 9, 2023 6:10 AM Andrew Dunstan <[email protected]> wrote:
> Thanks, I have committed this. Still looking at Robert's other request.
>
Hi,
I tried the new option --commit and found that it seems to try to indent files
which are deleted in the specified commit and reports an error.
cannot open file "src/backend/access/brin/brin.c": No such file or directory
It looks we should filter such files.
Regards,
Shi Yu
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-10 09:25 Jelte Fennema <[email protected]>
parent: [email protected] <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Jelte Fennema @ 2023-02-10 09:25 UTC (permalink / raw)
To: [email protected] <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Ah yes, I had seen that when I read the initial --commit patch but
then forgot about it when the flag didn't work at all when I tried it.
Attached is a patch that fixes the issue. And also implements the
--dirty and --staged flags in pgindent that Robert Haas requested.
On Fri, 10 Feb 2023 at 03:37, [email protected]
<[email protected]> wrote:
>
> On Thu, Feb 9, 2023 6:10 AM Andrew Dunstan <[email protected]> wrote:
> > Thanks, I have committed this. Still looking at Robert's other request.
> >
>
> Hi,
>
> I tried the new option --commit and found that it seems to try to indent files
> which are deleted in the specified commit and reports an error.
>
> cannot open file "src/backend/access/brin/brin.c": No such file or directory
>
> It looks we should filter such files.
>
> Regards,
> Shi Yu
Attachments:
[application/octet-stream] 0001-Fix-pgindent-commit-flag-and-add-dirty-and-staged-fl.patch (2.8K, ../../CAGECzQT44xN-zcTQ0+OxLpWa9nONxmEaYxQpaWM+My+A4o72PA@mail.gmail.com/2-0001-Fix-pgindent-commit-flag-and-add-dirty-and-staged-fl.patch)
download | inline diff:
From 72b448450142ac788c686e9bd96d5b604649a169 Mon Sep 17 00:00:00 2001
From: Jelte Fennema <[email protected]>
Date: Fri, 10 Feb 2023 10:14:07 +0100
Subject: [PATCH] Fix pgindent commit flag and add dirty and staged flags
Fixes a bug in --commit where pgindent would try to indent deleted
files. Also implement a --staged and a --dirty flag, which were
requested by Robert Haas.
---
src/tools/pgindent/pgindent | 26 +++++++++++++++++++++++---
1 file changed, 23 insertions(+), 3 deletions(-)
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index 07970f758c2..9245e06d04b 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -24,7 +24,7 @@ my $devnull = File::Spec->devnull;
my ($typedefs_file, $typedef_str, $code_base,
@excludes, $indent, $build,
$show_diff, $silent_diff, $help,
- @commits,);
+ @commits, $dirty, $staged);
$help = 0;
@@ -37,6 +37,8 @@ my %options = (
"excludes=s" => \@excludes,
"indent=s" => \$indent,
"build" => \$build,
+ "dirty" => \$dirty,
+ "staged" => \$staged,
"show-diff" => \$show_diff,
"silent-diff" => \$silent_diff,);
GetOptions(%options) || usage("bad command line argument");
@@ -60,7 +62,7 @@ $indent ||= $ENV{PGINDENT} || $ENV{INDENT} || "pg_bsd_indent";
# if no non-option arguments or commits are given, default to looking in the
# current directory
-$code_base ||= '.' unless (@ARGV || @commits);
+$code_base ||= '.' unless (@ARGV || @commits || $dirty || $staged);
my $sourcedir = locate_sourcedir();
@@ -401,6 +403,8 @@ Options:
--excludes=PATH file containing list of filename patterns to ignore
--indent=PATH path to pg_bsd_indent program
--build build the pg_bsd_indent program
+ --staged use files that are staged for commit
+ --dirty use files that are modified (but not staged)
--show-diff show the changes that would be made
--silent-diff exit with status 2 if any changes would be made
The --excludes and --commit options can be given more than once.
@@ -444,7 +448,23 @@ File::Find::find({wanted => $wanted}, @ARGV) if @ARGV;
foreach my $commit (@commits)
{
my $prev="$commit~";
- my @affected=`git diff-tree --no-commit-id --name-only -r $commit $prev`;
+ my @affected=`git diff --diff-filter=ACMR --name-only $prev $commit`;
+ die "git error" if $?;
+ chomp(@affected);
+ push(@files,@affected);
+}
+
+if ($staged)
+{
+ my @affected=`git diff --diff-filter=ACMR --name-only --staged`;
+ die "git error" if $?;
+ chomp(@affected);
+ push(@files,@affected);
+}
+
+if ($dirty)
+{
+ my @affected=`git diff --diff-filter=ACMR --name-only`;
die "git error" if $?;
chomp(@affected);
push(@files,@affected);
--
2.34.1
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-10 14:26 Andrew Dunstan <[email protected]>
parent: Jelte Fennema <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-10 14:26 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-07 Tu 11:32, Jelte Fennema wrote:
> On Tue, 7 Feb 2023 at 17:11, Robert Haas<[email protected]> wrote:
>> I don't know if that works or not, but it does seem plausible, at
>> least. My idea would have been to use the --name-status option, which
>> works for both git diff and git show. You just look and see which
>> lines in the output start with M or A and then take the file names
>> from those lines.
> If you add `--diff-filter=ACMR`, then git diff/show will only show
> Added, Copied, Modified, and Renamed files.
>
> The pre-commit hook that Andrew added to the wiki uses that in
> combination with --name-only to get the list of files that you want to
> check on commit:
> https://wiki.postgresql.org/wiki/Working_with_Git#Using_git_hooks
OK, here's a patch based on Robert's and Jelte's ideas.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
Attachments:
[text/x-patch] pgindent-dirty-cached.patch (2.2K, ../../[email protected]/3-pgindent-dirty-cached.patch)
download | inline diff:
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index 07970f758c..c9f42302c9 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -24,13 +24,15 @@ my $devnull = File::Spec->devnull;
my ($typedefs_file, $typedef_str, $code_base,
@excludes, $indent, $build,
$show_diff, $silent_diff, $help,
- @commits,);
+ @commits, $dirty, $cached);
$help = 0;
my %options = (
"help" => \$help,
"commit=s" => \@commits,
+ "dirty" => \$dirty,
+ "cached" => \$cached,
"typedefs=s" => \$typedefs_file,
"list-of-typedefs=s" => \$typedef_str,
"code-base=s" => \$code_base,
@@ -46,8 +48,11 @@ usage() if $help;
usage("Cannot have both --silent-diff and --show-diff")
if $silent_diff && $show_diff;
-usage("Cannot use --commit with --code-base or command line file list")
- if (@commits && ($code_base || @ARGV));
+usage("Cannot use --commit, --dirty or --cached with --code-base or command line file list")
+ if (($dirty || $cached || @commits) && ($code_base || @ARGV));
+
+usage( "Cannot use --dirty or --cached with --commit")
+ if (($dirty || $cached) && @commits);
run_build($code_base) if ($build);
@@ -395,6 +400,8 @@ pgindent [OPTION]... [FILE]...
Options:
--help show this message and quit
--commit=gitref use files modified by the named commit
+ --dirty use modified files under current directory
+ --cached use staged files under current directory
--typedefs=FILE file containing a list of typedefs
--list-of-typedefs=STR string containing typedefs, space separated
--code-base=DIR path to the base of PostgreSQL source code
@@ -450,6 +457,19 @@ foreach my $commit (@commits)
push(@files,@affected);
}
+# look for dirty / cached files under the CWD
+if ($dirty || $cached)
+{
+ my $gitcmd = "git diff --diff-filter=ACMR --name-only";
+ $gitcmd .= " --cached" if ($cached && !$dirty);
+ $gitcmd .= " HEAD" if ($dirty && $cached);
+ $gitcmd .= " -- .";
+ my @affected=`$gitcmd`;
+ die "git error" if $?;
+ chomp(@affected);
+ push(@files,@affected);
+}
+
# remove excluded files from the file list
process_exclude();
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-10 15:21 Andrew Dunstan <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-10 15:21 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-10 Fr 04:25, Jelte Fennema wrote:
> Ah yes, I had seen that when I read the initial --commit patch but
> then forgot about it when the flag didn't work at all when I tried it.
>
> Attached is a patch that fixes the issue. And also implements the
> --dirty and --staged flags in pgindent that Robert Haas requested.
[please don't top-post]
I don't think just adding a diff filter is really a sufficient fix. The
file might have been deleted since the commit(s) in question. Here's a
more general fix for missing files.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
Attachments:
[text/x-patch] pgindent-missing-file-fix.patch (1018B, ../../[email protected]/3-pgindent-missing-file-fix.patch)
download | inline diff:
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index 07970f758c..64284e7ea2 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -444,7 +444,7 @@ File::Find::find({wanted => $wanted}, @ARGV) if @ARGV;
foreach my $commit (@commits)
{
my $prev="$commit~";
- my @affected=`git diff-tree --no-commit-id --name-only -r $commit $prev`;
+ my @affected=`git diff --diff-filter=ACMR --name-only $prev $commit`;
die "git error" if $?;
chomp(@affected);
push(@files,@affected);
@@ -458,6 +458,12 @@ foreach my $source_filename (@files)
# ignore anything that's not a .c or .h file
next unless $source_filename =~ /\.[ch]$/;
+ # don't try to indent a file that doesn't exist
+ unless (-f $source_filename)
+ {
+ warn "Could not find $source_filename";
+ next;
+ }
# Automatically ignore .c and .h files that correspond to a .y or .l
# file. indent tends to get badly confused by Bison/flex output,
# and there's no value in indenting derived files anyway.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-12 14:16 Andrew Dunstan <[email protected]>
parent: Andrew Dunstan <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-12 14:16 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-10 Fr 10:21, Andrew Dunstan wrote:
>
>
> On 2023-02-10 Fr 04:25, Jelte Fennema wrote:
>> Ah yes, I had seen that when I read the initial --commit patch but
>> then forgot about it when the flag didn't work at all when I tried it.
>>
>> Attached is a patch that fixes the issue. And also implements the
>> --dirty and --staged flags in pgindent that Robert Haas requested.
>
>
>
> I don't think just adding a diff filter is really a sufficient fix.
> The file might have been deleted since the commit(s) in question.
> Here's a more general fix for missing files.
>
OK, I've pushed this along with a check to make sure we only process
each file once.
I'm not sure how much more I really want to do here. Given the way
pgindent now processes command line arguments, maybe the best thing is
for people to use that. Use of git aliases can help. Something like
these for example
[alias]
dirty = diff --name-only --diff-filter=ACMU -- .
staged = diff --name-only --cached --diff-filter=ACMU -- .
dstaged = diff --name-only --diff-filter=ACMU HEAD -- .
and then you could do
pgindent `git dirty`
The only danger would be if there were no dirty files. Maybe we need a
switch to inhibit using the current directory if there are no command
line files.
Thoughts?
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-12 16:24 Tom Lane <[email protected]>
parent: Andrew Dunstan <[email protected]>
1 sibling, 2 replies; 142+ messages in thread
From: Tom Lane @ 2023-02-12 16:24 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Andrew Dunstan <[email protected]> writes:
> ... then you could do
> pgindent `git dirty`
> The only danger would be if there were no dirty files. Maybe we need a
> switch to inhibit using the current directory if there are no command
> line files.
It seems like "indent the whole tree" is about to become a minority
use-case. Maybe instead of continuing to privilege that case, we
should say that it's invoked by some new switch like --all-files,
and without that only the stuff identified by command-line arguments
gets processed.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-12 20:41 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-12 20:41 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-12 Su 11:24, Tom Lane wrote:
> Andrew Dunstan<[email protected]> writes:
>> ... then you could do
>> pgindent `git dirty`
>> The only danger would be if there were no dirty files. Maybe we need a
>> switch to inhibit using the current directory if there are no command
>> line files.
> It seems like "indent the whole tree" is about to become a minority
> use-case. Maybe instead of continuing to privilege that case, we
> should say that it's invoked by some new switch like --all-files,
> and without that only the stuff identified by command-line arguments
> gets processed.
>
>
I don't think we need --all-files. The attached gets rid of the build
and code-base cruft, which is now in any case obsolete given we've put
pg_bsd_indent in our code base. So the way to spell this instead of
"pgindent --all-files" would be "pgindent ."
I added a warning if there are no files at all specified.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
Attachments:
[text/x-patch] pgindent-remove-build-and-code-base.patch (6.6K, ../../[email protected]/3-pgindent-remove-build-and-code-base.patch)
download | inline diff:
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index a793971e07..b098cef02a 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -21,7 +21,7 @@ my $indent_opts =
my $devnull = File::Spec->devnull;
-my ($typedefs_file, $typedef_str, $code_base,
+my ($typedefs_file, $typedef_str,
@excludes, $indent, $build,
$show_diff, $silent_diff, $help,
@commits,);
@@ -33,10 +33,8 @@ my %options = (
"commit=s" => \@commits,
"typedefs=s" => \$typedefs_file,
"list-of-typedefs=s" => \$typedef_str,
- "code-base=s" => \$code_base,
"excludes=s" => \@excludes,
"indent=s" => \$indent,
- "build" => \$build,
"show-diff" => \$show_diff,
"silent-diff" => \$silent_diff,);
GetOptions(%options) || usage("bad command line argument");
@@ -46,22 +44,16 @@ usage() if $help;
usage("Cannot have both --silent-diff and --show-diff")
if $silent_diff && $show_diff;
-usage("Cannot use --commit with --code-base or command line file list")
- if (@commits && ($code_base || @ARGV));
+usage("Cannot use --commit with command line file list")
+ if (@commits && @ARGV);
-run_build($code_base) if ($build);
-
-# command line option wins, then environment (which is how --build sets it) ,
-# then locations. based on current dir, then default location
+# command line option wins, then environment, then locations based on current
+# dir, then default location
$typedefs_file ||= $ENV{PGTYPEDEFS};
-# build mode sets PGINDENT
+# get indent location for environment or default
$indent ||= $ENV{PGINDENT} || $ENV{INDENT} || "pg_bsd_indent";
-# if no non-option arguments or commits are given, default to looking in the
-# current directory
-$code_base ||= '.' unless (@ARGV || @commits);
-
my $sourcedir = locate_sourcedir();
# if it's the base of a postgres tree, we will exclude the files
@@ -121,8 +113,7 @@ sub check_indent
sub locate_sourcedir
{
# try fairly hard to locate the sourcedir
- my $where = $code_base || '.';
- my $sub = "$where/src/tools/pgindent";
+ my $sub = "./src/tools/pgindent";
return $sub if -d $sub;
# try to find it from an ancestor directory
$sub = "../src/tools/pgindent";
@@ -320,72 +311,6 @@ sub show_diff
return $diff;
}
-sub run_build
-{
- eval "use LWP::Simple;"; ## no critic (ProhibitStringyEval);
-
- my $code_base = shift || '.';
- my $save_dir = getcwd();
-
- # look for the code root
- foreach (1 .. 5)
- {
- last if -d "$code_base/src/tools/pgindent";
- $code_base = "$code_base/..";
- }
-
- die "cannot locate src/tools/pgindent directory in \"$code_base\"\n"
- unless -d "$code_base/src/tools/pgindent";
-
- chdir "$code_base/src/tools/pgindent";
-
- my $typedefs_list_url =
- "https://buildfarm.postgresql.org/cgi-bin/typedefs.pl";
-
- my $rv = getstore($typedefs_list_url, "tmp_typedefs.list");
-
- die "cannot fetch typedefs list from $typedefs_list_url\n"
- unless is_success($rv);
-
- $ENV{PGTYPEDEFS} = abs_path('tmp_typedefs.list');
-
- my $indentrepo = "https://git.postgresql.org/git/pg_bsd_indent.git";
- system("git clone $indentrepo >$devnull 2>&1");
- die "could not fetch pg_bsd_indent sources from $indentrepo\n"
- unless $? == 0;
-
- chdir "pg_bsd_indent" || die;
- system("make all check >$devnull");
- die "could not build pg_bsd_indent from source\n"
- unless $? == 0;
-
- $ENV{PGINDENT} = abs_path('pg_bsd_indent');
-
- chdir $save_dir;
- return;
-}
-
-sub build_clean
-{
- my $code_base = shift || '.';
-
- # look for the code root
- foreach (1 .. 5)
- {
- last if -d "$code_base/src/tools/pgindent";
- $code_base = "$code_base/..";
- }
-
- die "cannot locate src/tools/pgindent directory in \"$code_base\"\n"
- unless -d "$code_base/src/tools/pgindent";
-
- chdir "$code_base";
-
- system("rm -rf src/tools/pgindent/pg_bsd_indent");
- system("rm -f src/tools/pgindent/tmp_typedefs.list");
- return;
-}
-
sub usage
{
my $message = shift;
@@ -397,10 +322,8 @@ Options:
--commit=gitref use files modified by the named commit
--typedefs=FILE file containing a list of typedefs
--list-of-typedefs=STR string containing typedefs, space separated
- --code-base=DIR path to the base of PostgreSQL source code
--excludes=PATH file containing list of filename patterns to ignore
--indent=PATH path to pg_bsd_indent program
- --build build the pg_bsd_indent program
--show-diff show the changes that would be made
--silent-diff exit with status 2 if any changes would be made
The --excludes and --commit options can be given more than once.
@@ -423,8 +346,6 @@ $filtered_typedefs_fh = load_typedefs();
check_indent();
-build_clean($code_base) if $build;
-
my $wanted = sub
{
my ($dev, $ino, $mode, $nlink, $uid, $gid);
@@ -434,9 +355,6 @@ my $wanted = sub
&& push(@files, $File::Find::name);
};
-# get the list of files under code base, if it's set
-File::Find::find({wanted => $wanted }, $code_base) if $code_base;
-
# any non-option arguments are files or directories to be processed
File::Find::find({wanted => $wanted}, @ARGV) if @ARGV;
@@ -450,6 +368,8 @@ foreach my $commit (@commits)
push(@files,@affected);
}
+warn "No files to process" unless @files;
+
# remove excluded files from the file list
process_exclude();
diff --git a/src/tools/pgindent/pgindent.man b/src/tools/pgindent/pgindent.man
index 7406794ba3..fe411ee699 100644
--- a/src/tools/pgindent/pgindent.man
+++ b/src/tools/pgindent/pgindent.man
@@ -8,14 +8,9 @@ You can see all the options by running:
pgindent --help
In its simplest form, if all the required objects are installed, simply run
-it without any parameters at the top of the source tree you want to process.
+it at the top of the source tree you want to process like this:
- pgindent
-
-If you don't have all the requirements installed, pgindent will fetch and build
-them for you, if you're in a PostgreSQL source tree:
-
- pgindent --build
+ pgindent .
If your pg_bsd_indent program is not installed in your path, you can specify
it by setting the environment variable INDENT, or PGINDENT, or by giving the
@@ -28,9 +23,6 @@ specified using the PGTYPEDEFS environment variable, or via the command line
--typedefs option. If neither is used, it will look for it within the
current source tree, or in /usr/local/etc/typedefs.list.
-If you want to indent a source tree other than the current working directory,
-you can specify it via the --code-base command line option.
-
We don't want to indent certain files in the PostgreSQL source. pgindent
will honor a file containing a list of patterns of files to avoid. This
file can be specified using the --excludes command line option. If indenting
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-12 20:59 Justin Pryzby <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Justin Pryzby @ 2023-02-12 20:59 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; [email protected]; Robert Haas <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Sun, Feb 12, 2023 at 11:24:14AM -0500, Tom Lane wrote:
> Andrew Dunstan <[email protected]> writes:
> > ... then you could do
> > pgindent `git dirty`
> > The only danger would be if there were no dirty files. Maybe we need a
> > switch to inhibit using the current directory if there are no command
> > line files.
>
> It seems like "indent the whole tree" is about to become a minority
> use-case. Maybe instead of continuing to privilege that case, we
> should say that it's invoked by some new switch like --all-files,
> and without that only the stuff identified by command-line arguments
> gets processed.
It seems like if pgindent knows about git, it ought to process only
tracked files. Then, it wouldn't need to manually exclude generated
files, and it wouldn't process vpath builds and who-knows-what else it
finds in CWD.
At least --commit doesn't seem to work when run outside of the root
source dir.
--
Justin
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-12 21:13 Tom Lane <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-02-12 21:13 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Andrew Dunstan <[email protected]> writes:
> On 2023-02-12 Su 11:24, Tom Lane wrote:
>> It seems like "indent the whole tree" is about to become a minority
>> use-case. Maybe instead of continuing to privilege that case, we
>> should say that it's invoked by some new switch like --all-files,
>> and without that only the stuff identified by command-line arguments
>> gets processed.
> I don't think we need --all-files. The attached gets rid of the build
> and code-base cruft, which is now in any case obsolete given we've put
> pg_bsd_indent in our code base. So the way to spell this instead of
> "pgindent --all-files" would be "pgindent ."
Ah, of course.
> I added a warning if there are no files at all specified.
LGTM.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-13 12:57 Andrew Dunstan <[email protected]>
parent: Justin Pryzby <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-13 12:57 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; Tom Lane <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected]; Robert Haas <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-12 Su 15:59, Justin Pryzby wrote:
> It seems like if pgindent knows about git, it ought to process only
> tracked files. Then, it wouldn't need to manually exclude generated
> files, and it wouldn't process vpath builds and who-knows-what else it
> finds in CWD.
for vpath builds use an exclude file that excludes the vpath you use.
I don't really want restrict this to tracked files because it would mean
you can't pgindent files before you `git add` them. And we would still
need to do manual exclusion for some files that are tracked, e.g. the
snowball files.
>
> At least --commit doesn't seem to work when run outside of the root
> source dir.
>
Yeah, I'll fix that, thanks for mentioning.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-13 13:27 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-13 13:27 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-12 Su 16:13, Tom Lane wrote:
> Andrew Dunstan<[email protected]> writes:
>> On 2023-02-12 Su 11:24, Tom Lane wrote:
>>> It seems like "indent the whole tree" is about to become a minority
>>> use-case. Maybe instead of continuing to privilege that case, we
>>> should say that it's invoked by some new switch like --all-files,
>>> and without that only the stuff identified by command-line arguments
>>> gets processed.
>> I don't think we need --all-files. The attached gets rid of the build
>> and code-base cruft, which is now in any case obsolete given we've put
>> pg_bsd_indent in our code base. So the way to spell this instead of
>> "pgindent --all-files" would be "pgindent ."
> Ah, of course.
>
>> I added a warning if there are no files at all specified.
> LGTM.
Thanks, pushed.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-13 14:02 Jelte Fennema <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Jelte Fennema @ 2023-02-13 14:02 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Sun, 12 Feb 2023 at 15:16, Andrew Dunstan <[email protected]> wrote:
> I'm not sure how much more I really want to do here. Given the way pgindent now processes command line arguments, maybe the best thing is for people to use that. Use of git aliases can help. Something like these for example
>
>
> [alias]
>
> dirty = diff --name-only --diff-filter=ACMU -- .
> staged = diff --name-only --cached --diff-filter=ACMU -- .
> dstaged = diff --name-only --diff-filter=ACMU HEAD -- .
>
>
> and then you could do
>
> pgindent `git dirty`
>
>
> The only danger would be if there were no dirty files. Maybe we need a switch to inhibit using the current directory if there are no command line files.
>
>
> Thoughts?
I think indenting staged or dirty files is probably the most common
operation that people want to do with pgindent. So I think that having
dedicated flags makes sense. I agree that it's not strictly necessary
and git aliases help a lot. But the git aliases require you to set
them up. To me making the most common operation as easy as possible to
do, seems worth the few extra lines to pgindent.
Sidenote: You mentioned untracked files in another email. I think that
the --dirty flag should probably also include untracked files. A
command to do so is: git ls-files --others --exclude-standard
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-13 16:46 Andrew Dunstan <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-13 16:46 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-13 Mo 09:02, Jelte Fennema wrote:
> On Sun, 12 Feb 2023 at 15:16, Andrew Dunstan<[email protected]> wrote:
>> I'm not sure how much more I really want to do here. Given the way pgindent now processes command line arguments, maybe the best thing is for people to use that. Use of git aliases can help. Something like these for example
>>
>>
>> [alias]
>>
>> dirty = diff --name-only --diff-filter=ACMU -- .
>> staged = diff --name-only --cached --diff-filter=ACMU -- .
>> dstaged = diff --name-only --diff-filter=ACMU HEAD -- .
>>
>>
>> and then you could do
>>
>> pgindent `git dirty`
>>
>>
>> The only danger would be if there were no dirty files. Maybe we need a switch to inhibit using the current directory if there are no command line files.
>>
>>
>> Thoughts?
> I think indenting staged or dirty files is probably the most common
> operation that people want to do with pgindent. So I think that having
> dedicated flags makes sense. I agree that it's not strictly necessary
> and git aliases help a lot. But the git aliases require you to set
> them up. To me making the most common operation as easy as possible to
> do, seems worth the few extra lines to pgindent.
OK, but I'd like to hear from more people about what they want.
Experience tells me that making assumptions about how people work is not
a good idea. I doubt anyone's work pattern is like mine. I don't want to
implement an option that three people are going to use.
>
> Sidenote: You mentioned untracked files in another email. I think that
> the --dirty flag should probably also include untracked files. A
> command to do so is: git ls-files --others --exclude-standard
Thanks for the info.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-13 18:29 Jelte Fennema <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Jelte Fennema @ 2023-02-13 18:29 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Mon, 13 Feb 2023 at 17:47, Andrew Dunstan <[email protected]> wrote:
> OK, but I'd like to hear from more people about what they want. Experience tells me that making assumptions about how people work is not a good idea. I doubt anyone's work pattern is like mine. I don't want to implement an option that three people are going to use.
In the general case I agree with you. But in this specific case I
don't. To me the whole point of this email thread is to nudge people
towards indenting the changes that they are committing. Thus indenting
those changes (either before or after adding) is the workflow that we
want to make as easy as possible. Because even if it's not people
their current workflow, by adding the flag it hopefully becomes their
workflow, because it's so easy to use. So my point is we want to
remove as few hurdles as possible for people to indent their changes
(and setting up git aliases or pre-commit hooks are all hurdles).
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-15 18:45 Justin Pryzby <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Justin Pryzby @ 2023-02-15 18:45 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Tom Lane <[email protected]>; Jelte Fennema <[email protected]>; [email protected]; Robert Haas <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Mon, Feb 13, 2023 at 07:57:25AM -0500, Andrew Dunstan wrote:
>
> On 2023-02-12 Su 15:59, Justin Pryzby wrote:
> > It seems like if pgindent knows about git, it ought to process only
> > tracked files. Then, it wouldn't need to manually exclude generated
> > files, and it wouldn't process vpath builds and who-knows-what else it
> > finds in CWD.
>
> I don't really want restrict this to tracked files because it would mean you
> can't pgindent files before you `git add` them.
I think you'd allow indenting files which were either tracked *or*
specified on the command line.
Also, it makes a more sense to "add" the file before indenting it, to
allow checking the output and remove unrelated changes. So that doesn't
seem to me like a restriction of any significance.
But I would never want to indent an untracked file unless I specified
it.
--
Justin
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-15 21:00 Jelte Fennema <[email protected]>
parent: Justin Pryzby <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Jelte Fennema @ 2023-02-15 21:00 UTC (permalink / raw)
To: Justin Pryzby <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Tom Lane <[email protected]>; [email protected]; Robert Haas <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
> Also, it makes a more sense to "add" the file before indenting it, to
> allow checking the output and remove unrelated changes. So that doesn't
> seem to me like a restriction of any significance.
For my workflow it would be the same, but afaik there's two ways that
people commonly use git (mine is 1):
1. Adding changes/files to the staging area using and then committing
those changes:
git add (-p)/emacs magit/some other editor integration
2. Just add everything that's changed and commit all of it:
git add -A + git commit/git commit -a
For workflow 1, a --staged/--cached flag would be enough IMHO. But
that's not at all helpful for workflow 2. That's why I proposed
--uncommitted too, to make indenting easier for workflow 2.
> But I would never want to indent an untracked file unless I specified
> it.
Would the --uncommitted flag I proposed be enough of an explicit way
of specifying that you want to indent untracked files?
^ permalink raw reply [nested|flat] 142+ messages in thread
* RE: run pgindent on a regular basis / scripted manner
@ 2023-02-16 08:26 [email protected] <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: [email protected] @ 2023-02-16 08:26 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Thu, Feb 9, 2023 6:10 AM Andrew Dunstan <[email protected]> wrote:
> Thanks, I have committed this. Still looking at Robert's other request.
>
Hi,
Commit #068a243b7 supported directories to be non-option arguments of pgindent.
But the help text doesn't mention that. Should we update it? Attach a small
patch which did that.
Regards,
Shi Yu
Attachments:
[application/octet-stream] v1-0001-Update-help-text-of-pgindent.patch (755B, ../../OSZPR01MB63100E1C06045FB66B16728BFDA09@OSZPR01MB6310.jpnprd01.prod.outlook.com/2-v1-0001-Update-help-text-of-pgindent.patch)
download | inline diff:
From 4782ac969a002d64e55083a526d9466d491846ad Mon Sep 17 00:00:00 2001
From: Shi Yu <[email protected]>
Date: Thu, 16 Feb 2023 16:27:04 +0800
Subject: [PATCH v1] Update help text of pgindent
---
src/tools/pgindent/pgindent | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index 8a2d39c4f6..3c48f3c46b 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -316,7 +316,7 @@ sub usage
my $message = shift;
my $helptext = <<'EOF';
Usage:
-pgindent [OPTION]... [FILE]...
+pgindent [OPTION]... [FILE|DIR]...
Options:
--help show this message and quit
--commit=gitref use files modified by the named commit
--
2.31.1
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-02-16 16:44 Andrew Dunstan <[email protected]>
parent: [email protected] <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-02-16 16:44 UTC (permalink / raw)
To: [email protected] <[email protected]>; Jelte Fennema <[email protected]>; +Cc: Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-02-16 Th 03:26, [email protected] wrote:
> On Thu, Feb 9, 2023 6:10 AM Andrew Dunstan<[email protected]> wrote:
>> Thanks, I have committed this. Still looking at Robert's other request.
>>
> Hi,
>
> Commit #068a243b7 supported directories to be non-option arguments of pgindent.
> But the help text doesn't mention that. Should we update it? Attach a small
> patch which did that.
>
Thanks, pushed.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-21 07:58 Jelte Fennema <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Jelte Fennema @ 2023-04-21 07:58 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Now that the PG16 feature freeze happened I think it's time to bump
this thread again. As far as I remember everyone that responded (even
previously silent people) were themselves proponents of being more
strict around pgindent.
I think there's two things needed to actually start doing this:
1. We need to reindent the tree to create an indented baseline
2. We need some automation to complain about unindented code being committed
For 2 the upstream thread listed two approaches:
a. Install a pre-receive git hook on the git server that rejects
pushes to master that are not indented
b. Add a test suite that checks if the code is correctly indented, so
the build farm would complain about it. (Suggested by Peter E)
I think both a and b would work to achieve 2. But as Peter E said, b
indeed sounds like less of a divergence of the status quo. So my vote
would be for b. If that doesn't achieve 2 for some reason, or turns
out to have problems we can always change to a afterwards.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 08:50 Michael Paquier <[email protected]>
parent: Jelte Fennema <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Michael Paquier @ 2023-04-22 08:50 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote:
> For 2 the upstream thread listed two approaches:
> a. Install a pre-receive git hook on the git server that rejects
> pushes to master that are not indented
> b. Add a test suite that checks if the code is correctly indented, so
> the build farm would complain about it. (Suggested by Peter E)
>
> I think both a and b would work to achieve 2. But as Peter E said, b
> indeed sounds like less of a divergence of the status quo. So my vote
> would be for b.
FWIW, I think that there is value for both of them. Anyway, isn't 'a'
exactly the same as 'b' in design? Both require a build of
pg_bsd_indent, meaning that 'a' would also need to run an equivalent
of the regression test suite, but it would be actually costly
especially if pg_bsd_indent itself is patched. I think that getting
more noisy on this matter with 'b' would be enough, but as an extra
PG_TEST_EXTRA for committers to set.
Such a test suite would need a dependency to the 'git' command itself,
which is not something that could be safely run in a release tarball,
in any case.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 11:42 Andrew Dunstan <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 3 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-04-22 11:42 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; Jelte Fennema <[email protected]>; +Cc: [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-04-22 Sa 04:50, Michael Paquier wrote:
> On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote:
>> For 2 the upstream thread listed two approaches:
>> a. Install a pre-receive git hook on the git server that rejects
>> pushes to master that are not indented
>> b. Add a test suite that checks if the code is correctly indented, so
>> the build farm would complain about it. (Suggested by Peter E)
>>
>> I think both a and b would work to achieve 2. But as Peter E said, b
>> indeed sounds like less of a divergence of the status quo. So my vote
>> would be for b.
> FWIW, I think that there is value for both of them. Anyway, isn't 'a'
> exactly the same as 'b' in design? Both require a build of
> pg_bsd_indent, meaning that 'a' would also need to run an equivalent
> of the regression test suite, but it would be actually costly
> especially if pg_bsd_indent itself is patched. I think that getting
> more noisy on this matter with 'b' would be enough, but as an extra
> PG_TEST_EXTRA for committers to set.
>
> Such a test suite would need a dependency to the 'git' command itself,
> which is not something that could be safely run in a release tarball,
> in any case.
Perhaps we should start with a buildfarm module, which would run
pg_indent --show-diff. That would only need to run on one animal, so a
failure wouldn't send the whole buildfarm red. This would be pretty easy
to do.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 12:10 Michael Paquier <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 0 replies; 142+ messages in thread
From: Michael Paquier @ 2023-04-22 12:10 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sat, Apr 22, 2023 at 07:42:36AM -0400, Andrew Dunstan wrote:
> Perhaps we should start with a buildfarm module, which would run pg_indent
> --show-diff.
Nice, I didn't know this one and it has been mentioned a bit on this
thread. Indeed, it is possible to just rely on that.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 12:47 Magnus Hagander <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 2 replies; 142+ messages in thread
From: Magnus Hagander @ 2023-04-22 12:47 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Michael Paquier <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sat, Apr 22, 2023 at 1:42 PM Andrew Dunstan <[email protected]> wrote:
>
>
> On 2023-04-22 Sa 04:50, Michael Paquier wrote:
>
> On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote:
>
> For 2 the upstream thread listed two approaches:
> a. Install a pre-receive git hook on the git server that rejects
> pushes to master that are not indented
> b. Add a test suite that checks if the code is correctly indented, so
> the build farm would complain about it. (Suggested by Peter E)
>
> I think both a and b would work to achieve 2. But as Peter E said, b
> indeed sounds like less of a divergence of the status quo. So my vote
> would be for b.
>
> FWIW, I think that there is value for both of them. Anyway, isn't 'a'
> exactly the same as 'b' in design? Both require a build of
> pg_bsd_indent, meaning that 'a' would also need to run an equivalent
> of the regression test suite, but it would be actually costly
> especially if pg_bsd_indent itself is patched. I think that getting
> more noisy on this matter with 'b' would be enough, but as an extra
> PG_TEST_EXTRA for committers to set.
>
> Such a test suite would need a dependency to the 'git' command itself,
> which is not something that could be safely run in a release tarball,
> in any case.
>
>
> Perhaps we should start with a buildfarm module, which would run pg_indent --show-diff. That would only need to run on one animal, so a failure wouldn't send the whole buildfarm red. This would be pretty easy to do.
Just to be clear, you guys are aware we already have a git repo
that's supposed to track "head + pg_indent" at
https://git.postgresql.org/gitweb/?p=postgresql-pgindent.git;a=shortlog;h=refs/heads/master-pgindent
right?
I see it is currently not working and this has not been noticed by
anyone, so I guess it kind of indicates nobody is using it today. The
reason appears to be that it uses pg_bsd_indent that's in our apt
repos and that's 2.1.1 and not 2.1.2 at this point. But if this is a
service that would actually be useful, this could certainly be ficked
pretty easy.
But bottom line is that if pgindent is as predictable as it should be,
it might be easier to use that one central place that already does it
rather than have to build a buildfarm module?
//Magnus
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 14:12 Andrew Dunstan <[email protected]>
parent: Magnus Hagander <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-04-22 14:12 UTC (permalink / raw)
To: Magnus Hagander <[email protected]>; +Cc: Michael Paquier <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-04-22 Sa 08:47, Magnus Hagander wrote:
> On Sat, Apr 22, 2023 at 1:42 PM Andrew Dunstan<[email protected]> wrote:
>>
>> On 2023-04-22 Sa 04:50, Michael Paquier wrote:
>>
>> On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote:
>>
>> For 2 the upstream thread listed two approaches:
>> a. Install a pre-receive git hook on the git server that rejects
>> pushes to master that are not indented
>> b. Add a test suite that checks if the code is correctly indented, so
>> the build farm would complain about it. (Suggested by Peter E)
>>
>> I think both a and b would work to achieve 2. But as Peter E said, b
>> indeed sounds like less of a divergence of the status quo. So my vote
>> would be for b.
>>
>> FWIW, I think that there is value for both of them. Anyway, isn't 'a'
>> exactly the same as 'b' in design? Both require a build of
>> pg_bsd_indent, meaning that 'a' would also need to run an equivalent
>> of the regression test suite, but it would be actually costly
>> especially if pg_bsd_indent itself is patched. I think that getting
>> more noisy on this matter with 'b' would be enough, but as an extra
>> PG_TEST_EXTRA for committers to set.
>>
>> Such a test suite would need a dependency to the 'git' command itself,
>> which is not something that could be safely run in a release tarball,
>> in any case.
>>
>>
>> Perhaps we should start with a buildfarm module, which would run pg_indent --show-diff. That would only need to run on one animal, so a failure wouldn't send the whole buildfarm red. This would be pretty easy to do.
>
> Just to be clear, you guys are aware we already have a git repo
> that's supposed to track "head + pg_indent" at
> https://git.postgresql.org/gitweb/?p=postgresql-pgindent.git;a=shortlog;h=refs/heads/master-pgindent
> right?
>
> I see it is currently not working and this has not been noticed by
> anyone, so I guess it kind of indicates nobody is using it today. The
> reason appears to be that it uses pg_bsd_indent that's in our apt
> repos and that's 2.1.1 and not 2.1.2 at this point. But if this is a
> service that would actually be useful, this could certainly be ficked
> pretty easy.
>
> But bottom line is that if pgindent is as predictable as it should be,
> it might be easier to use that one central place that already does it
> rather than have to build a buildfarm module?
>
Now that pg_bsd_indent is in the core code why not just use that?
Happy if you can make something work without further effort on my part :-)
cheers
andew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 14:24 Tom Lane <[email protected]>
parent: Magnus Hagander <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Tom Lane @ 2023-04-22 14:24 UTC (permalink / raw)
To: Magnus Hagander <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Michael Paquier <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Magnus Hagander <[email protected]> writes:
> On Sat, Apr 22, 2023 at 1:42 PM Andrew Dunstan <[email protected]> wrote:
>> For 2 the upstream thread listed two approaches:
>> a. Install a pre-receive git hook on the git server that rejects
>> pushes to master that are not indented
>> b. Add a test suite that checks if the code is correctly indented, so
>> the build farm would complain about it. (Suggested by Peter E)
>>
>> I think both a and b would work to achieve 2. But as Peter E said, b
>> indeed sounds like less of a divergence of the status quo. So my vote
>> would be for b.
I am absolutely against a pre-receive hook on gitmaster. A buildfarm
check seems far more appropriate.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 14:39 Tom Lane <[email protected]>
parent: Jelte Fennema <[email protected]>
1 sibling, 2 replies; 142+ messages in thread
From: Tom Lane @ 2023-04-22 14:39 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Jelte Fennema <[email protected]> writes:
> I think there's two things needed to actually start doing this:
> 1. We need to reindent the tree to create an indented baseline
As far as (1) goes, I've been holding off on that because there
are some large patches that still seem in danger of getting
reverted, notably 2489d76c4 and follow-ups. A pgindent run
would change any such reversions from being mechanical into
possibly a fair bit of work. We still have a couple of weeks
before it's necessary to make such decisions, so I don't want
to do the pgindent run before that.
Another obstacle in the way of (1) is that there was some discussion
of changing perltidy version and/or options. But I don't believe
we have a final proposal on that, much less committed code.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 15:10 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-04-22 15:10 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Jelte Fennema <[email protected]>; +Cc: [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-04-22 Sa 10:39, Tom Lane wrote:
>
> Another obstacle in the way of (1) is that there was some discussion
> of changing perltidy version and/or options. But I don't believe
> we have a final proposal on that, much less committed code.
Well, I posted a fairly concrete suggestion with an example patch
upthread at
<https://www.postgresql.org/message-id/47011581-ddec-1a87-6828-6edfabe6b7b6%40dunslane.net;
I still think that's worth doing.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 15:37 Tom Lane <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-04-22 15:37 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Andrew Dunstan <[email protected]> writes:
> On 2023-04-22 Sa 10:39, Tom Lane wrote:
>> Another obstacle in the way of (1) is that there was some discussion
>> of changing perltidy version and/or options. But I don't believe
>> we have a final proposal on that, much less committed code.
> Well, I posted a fairly concrete suggestion with an example patch
> upthread at
> <https://www.postgresql.org/message-id/47011581-ddec-1a87-6828-6edfabe6b7b6%40dunslane.net;
> I still think that's worth doing.
OK, so plan is (a) update perltidyrc to add --valign-exclusion-list,
(b) adjust pgindent/README to recommend perltidy version 20221112.
Questions:
* I see that there's now a 20230309 release, should we consider that
instead?
* emacs.samples provides pgsql-perl-style that claims to match
perltidy's rules. Does that need any adjustment? I don't see
anything in it that looks relevant, but I'm not terribly familiar
with emacs' Perl formatting options.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 19:52 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-04-22 19:52 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-04-22 Sa 11:37, Tom Lane wrote:
> Andrew Dunstan<[email protected]> writes:
>> On 2023-04-22 Sa 10:39, Tom Lane wrote:
>>> Another obstacle in the way of (1) is that there was some discussion
>>> of changing perltidy version and/or options. But I don't believe
>>> we have a final proposal on that, much less committed code.
>> Well, I posted a fairly concrete suggestion with an example patch
>> upthread at
>> <https://www.postgresql.org/message-id/47011581-ddec-1a87-6828-6edfabe6b7b6%40dunslane.net;
>> I still think that's worth doing.
> OK, so plan is (a) update perltidyrc to add --valign-exclusion-list,
> (b) adjust pgindent/README to recommend perltidy version 20221112.
>
> Questions:
>
> * I see that there's now a 20230309 release, should we consider that
> instead?
A test I just ran gave identical results to those from 20221112
>
> * emacs.samples provides pgsql-perl-style that claims to match
> perltidy's rules. Does that need any adjustment? I don't see
> anything in it that looks relevant, but I'm not terribly familiar
> with emacs' Perl formatting options.
At least w.r.t. the vertical alignment issue, AFAICT the emacs style
does not attempt to align anything vertically except the first non-space
thing on the line. So if anything, by abandoning a lot of vertical
alignment it would actually be closer to what the sample emacs style does.
The great advantage of not doing this alignment is that there is far
less danger of perltidy trying to realign lines that have not in fact
changed, because some nearby line has changed. So we'd have a good deal
less pointless churn.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-22 19:58 Tom Lane <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Tom Lane @ 2023-04-22 19:58 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Andrew Dunstan <[email protected]> writes:
> On 2023-04-22 Sa 11:37, Tom Lane wrote:
>> * I see that there's now a 20230309 release, should we consider that
>> instead?
> A test I just ran gave identical results to those from 20221112
Cool, let's use perltidy 20230309 then.
> The great advantage of not doing this alignment is that there is far
> less danger of perltidy trying to realign lines that have not in fact
> changed, because some nearby line has changed. So we'd have a good deal
> less pointless churn.
Yes, exactly.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-23 15:04 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-04-23 15:04 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-04-22 Sa 15:58, Tom Lane wrote:
> Andrew Dunstan<[email protected]> writes:
>> On 2023-04-22 Sa 11:37, Tom Lane wrote:
>>> * I see that there's now a 20230309 release, should we consider that
>>> instead?
>> A test I just ran gave identical results to those from 20221112
> Cool, let's use perltidy 20230309 then.
>
>
OK, so when would we do this? The change to 20230309 + valign changes is
fairly large:
188 files changed, 3657 insertions(+), 3395 deletions(-)
Maybe right before we fork the tree?
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-23 15:16 Tom Lane <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-04-23 15:16 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Andrew Dunstan <[email protected]> writes:
> On 2023-04-22 Sa 15:58, Tom Lane wrote:
>> Cool, let's use perltidy 20230309 then.
> OK, so when would we do this? The change to 20230309 + valign changes is
> fairly large:
I think we could go ahead and commit the perltidyrc and README changes
now. But the ensuing reformatting should happen as part of the mass
pgindent run, probably next month sometime.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-23 15:29 Jelte Fennema <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Jelte Fennema @ 2023-04-23 15:29 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sun, 23 Apr 2023 at 17:16, Tom Lane <[email protected]> wrote:
> I think we could go ahead and commit the perltidyrc and README changes
> now. But the ensuing reformatting should happen as part of the mass
> pgindent run, probably next month sometime.
I think it's better to make the changes close together, not with a
month in between. Otherwise no-one will be able to run perltidy on
their patches, because the config and the files are even more out of
sync than they are now. So I'd propose to commit the perltidyrc
changes right before the pgindent run.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-23 21:52 Magnus Hagander <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Magnus Hagander @ 2023-04-23 21:52 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Michael Paquier <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sat, Apr 22, 2023 at 4:12 PM Andrew Dunstan <[email protected]> wrote:
>
>
> On 2023-04-22 Sa 08:47, Magnus Hagander wrote:
>
> On Sat, Apr 22, 2023 at 1:42 PM Andrew Dunstan <[email protected]> wrote:
>
> On 2023-04-22 Sa 04:50, Michael Paquier wrote:
>
> On Fri, Apr 21, 2023 at 09:58:17AM +0200, Jelte Fennema wrote:
>
> For 2 the upstream thread listed two approaches:
> a. Install a pre-receive git hook on the git server that rejects
> pushes to master that are not indented
> b. Add a test suite that checks if the code is correctly indented, so
> the build farm would complain about it. (Suggested by Peter E)
>
> I think both a and b would work to achieve 2. But as Peter E said, b
> indeed sounds like less of a divergence of the status quo. So my vote
> would be for b.
>
> FWIW, I think that there is value for both of them. Anyway, isn't 'a'
> exactly the same as 'b' in design? Both require a build of
> pg_bsd_indent, meaning that 'a' would also need to run an equivalent
> of the regression test suite, but it would be actually costly
> especially if pg_bsd_indent itself is patched. I think that getting
> more noisy on this matter with 'b' would be enough, but as an extra
> PG_TEST_EXTRA for committers to set.
>
> Such a test suite would need a dependency to the 'git' command itself,
> which is not something that could be safely run in a release tarball,
> in any case.
>
>
> Perhaps we should start with a buildfarm module, which would run pg_indent --show-diff. That would only need to run on one animal, so a failure wouldn't send the whole buildfarm red. This would be pretty easy to do.
>
> Just to be clear, you guys are aware we already have a git repo
> that's supposed to track "head + pg_indent" at
> https://git.postgresql.org/gitweb/?p=postgresql-pgindent.git;a=shortlog;h=refs/heads/master-pgindent
> right?
>
> I see it is currently not working and this has not been noticed by
> anyone, so I guess it kind of indicates nobody is using it today. The
> reason appears to be that it uses pg_bsd_indent that's in our apt
> repos and that's 2.1.1 and not 2.1.2 at this point. But if this is a
> service that would actually be useful, this could certainly be ficked
> pretty easy.
>
> But bottom line is that if pgindent is as predictable as it should be,
> it might be easier to use that one central place that already does it
> rather than have to build a buildfarm module?
>
>
> Now that pg_bsd_indent is in the core code why not just use that?
yeah, it just required building. And the lazy approach was to use the DEB :)
For a quick fix I've built the current HEAD and have it just using
that one -- right now it'll fail again when a change is made to it,
but I'll get that cleaned up.
It's back up and running, and results are at
https://git.postgresql.org/gitweb/?p=postgresql-pgindent.git;a=shortlog;h=refs/heads/master-pgindent
--
Magnus Hagander
Me: https://www.hagander.net/
Work: https://www.redpill-linpro.com/
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-24 14:09 Peter Eisentraut <[email protected]>
parent: Jelte Fennema <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Peter Eisentraut @ 2023-04-24 14:09 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; Tom Lane <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 23.04.23 17:29, Jelte Fennema wrote:
> On Sun, 23 Apr 2023 at 17:16, Tom Lane <[email protected]> wrote:
>> I think we could go ahead and commit the perltidyrc and README changes
>> now. But the ensuing reformatting should happen as part of the mass
>> pgindent run, probably next month sometime.
>
> I think it's better to make the changes close together, not with a
> month in between. Otherwise no-one will be able to run perltidy on
> their patches, because the config and the files are even more out of
> sync than they are now. So I'd propose to commit the perltidyrc
> changes right before the pgindent run.
Does anyone find perltidy useful? To me, it functions more like a
JavaScript compiler in that once you process the source code, it is no
longer useful for manual editing. If we are going to have the buildfarm
check indentation and that is going to be extended to Perl code, I have
some concerns about that.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-24 14:14 Tom Lane <[email protected]>
parent: Jelte Fennema <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-04-24 14:14 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Peter Eisentraut <[email protected]> writes:
> Does anyone find perltidy useful? To me, it functions more like a
> JavaScript compiler in that once you process the source code, it is no
> longer useful for manual editing. If we are going to have the buildfarm
> check indentation and that is going to be extended to Perl code, I have
> some concerns about that.
I certainly don't like its current behavior where adding/changing one
line can have side-effects on nearby lines. But we have a proposal
to clean that up, and I'm cautiously optimistic that it'll be better
in future. Did you have other specific concerns?
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-26 07:38 Peter Eisentraut <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Peter Eisentraut @ 2023-04-26 07:38 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 24.04.23 16:14, Tom Lane wrote:
> Peter Eisentraut <[email protected]> writes:
>> Does anyone find perltidy useful? To me, it functions more like a
>> JavaScript compiler in that once you process the source code, it is no
>> longer useful for manual editing. If we are going to have the buildfarm
>> check indentation and that is going to be extended to Perl code, I have
>> some concerns about that.
>
> I certainly don't like its current behavior where adding/changing one
> line can have side-effects on nearby lines. But we have a proposal
> to clean that up, and I'm cautiously optimistic that it'll be better
> in future. Did you have other specific concerns?
I think the worst is how it handles multi-line data structures like
$newnode->command_ok(
[
'psql', '-X',
'-v', 'ON_ERROR_STOP=1',
'-c', $upcmds,
'-d', $oldnode->connstr($updb),
],
"ran version adaptation commands for database $updb");
or
$node->command_fails_like(
[
'pg_basebackup', '-D',
"$tempdir/backup", '--compress',
$cft->[0]
],
qr/$cfail/,
'client ' . $cft->[2]);
Perhaps that is included in the upcoming changes you are referring to?
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-26 13:27 Tom Lane <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Tom Lane @ 2023-04-26 13:27 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Peter Eisentraut <[email protected]> writes:
> On 24.04.23 16:14, Tom Lane wrote:
>> I certainly don't like its current behavior where adding/changing one
>> line can have side-effects on nearby lines. But we have a proposal
>> to clean that up, and I'm cautiously optimistic that it'll be better
>> in future. Did you have other specific concerns?
> I think the worst is how it handles multi-line data structures like
> $newnode->command_ok(
> [
> 'psql', '-X',
> '-v', 'ON_ERROR_STOP=1',
> '-c', $upcmds,
> '-d', $oldnode->connstr($updb),
> ],
> "ran version adaptation commands for database $updb");
Yeah, I agree, there is no case where that doesn't suck. I don't
mind it imposing specific placements of brackets and so on ---
that's very analogous to what pgindent will do. But it likes to
re-flow comma-separated lists, and generally manages to make a
complete logical hash of them when it does, as in your other
example:
> $node->command_fails_like(
> [
> 'pg_basebackup', '-D',
> "$tempdir/backup", '--compress',
> $cft->[0]
> ],
> qr/$cfail/,
> 'client ' . $cft->[2]);
Can we fix it to preserve the programmer's choices of line breaks
in comma-separated lists?
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-26 19:44 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-04-26 19:44 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-04-26 We 09:27, Tom Lane wrote:
> Peter Eisentraut<[email protected]> writes:
>> On 24.04.23 16:14, Tom Lane wrote:
>>> I certainly don't like its current behavior where adding/changing one
>>> line can have side-effects on nearby lines. But we have a proposal
>>> to clean that up, and I'm cautiously optimistic that it'll be better
>>> in future. Did you have other specific concerns?
>> I think the worst is how it handles multi-line data structures like
>> $newnode->command_ok(
>> [
>> 'psql', '-X',
>> '-v', 'ON_ERROR_STOP=1',
>> '-c', $upcmds,
>> '-d', $oldnode->connstr($updb),
>> ],
>> "ran version adaptation commands for database $updb");
> Yeah, I agree, there is no case where that doesn't suck. I don't
> mind it imposing specific placements of brackets and so on ---
> that's very analogous to what pgindent will do. But it likes to
> re-flow comma-separated lists, and generally manages to make a
> complete logical hash of them when it does, as in your other
> example:
>
>> $node->command_fails_like(
>> [
>> 'pg_basebackup', '-D',
>> "$tempdir/backup", '--compress',
>> $cft->[0]
>> ],
>> qr/$cfail/,
>> 'client ' . $cft->[2]);
> Can we fix it to preserve the programmer's choices of line breaks
> in comma-separated lists?
I doubt there's something like that. You can freeze arbitrary blocks of
code like this (from the manual)
#<<< format skipping: do not let perltidy change my nice formatting
my @list = (1,
1, 1,
1, 2, 1,
1, 3, 3, 1,
1, 4, 6, 4, 1,);
#>>>
But that gets old and ugly pretty quickly.
There is a --freeze-newlines option, but it's global. I don't think we
want that.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-26 20:05 Tom Lane <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-04-26 20:05 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Andrew Dunstan <[email protected]> writes:
> On 2023-04-26 We 09:27, Tom Lane wrote:
>> Yeah, I agree, there is no case where that doesn't suck. I don't
>> mind it imposing specific placements of brackets and so on ---
>> that's very analogous to what pgindent will do. But it likes to
>> re-flow comma-separated lists, and generally manages to make a
>> complete logical hash of them when it does, as in your other
>> example:
> I doubt there's something like that.
I had a read-through of the latest version's man page, and found
this promising-looking entry:
-boc, --break-at-old-comma-breakpoints
The -boc flag is another way to prevent comma-separated lists from
being reformatted. Using -boc on the above example, plus additional
flags to retain the original style, yields
# perltidy -boc -lp -pt=2 -vt=1 -vtc=1
my @list = (1,
1, 1,
1, 2, 1,
1, 3, 3, 1,
1, 4, 6, 4, 1,);
A disadvantage of this flag compared to the methods discussed above is
that all tables in the file must already be nicely formatted.
I've not tested this, but it looks like it would do what we need,
modulo needing to fix all the existing damage by hand ...
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-28 09:25 Alvaro Herrera <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Alvaro Herrera @ 2023-04-28 09:25 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Andres Freund <[email protected]>; Noah Misch <[email protected]>; Tom Lane <[email protected]>; Jelte Fennema <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-Feb-05, Andrew Dunstan wrote:
> So here's a diff made from running perltidy v20221112 with the additional
> setting --valign-exclusion-list=", = => || && if unless"
I ran this experimentally with perltidy 20230309, and compared that with
the --novalign behavior (not to propose the latter -- just to be aware
of what else is vertical alignment doing.)
Based on the differences between both, I think we'll definitely want to
include =~ and |= in this list, and I think we should discuss whether to
also include "or" (for "do_stuff or die()" type of constructs) and "qw"
(mainly used in 'use Foo qw(one two)' import lists). All these have
effects (albeit smaller than the list you gave) on our existing code.
If you change from an exclusion list to --novalign then you lose
alignment of trailing # comments, which personally I find a loss, even
though they're still a multi-line effect. Another change would be that
it ditches alignment of "{" but that only changes msvc/Install.pm, so I
think we shouldn't worry; and then there's this one:
-use PostgreSQL::Test::Utils ();
+use PostgreSQL::Test::Utils ();
use PostgreSQL::Test::BackgroundPsql ();
which I think we could just change to qw() if we cared enough (but I bet
we don't).
All in all, I think sticking to
--valign-exclusion-list=", = => =~ |= || && if or qw unless"
is a good deal.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Ellos andaban todos desnudos como su madre los parió, y también las mujeres,
aunque no vi más que una, harto moza, y todos los que yo vi eran todos
mancebos, que ninguno vi de edad de más de XXX años" (Cristóbal Colón)
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-28 18:08 Bruce Momjian <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Bruce Momjian @ 2023-04-28 18:08 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Wed, Apr 26, 2023 at 03:44:47PM -0400, Andrew Dunstan wrote:
> On 2023-04-26 We 09:27, Tom Lane wrote:
> I doubt there's something like that. You can freeze arbitrary blocks of code
> like this (from the manual)
>
> #<<< format skipping: do not let perltidy change my nice formatting
> my @list = (1,
> 1, 1,
> 1, 2, 1,
> 1, 3, 3, 1,
> 1, 4, 6, 4, 1,);
> #>>>
>
>
> But that gets old and ugly pretty quickly.
Can those comments be added by a preprocessor before calling perltidy,
and then removed on completion?
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Embrace your flaws. They make you human, rather than perfect,
which you will never be.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-30 13:02 Andrew Dunstan <[email protected]>
parent: Bruce Momjian <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-04-30 13:02 UTC (permalink / raw)
To: Bruce Momjian <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-04-28 Fr 14:08, Bruce Momjian wrote:
> On Wed, Apr 26, 2023 at 03:44:47PM -0400, Andrew Dunstan wrote:
>> On 2023-04-26 We 09:27, Tom Lane wrote:
>> I doubt there's something like that. You can freeze arbitrary blocks of code
>> like this (from the manual)
>>
>> #<<< format skipping: do not let perltidy change my nice formatting
>> my @list = (1,
>> 1, 1,
>> 1, 2, 1,
>> 1, 3, 3, 1,
>> 1, 4, 6, 4, 1,);
>> #>>>
>>
>>
>> But that gets old and ugly pretty quickly.
> Can those comments be added by a preprocessor before calling perltidy,
> and then removed on completion?
>
I imagine so, but we'd need a way of determining algorithmically which
lines to protect. That might not be at all simple. And then we'd have
the maintenance burden of the preprocessor.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-04-30 14:32 Tom Lane <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Tom Lane @ 2023-04-30 14:32 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Bruce Momjian <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
Andrew Dunstan <[email protected]> writes:
> On 2023-04-28 Fr 14:08, Bruce Momjian wrote:
>> Can those comments be added by a preprocessor before calling perltidy,
>> and then removed on completion?
> I imagine so, but we'd need a way of determining algorithmically which
> lines to protect. That might not be at all simple. And then we'd have
> the maintenance burden of the preprocessor.
Yeah, it's hard to see how you'd do that without writing a full Perl
parser.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-05-17 21:10 Tom Lane <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-05-17 21:10 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
I wrote:
> Andrew Dunstan <[email protected]> writes:
>> I doubt there's something like that.
> I had a read-through of the latest version's man page, and found
> this promising-looking entry:
> -boc, --break-at-old-comma-breakpoints
Sadly, this seems completely not ready for prime time. I experimented
with it under perltidy 20230309, and found that it caused hundreds
of kilobytes of gratuitous changes that don't seem to have a direct
connection to the claimed purpose. Most of these seemed to be from
forcing a line break after a function call's open paren, like
@@ -50,10 +50,12 @@ detects_heap_corruption(
#
fresh_test_table('test');
$node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
-detects_no_corruption("verify_heapam('test')",
+detects_no_corruption(
+ "verify_heapam('test')",
"all-frozen not corrupted table");
corrupt_first_page('test');
-detects_heap_corruption("verify_heapam('test')",
+detects_heap_corruption(
+ "verify_heapam('test')",
"all-frozen corrupted table");
detects_no_corruption(
"verify_heapam('test', skip := 'all-frozen')",
although in some places it just wanted to insert a space, like this:
@@ -77,9 +81,9 @@ print "standby 2: $result\n";
is($result, qq(33|0|t), 'check streamed sequence content on standby 2');
# Check that only READ-only queries can run on standbys
-is($node_standby_1->psql('postgres', 'INSERT INTO tab_int VALUES (1)'),
+is( $node_standby_1->psql('postgres', 'INSERT INTO tab_int VALUES (1)'),
3, 'read-only queries on standby 1');
-is($node_standby_2->psql('postgres', 'INSERT INTO tab_int VALUES (1)'),
+is( $node_standby_2->psql('postgres', 'INSERT INTO tab_int VALUES (1)'),
3, 'read-only queries on standby 2');
# Tests for connection parameter target_session_attrs
So I don't think we want that. Maybe in some future version it'll
be more under control.
Barring objections, I'll use the attached on Friday.
regards, tom lane
Attachments:
[text/x-diff] v1-perltidy-option-changes.patch (1.7K, ../../[email protected]/2-v1-perltidy-option-changes.patch)
download | inline diff:
commit 7874d0f178f2bcdc889ce410d3e126e6750d96b4
Author: Tom Lane <[email protected]>
Date: Wed May 17 16:43:38 2023 -0400
Make agreed updates in perltidy options.
Discussion: https://postgr.es/m/[email protected]
diff --git a/src/tools/pgindent/README b/src/tools/pgindent/README
index 43c736b0a1..08874d12eb 100644
--- a/src/tools/pgindent/README
+++ b/src/tools/pgindent/README
@@ -14,16 +14,16 @@ PREREQUISITES:
sibling directory src/tools/pg_bsd_indent; see the directions
in that directory's README file.
-2) Install perltidy. Please be sure it is version 20170521 (older and newer
+2) Install perltidy. Please be sure it is version 20230309 (older and newer
versions make different formatting choices, and we want consistency).
You can get the correct version from
https://cpan.metacpan.org/authors/id/S/SH/SHANCOCK/
To install, follow the usual install process for a Perl module
("man perlmodinstall" explains it). Or, if you have cpan installed,
this should work:
- cpan SHANCOCK/Perl-Tidy-20170521.tar.gz
+ cpan SHANCOCK/Perl-Tidy-20230309.tar.gz
Or if you have cpanm installed, you can just use:
- cpanm https://cpan.metacpan.org/authors/id/S/SH/SHANCOCK/Perl-Tidy-20170521.tar.gz
+ cpanm https://cpan.metacpan.org/authors/id/S/SH/SHANCOCK/Perl-Tidy-20230309.tar.gz
DOING THE INDENT RUN:
diff --git a/src/tools/pgindent/perltidyrc b/src/tools/pgindent/perltidyrc
index 9f09f0a64e..589d6e1f06 100644
--- a/src/tools/pgindent/perltidyrc
+++ b/src/tools/pgindent/perltidyrc
@@ -14,3 +14,4 @@
--paren-vertical-tightness=2
--paren-vertical-tightness-closing=2
--noblanks-before-comments
+--valign-exclusion-list=", = => =~ |= || && if or qw unless"
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-05-18 12:59 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-05-18 12:59 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-05-17 We 17:10, Tom Lane wrote:
> I wrote:
>> Andrew Dunstan<[email protected]> writes:
>>> I doubt there's something like that.
>> I had a read-through of the latest version's man page, and found
>> this promising-looking entry:
>> -boc, --break-at-old-comma-breakpoints
> Sadly, this seems completely not ready for prime time. I experimented
> with it under perltidy 20230309, and found that it caused hundreds
> of kilobytes of gratuitous changes that don't seem to have a direct
> connection to the claimed purpose. Most of these seemed to be from
> forcing a line break after a function call's open paren, like
>
> @@ -50,10 +50,12 @@ detects_heap_corruption(
> #
> fresh_test_table('test');
> $node->safe_psql('postgres', q(VACUUM (FREEZE, DISABLE_PAGE_SKIPPING) test));
> -detects_no_corruption("verify_heapam('test')",
> +detects_no_corruption(
> + "verify_heapam('test')",
> "all-frozen not corrupted table");
> corrupt_first_page('test');
> -detects_heap_corruption("verify_heapam('test')",
> +detects_heap_corruption(
> + "verify_heapam('test')",
> "all-frozen corrupted table");
> detects_no_corruption(
> "verify_heapam('test', skip := 'all-frozen')",
>
> although in some places it just wanted to insert a space, like this:
>
> @@ -77,9 +81,9 @@ print "standby 2: $result\n";
> is($result, qq(33|0|t), 'check streamed sequence content on standby 2');
>
> # Check that only READ-only queries can run on standbys
> -is($node_standby_1->psql('postgres', 'INSERT INTO tab_int VALUES (1)'),
> +is( $node_standby_1->psql('postgres', 'INSERT INTO tab_int VALUES (1)'),
> 3, 'read-only queries on standby 1');
> -is($node_standby_2->psql('postgres', 'INSERT INTO tab_int VALUES (1)'),
> +is( $node_standby_2->psql('postgres', 'INSERT INTO tab_int VALUES (1)'),
> 3, 'read-only queries on standby 2');
>
> # Tests for connection parameter target_session_attrs
>
>
> So I don't think we want that. Maybe in some future version it'll
> be more under control.
>
> Barring objections, I'll use the attached on Friday.
>
>
LGTM
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-06-15 15:26 Jelte Fennema <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: Jelte Fennema @ 2023-06-15 15:26 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sat, 22 Apr 2023 at 13:42, Andrew Dunstan <[email protected]> wrote:
> Perhaps we should start with a buildfarm module, which would run pg_indent --show-diff. That would only need to run on one animal, so a failure wouldn't send the whole buildfarm red. This would be pretty easy to do.
Just to be clear on where we are. Is there anything blocking us from
doing this, except for the PG16 branch cut? (that I guess is planned
somewhere in July?)
Just doing this for pgindent and not for perltidy would already be a
huge improvement over the current situation IMHO.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-06-15 16:12 Andrew Dunstan <[email protected]>
parent: Jelte Fennema <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-06-15 16:12 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-06-15 Th 11:26, Jelte Fennema wrote:
> On Sat, 22 Apr 2023 at 13:42, Andrew Dunstan<[email protected]> wrote:
>> Perhaps we should start with a buildfarm module, which would run pg_indent --show-diff. That would only need to run on one animal, so a failure wouldn't send the whole buildfarm red. This would be pretty easy to do.
> Just to be clear on where we are. Is there anything blocking us from
> doing this, except for the PG16 branch cut? (that I guess is planned
> somewhere in July?)
>
> Just doing this for pgindent and not for perltidy would already be a
> huge improvement over the current situation IMHO.
The short answer is that some high priority demands from $dayjob got in
the way. However, I hope to have it done soon.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-06-17 14:08 Andrew Dunstan <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 3 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-06-17 14:08 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-06-15 Th 12:12, Andrew Dunstan wrote:
>
>
> On 2023-06-15 Th 11:26, Jelte Fennema wrote:
>> On Sat, 22 Apr 2023 at 13:42, Andrew Dunstan<[email protected]> wrote:
>>> Perhaps we should start with a buildfarm module, which would run pg_indent --show-diff. That would only need to run on one animal, so a failure wouldn't send the whole buildfarm red. This would be pretty easy to do.
>> Just to be clear on where we are. Is there anything blocking us from
>> doing this, except for the PG16 branch cut? (that I guess is planned
>> somewhere in July?)
>>
>> Just doing this for pgindent and not for perltidy would already be a
>> huge improvement over the current situation IMHO.
>
>
> The short answer is that some high priority demands from $dayjob got
> in the way. However, I hope to have it done soon.
>
See
<https://github.com/PGBuildFarm/client-code/commit/f9c1c15048b412d34ccda8020d989b3a7b566c05;
I have set up a new buildfarm animal called koel which will run the module.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-06-19 21:07 Tom Lane <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-06-19 21:07 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Andrew Dunstan <[email protected]> writes:
> I have set up a new buildfarm animal called koel which will run the module.
Is koel tracking the right repo? It just spit up with a bunch of
diffs that seem to have little to do with the commit it's claiming
caused them:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=koel&dt=2023-06-19%2019%3A49%3A03
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-06-20 02:09 Michael Paquier <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 0 replies; 142+ messages in thread
From: Michael Paquier @ 2023-06-20 02:09 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sat, Jun 17, 2023 at 10:08:32AM -0400, Andrew Dunstan wrote:
> See <https://github.com/PGBuildFarm/client-code/commit/f9c1c15048b412d34ccda8020d989b3a7b566c05;
> I have set up a new buildfarm animal called koel which will run the module.
That's really cool! Thanks for taking the time to do that!
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-06-20 12:04 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-06-20 12:04 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-06-19 Mo 17:07, Tom Lane wrote:
> Andrew Dunstan<[email protected]> writes:
>> I have set up a new buildfarm animal called koel which will run the module.
> Is koel tracking the right repo? It just spit up with a bunch of
> diffs that seem to have little to do with the commit it's claiming
> caused them:
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=koel&dt=2023-06-19%2019%3A49%3A03
>
>
Yeah, I changed it so that instead of just checking new commits it would
check the whole tree. The problem with the incremental approach is that
the next run it might turn green again but the issue would not have been
fixed.
I think this is a one-off issue. Once we clean up the tree the problem
would disappear and the commits it shows would be correct. I imaging
that's going to happen any day now?
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-06-20 13:08 Tom Lane <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-06-20 13:08 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Andrew Dunstan <[email protected]> writes:
> On 2023-06-19 Mo 17:07, Tom Lane wrote:
>> Is koel tracking the right repo? It just spit up with a bunch of
>> diffs that seem to have little to do with the commit it's claiming
>> caused them:
> Yeah, I changed it so that instead of just checking new commits it would
> check the whole tree. The problem with the incremental approach is that
> the next run it might turn green again but the issue would not have been
> fixed.
Ah.
> I think this is a one-off issue. Once we clean up the tree the problem
> would disappear and the commits it shows would be correct. I imaging
> that's going to happen any day now?
I can go fix the problems now that we know there are some (already).
However, if what you're saying is that koel only checks recently-changed
files, that's going to be pretty misleading in future too. If people
don't react to such reports right away, they'll disappear, no?
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-06-20 13:21 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-06-20 13:21 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Peter Geoghegan <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-06-20 Tu 09:08, Tom Lane wrote:
> Andrew Dunstan<[email protected]> writes:
>> On 2023-06-19 Mo 17:07, Tom Lane wrote:
>>> Is koel tracking the right repo? It just spit up with a bunch of
>>> diffs that seem to have little to do with the commit it's claiming
>>> caused them:
>> Yeah, I changed it so that instead of just checking new commits it would
>> check the whole tree. The problem with the incremental approach is that
>> the next run it might turn green again but the issue would not have been
>> fixed.
> Ah.
>
>> I think this is a one-off issue. Once we clean up the tree the problem
>> would disappear and the commits it shows would be correct. I imaging
>> that's going to happen any day now?
> I can go fix the problems now that we know there are some (already).
> However, if what you're saying is that koel only checks recently-changed
> files, that's going to be pretty misleading in future too. If people
> don't react to such reports right away, they'll disappear, no?
>
>
That's what would have happened if I hadn't changed the way it worked
(and that's why I changed it). Now it doesn't just check recent commits,
it checks the whole tree, and will stay red until the tree is fixed.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-11 20:59 Peter Geoghegan <[email protected]>
parent: Andrew Dunstan <[email protected]>
2 siblings, 4 replies; 142+ messages in thread
From: Peter Geoghegan @ 2023-08-11 20:59 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sat, Jun 17, 2023 at 7:08 AM Andrew Dunstan <[email protected]> wrote:
> I have set up a new buildfarm animal called koel which will run the module.
I'm starting to have doubts about this policy. There have now been
quite a few follow-up "fixes" to indentation issues that koel
complained about. None of these fixups have been included in
.git-blame-ignore-revs. If things continue like this then "git blame"
is bound to become much less usable over time.
I don't think that it makes sense to invent yet another rule for
.git-blame-ignore-revs, though. Will we need another buildfarm member
to enforce that rule, too?
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-11 21:25 Andres Freund <[email protected]>
parent: Peter Geoghegan <[email protected]>
3 siblings, 1 reply; 142+ messages in thread
From: Andres Freund @ 2023-08-11 21:25 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Hi,
On 2023-08-11 13:59:40 -0700, Peter Geoghegan wrote:
> On Sat, Jun 17, 2023 at 7:08 AM Andrew Dunstan <[email protected]> wrote:
> > I have set up a new buildfarm animal called koel which will run the module.
>
> I'm starting to have doubts about this policy. There have now been
> quite a few follow-up "fixes" to indentation issues that koel
> complained about. None of these fixups have been included in
> .git-blame-ignore-revs. If things continue like this then "git blame"
> is bound to become much less usable over time.
I'm not sure I buy that that's going to be a huge problem - most of the time
such fixups are pretty small compared to larger reindents.
> I don't think that it makes sense to invent yet another rule for
> .git-blame-ignore-revs, though. Will we need another buildfarm member
> to enforce that rule, too?
We could a test that fails when there's some mis-indented code. That seems
like it might catch things earlier?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-11 21:48 Peter Geoghegan <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Peter Geoghegan @ 2023-08-11 21:48 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Fri, Aug 11, 2023 at 2:25 PM Andres Freund <[email protected]> wrote:
> > I don't think that it makes sense to invent yet another rule for
> > .git-blame-ignore-revs, though. Will we need another buildfarm member
> > to enforce that rule, too?
>
> We could a test that fails when there's some mis-indented code. That seems
> like it might catch things earlier?
It definitely would. That would go a long way towards addressing my
concerns. But I suspect that that would run into problems that stem
from the fact that the buildfarm is testing something that isn't all
that simple. Don't typedefs need to be downloaded from some other
blessed buildfarm animal?
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-11 22:30 Tom Lane <[email protected]>
parent: Peter Geoghegan <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Tom Lane @ 2023-08-11 22:30 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Peter Geoghegan <[email protected]> writes:
> On Fri, Aug 11, 2023 at 2:25 PM Andres Freund <[email protected]> wrote:
>> We could a test that fails when there's some mis-indented code. That seems
>> like it might catch things earlier?
+1 for including this in CI tests
> It definitely would. That would go a long way towards addressing my
> concerns. But I suspect that that would run into problems that stem
> from the fact that the buildfarm is testing something that isn't all
> that simple. Don't typedefs need to be downloaded from some other
> blessed buildfarm animal?
No. I presume koel is using src/tools/pgindent/typedefs.list,
which has always been the "canonical" list but up to now we've
been lazy about maintaining it. Part of the new regime is that
typedefs.list should now be updated on-the-fly by patches that
add new typedefs.
We should still compare against the buildfarm's list periodically;
but I imagine that the primary result of that will be to remove
no-longer-used typedefs from typedefs.list.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-11 22:46 Peter Geoghegan <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Peter Geoghegan @ 2023-08-11 22:46 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Fri, Aug 11, 2023 at 3:30 PM Tom Lane <[email protected]> wrote:
> No. I presume koel is using src/tools/pgindent/typedefs.list,
> which has always been the "canonical" list but up to now we've
> been lazy about maintaining it. Part of the new regime is that
> typedefs.list should now be updated on-the-fly by patches that
> add new typedefs.
My workflow up until now has avoiding making updates to typedefs.list
in patches. I only update typedefs locally, for long enough to indent
my code. The final patch doesn't retain any typedefs.list changes.
> We should still compare against the buildfarm's list periodically;
> but I imagine that the primary result of that will be to remove
> no-longer-used typedefs from typedefs.list.
I believe that I came up with my current workflow due to the
difficulty of maintaining the typedef file itself. Random
platform/binutils implementation details created a lot of noise,
presumably because my setup wasn't exactly the same as Bruce's setup,
in whatever way. For example, the order of certain lines would change,
in a way that had nothing whatsoever to do with structs that my patch
added.
I guess that I can't do that anymore. Hopefully maintaining the
typedefs.list file isn't as inconvenient as it once seemed to me to
be.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-11 23:02 Tom Lane <[email protected]>
parent: Peter Geoghegan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-08-11 23:02 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Peter Geoghegan <[email protected]> writes:
> My workflow up until now has avoiding making updates to typedefs.list
> in patches. I only update typedefs locally, for long enough to indent
> my code. The final patch doesn't retain any typedefs.list changes.
Yeah, I've done the same and will have to stop.
> I guess that I can't do that anymore. Hopefully maintaining the
> typedefs.list file isn't as inconvenient as it once seemed to me to
> be.
I don't think it'll be a problem. If your rule is "add new typedef
names added by your patch to typedefs.list, keeping them in
alphabetical order" then it doesn't seem very complicated, and
hopefully conflicts between concurrently-developed patches won't
be common.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-11 23:17 Tom Lane <[email protected]>
parent: Peter Geoghegan <[email protected]>
3 siblings, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-08-11 23:17 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Peter Geoghegan <[email protected]> writes:
> I'm starting to have doubts about this policy. There have now been
> quite a few follow-up "fixes" to indentation issues that koel
> complained about. None of these fixups have been included in
> .git-blame-ignore-revs. If things continue like this then "git blame"
> is bound to become much less usable over time.
FWIW, I'm much more optimistic than that. I think what we're seeing
is just the predictable result of not all committers having yet
incorporated "pgindent it before committing" into their workflow.
The need for followup fixes should diminish as people start doing
that. If you want to hurry things along, peer pressure on committers
who clearly aren't bothering is the solution.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-11 23:18 Jelte Fennema <[email protected]>
parent: Peter Geoghegan <[email protected]>
3 siblings, 0 replies; 142+ messages in thread
From: Jelte Fennema @ 2023-08-11 23:18 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Fri, 11 Aug 2023 at 23:00, Peter Geoghegan <[email protected]> wrote:
> I'm starting to have doubts about this policy. There have now been
> quite a few follow-up "fixes" to indentation issues that koel
> complained about.
I think one thing that would help a lot in reducing the is for
committers to set up the local git commit hook that's on the wiki:
https://wiki.postgresql.org/wiki/Working_with_Git
That one fails the commit if there's wrongly indented files in the
commit. And if you still want to opt out for whatever reason you can
use git commit --no-verify
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-11 23:20 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Andres Freund @ 2023-08-11 23:20 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Hi,
On 2023-08-11 18:30:02 -0400, Tom Lane wrote:
> Peter Geoghegan <[email protected]> writes:
> > On Fri, Aug 11, 2023 at 2:25 PM Andres Freund <[email protected]> wrote:
> >> We could a test that fails when there's some mis-indented code. That seems
> >> like it might catch things earlier?
>
> +1 for including this in CI tests
I didn't even mean CI - I meant 'make check-world' / 'meson test'. Which of
course would include CI automatically.
> > It definitely would. That would go a long way towards addressing my
> > concerns. But I suspect that that would run into problems that stem
> > from the fact that the buildfarm is testing something that isn't all
> > that simple. Don't typedefs need to be downloaded from some other
> > blessed buildfarm animal?
>
> No. I presume koel is using src/tools/pgindent/typedefs.list,
> which has always been the "canonical" list but up to now we've
> been lazy about maintaining it. Part of the new regime is that
> typedefs.list should now be updated on-the-fly by patches that
> add new typedefs.
Yea. Otherwise nobody else can indent reliably, without repeating the work of
adding typedefs.list entries of all the patches since the last time it was
updated in the repository.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-12 00:11 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Tom Lane @ 2023-08-12 00:11 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Andres Freund <[email protected]> writes:
> On 2023-08-11 18:30:02 -0400, Tom Lane wrote:
>> +1 for including this in CI tests
> I didn't even mean CI - I meant 'make check-world' / 'meson test'. Which of
> course would include CI automatically.
Hmm. I'm allergic to anything that significantly increases the cost
of check-world, and this seems like it'd do that.
Maybe we could automate it, but not as part of check-world per se?
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-12 00:46 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Andres Freund @ 2023-08-12 00:46 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Hi,
On 2023-08-11 20:11:34 -0400, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
> > On 2023-08-11 18:30:02 -0400, Tom Lane wrote:
> >> +1 for including this in CI tests
>
> > I didn't even mean CI - I meant 'make check-world' / 'meson test'. Which of
> > course would include CI automatically.
>
> Hmm. I'm allergic to anything that significantly increases the cost
> of check-world, and this seems like it'd do that.
Hm, compared to the cost of check-world it's not that large, but still,
annoying to make it larger.
We can make it lot cheaper, but perhaps not in a general enough fashion that
it's suitable for a test.
pgindent already can query git (for --commit). We could teach pgindent to
ask git what remote branch is being tracked, and constructed a list of files
of the difference between the remote branch and the local branch?
That option could do something like:
git diff --name-only $(git rev-parse --abbrev-ref --symbolic-full-name @{upstream})
That's pretty quick, even for a relatively large delta.
> Maybe we could automate it, but not as part of check-world per se?
We should definitely do that. Another related thing that'd be useful to
script is updating typedefs.list with the additional typedefs found
locally. Right now the script for that still lives in the
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-12 15:57 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-08-12 15:57 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Peter Geoghegan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-08-11 Fr 19:17, Tom Lane wrote:
> Peter Geoghegan<[email protected]> writes:
>> I'm starting to have doubts about this policy. There have now been
>> quite a few follow-up "fixes" to indentation issues that koel
>> complained about. None of these fixups have been included in
>> .git-blame-ignore-revs. If things continue like this then "git blame"
>> is bound to become much less usable over time.
> FWIW, I'm much more optimistic than that. I think what we're seeing
> is just the predictable result of not all committers having yet
> incorporated "pgindent it before committing" into their workflow.
> The need for followup fixes should diminish as people start doing
> that. If you want to hurry things along, peer pressure on committers
> who clearly aren't bothering is the solution.
Yeah, part of the point of creating koel was to give committers a bit of
a nudge in that direction.
With a git pre-commit hook it's pretty painless.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-12 21:03 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-08-12 21:03 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Peter Geoghegan <[email protected]>; +Cc: Andres Freund <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-08-11 Fr 19:02, Tom Lane wrote:
> Peter Geoghegan<[email protected]> writes:
>> My workflow up until now has avoiding making updates to typedefs.list
>> in patches. I only update typedefs locally, for long enough to indent
>> my code. The final patch doesn't retain any typedefs.list changes.
> Yeah, I've done the same and will have to stop.
>
>> I guess that I can't do that anymore. Hopefully maintaining the
>> typedefs.list file isn't as inconvenient as it once seemed to me to
>> be.
> I don't think it'll be a problem. If your rule is "add new typedef
> names added by your patch to typedefs.list, keeping them in
> alphabetical order" then it doesn't seem very complicated, and
> hopefully conflicts between concurrently-developed patches won't
> be common.
>
>
My recollection is that missing typedefs cause indentation that kinda
sticks out like a sore thumb.
The reason we moved to a buildfarm based typedefs list was that some
typedefs are platform dependent, so any list really needs to be the
union of the found typedefs on various platforms, and the buildfarm was
a convenient vehicle for doing that. But that doesn't mean you shouldn't
manually add a typedef you have added in your code.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-12 21:14 Andres Freund <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Andres Freund @ 2023-08-12 21:14 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Geoghegan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Hi,
On 2023-08-12 17:03:37 -0400, Andrew Dunstan wrote:
> On 2023-08-11 Fr 19:02, Tom Lane wrote:
> > Peter Geoghegan<[email protected]> writes:
> > > My workflow up until now has avoiding making updates to typedefs.list
> > > in patches. I only update typedefs locally, for long enough to indent
> > > my code. The final patch doesn't retain any typedefs.list changes.
> > Yeah, I've done the same and will have to stop.
> >
> > > I guess that I can't do that anymore. Hopefully maintaining the
> > > typedefs.list file isn't as inconvenient as it once seemed to me to
> > > be.
> > I don't think it'll be a problem. If your rule is "add new typedef
> > names added by your patch to typedefs.list, keeping them in
> > alphabetical order" then it doesn't seem very complicated, and
> > hopefully conflicts between concurrently-developed patches won't
> > be common.
>
> My recollection is that missing typedefs cause indentation that kinda sticks
> out like a sore thumb.
>
> The reason we moved to a buildfarm based typedefs list was that some
> typedefs are platform dependent, so any list really needs to be the union of
> the found typedefs on various platforms, and the buildfarm was a convenient
> vehicle for doing that. But that doesn't mean you shouldn't manually add a
> typedef you have added in your code.
It's a somewhat annoying task though, find all the typedefs, add them to the
right place in the file (we have an out of order entry right now). I think a
script that *adds* (but doesn't remove) local typedefs would make this less
painful.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-12 22:46 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-08-12 22:46 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Peter Geoghegan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Andres Freund <[email protected]> writes:
> On 2023-08-12 17:03:37 -0400, Andrew Dunstan wrote:
>> My recollection is that missing typedefs cause indentation that kinda sticks
>> out like a sore thumb.
Yeah, it's usually pretty obvious: "typedef *var" gets changed to
"typedef * var", and similar oddities.
> It's a somewhat annoying task though, find all the typedefs, add them to the
> right place in the file (we have an out of order entry right now). I think a
> script that *adds* (but doesn't remove) local typedefs would make this less
> painful.
My practice has always been "add typedefs until pgindent doesn't do
anything I don't want". If you have a new typedef that doesn't happen
to be used in a way that pgindent mangles, it's not that critical
to get it into the file right away.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-13 00:13 Peter Geoghegan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Peter Geoghegan @ 2023-08-13 00:13 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sat, Aug 12, 2023 at 3:47 PM Tom Lane <[email protected]> wrote:
> > It's a somewhat annoying task though, find all the typedefs, add them to the
> > right place in the file (we have an out of order entry right now). I think a
> > script that *adds* (but doesn't remove) local typedefs would make this less
> > painful.
>
> My practice has always been "add typedefs until pgindent doesn't do
> anything I don't want". If you have a new typedef that doesn't happen
> to be used in a way that pgindent mangles, it's not that critical
> to get it into the file right away.
We seem to be seriously contemplating making every patch author do
this every time they need to get the tests to pass (after adding or
renaming a struct). Is that really an improvement over the old status
quo?
In principle I'm in favor of strictly enforcing indentation rules like
this. But it seems likely that our current tooling just isn't up to
the task.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-13 00:20 Tom Lane <[email protected]>
parent: Peter Geoghegan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-08-13 00:20 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Peter Geoghegan <[email protected]> writes:
> We seem to be seriously contemplating making every patch author do
> this every time they need to get the tests to pass (after adding or
> renaming a struct). Is that really an improvement over the old status
> quo?
Hm. I was envisioning that we should expect committers to deal
with this, not original patch submitters. So that's an argument
against including it in the CI tests. But I'm in favor of anything
we can do to make it more painless for committers to fix up patch
indentation.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-13 00:53 Peter Geoghegan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Peter Geoghegan @ 2023-08-13 00:53 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sat, Aug 12, 2023 at 5:20 PM Tom Lane <[email protected]> wrote:
> Hm. I was envisioning that we should expect committers to deal
> with this, not original patch submitters. So that's an argument
> against including it in the CI tests. But I'm in favor of anything
> we can do to make it more painless for committers to fix up patch
> indentation.
Making it a special responsibility for committers comes with the same
set of problems that we see with catversion bumps. People are much
more likely to forget to do something that must happen last.
Maybe I'm wrong -- maybe the new policy is practicable. It might even
turn out to be worth the bother. Time will tell.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-13 14:33 Andrew Dunstan <[email protected]>
parent: Peter Geoghegan <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Andrew Dunstan @ 2023-08-13 14:33 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-08-12 Sa 20:53, Peter Geoghegan wrote:
> On Sat, Aug 12, 2023 at 5:20 PM Tom Lane<[email protected]> wrote:
>> Hm. I was envisioning that we should expect committers to deal
>> with this, not original patch submitters. So that's an argument
>> against including it in the CI tests. But I'm in favor of anything
>> we can do to make it more painless for committers to fix up patch
>> indentation.
I agree with this.
> Making it a special responsibility for committers comes with the same
> set of problems that we see with catversion bumps. People are much
> more likely to forget to do something that must happen last.
After I'd been caught by this once or twice I implemented a git hook
test for that too - in fact it was the first hook I did. It's not
perfect but it's saved me a couple of times:
check_catalog_version () {
# only do this on master
test "$branch" = "master" || return 0
case "$files" in
*src/include/catalog/catversion.h*)
return 0;
;;
*src/include/catalog/*)
;;
*)
return 0;
;;
esac
# changes include catalog but not catversion.h, so warn about it
{
echo 'Commit on master alters catalog but catversion not bumped'
echo 'It can be forced with git commit --no-verify'
} >&2
exit 1
}
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-13 23:33 Michael Paquier <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Michael Paquier @ 2023-08-13 23:33 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sun, Aug 13, 2023 at 10:33:21AM -0400, Andrew Dunstan wrote:
> After I'd been caught by this once or twice I implemented a git hook test
> for that too - in fact it was the first hook I did. It's not perfect but
> it's saved me a couple of times:
>
> check_catalog_version () {
I find that pretty cool. Thanks for sharing.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../ZNloVL4%[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-14 14:04 Peter Eisentraut <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Peter Eisentraut @ 2023-08-14 14:04 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Geoghegan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 12.08.23 23:14, Andres Freund wrote:
> It's a somewhat annoying task though, find all the typedefs, add them to the
> right place in the file (we have an out of order entry right now). I think a
> script that*adds* (but doesn't remove) local typedefs would make this less
> painful.
I was puzzled once that there does not appear to be such a script
available. Whatever the buildfarm does (before it merges it all
together) should be available locally. Then the workflow could be
type type type
compile
update typedefs
pgindent
commit
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-14 14:08 Peter Eisentraut <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Peter Eisentraut @ 2023-08-14 14:08 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Andres Freund <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 12.08.23 02:11, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
>> On 2023-08-11 18:30:02 -0400, Tom Lane wrote:
>>> +1 for including this in CI tests
>
>> I didn't even mean CI - I meant 'make check-world' / 'meson test'. Which of
>> course would include CI automatically.
>
> Hmm. I'm allergic to anything that significantly increases the cost
> of check-world, and this seems like it'd do that.
>
> Maybe we could automate it, but not as part of check-world per se?
Also, during development, the code in progress is not always perfectly
formatted, but I do want to keep running the test suites.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-14 19:58 Andrew Dunstan <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-08-14 19:58 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Geoghegan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-08-14 Mo 10:04, Peter Eisentraut wrote:
> On 12.08.23 23:14, Andres Freund wrote:
>> It's a somewhat annoying task though, find all the typedefs, add them
>> to the
>> right place in the file (we have an out of order entry right now). I
>> think a
>> script that*adds* (but doesn't remove) local typedefs would make
>> this less
>> painful.
>
> I was puzzled once that there does not appear to be such a script
> available. Whatever the buildfarm does (before it merges it all
> together) should be available locally. Then the workflow could be
>
> type type type
> compile
> update typedefs
> pgindent
> commit
It's a bit more complicated :-)
You can see what the buildfarm does at
<https://github.com/PGBuildFarm/client-code/blob/ec4cf43613a74cb88f228efcde09931cf9fd57e7/run_build.p...;
It's been somewhat fragile over the years, which most people other than
Tom and I have probably not noticed.
On most platforms it needs postgres to have been installed before
looking for the typedefs.
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-15 20:31 Nathan Bossart <[email protected]>
parent: Peter Geoghegan <[email protected]>
3 siblings, 1 reply; 142+ messages in thread
From: Nathan Bossart @ 2023-08-15 20:31 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Fri, Aug 11, 2023 at 01:59:40PM -0700, Peter Geoghegan wrote:
> I'm starting to have doubts about this policy. There have now been
> quite a few follow-up "fixes" to indentation issues that koel
> complained about. None of these fixups have been included in
> .git-blame-ignore-revs. If things continue like this then "git blame"
> is bound to become much less usable over time.
Should we add those? Patch attached.
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
Attachments:
[text/x-diff] v1-0001-Add-a-few-recent-commits-to-.git-blame-ignore-rev.patch (1.6K, ../../20230815203109.GA2596919@nathanxps13/2-v1-0001-Add-a-few-recent-commits-to-.git-blame-ignore-rev.patch)
download | inline diff:
From f4f158e153f9027330ce841ca79b5650dfd24a0e Mon Sep 17 00:00:00 2001
From: Nathan Bossart <[email protected]>
Date: Tue, 15 Aug 2023 13:24:18 -0700
Subject: [PATCH v1 1/1] Add a few recent commits to .git-blame-ignore-revs.
---
.git-blame-ignore-revs | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/.git-blame-ignore-revs b/.git-blame-ignore-revs
index 00b548c7b0..2ff86c8d0d 100644
--- a/.git-blame-ignore-revs
+++ b/.git-blame-ignore-revs
@@ -14,6 +14,27 @@
#
# $ git log --pretty=format:"%H # %cd%n# %s" $PGINDENTGITHASH -1 --date=iso
+89be0b89ae60c63856fd26d82a104781540e2312 # 2023-08-15 17:45:00 +0900
+# Fix code indentation vioaltion introduced in commit 9e9931d2b.
+
+5dc456b7dda4f7d0d7735b066607c190c74d174a # 2023-08-11 20:43:34 +0900
+# Fix code indentation violations introduced by recent commit
+
+62e9af4c63fbd36fb9af8450fb44bece76d7766f # 2023-07-25 12:35:58 +0530
+# Fix code indentation vioaltion introduced in commit d38ad8e31d.
+
+4e465aac36ce9a9533c68dbdc83e67579880e628 # 2023-07-18 14:04:31 +0900
+# Fix indentation in twophase.c
+
+328f492d2565cfbe383f13a69425d751fd79415f # 2023-07-13 22:26:10 +0900
+# Fix code indentation violation in commit b6e1157e7d
+
+69a674a170058e63e8178aec8a36a673efce8801 # 2023-07-06 11:49:18 +0530
+# Fix code indentation vioaltion introduced in commit cc32ec24fd.
+
+a4cfeeca5a97f2b5969c31aa69ba775af95ee5a3 # 2023-07-03 12:47:49 +0200
+# Fix code indentation violations
+
b334612b8aee9f9a34378982d8938b201dfad323 # 2023-06-20 09:50:43 -0400
# Pre-beta2 mechanical code beautification.
--
2.25.1
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-16 20:15 Peter Geoghegan <[email protected]>
parent: Nathan Bossart <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Peter Geoghegan @ 2023-08-16 20:15 UTC (permalink / raw)
To: Nathan Bossart <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, Aug 15, 2023 at 1:31 PM Nathan Bossart <[email protected]> wrote:
> Should we add those? Patch attached.
I think that that makes sense. I just don't want to normalize updating
.git-blame-ignore-revs very frequently. (Actually, it's more like I
don't want to normalize any scheme that makes updating the ignore list
very frequently start to seem reasonable.)
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-08-17 14:40 Nathan Bossart <[email protected]>
parent: Peter Geoghegan <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Nathan Bossart @ 2023-08-17 14:40 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Tom Lane <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Wed, Aug 16, 2023 at 01:15:55PM -0700, Peter Geoghegan wrote:
> On Tue, Aug 15, 2023 at 1:31 PM Nathan Bossart <[email protected]> wrote:
>> Should we add those? Patch attached.
>
> I think that that makes sense.
Committed.
> I just don't want to normalize updating
> .git-blame-ignore-revs very frequently. (Actually, it's more like I
> don't want to normalize any scheme that makes updating the ignore list
> very frequently start to seem reasonable.)
Agreed. I've found myself habitually running pgindent since becoming a
committer, but I'm sure I'll forget it one of these days. From a quick
skim of this thread, it sounds like a pre-commit hook [0] might be the best
option at the moment.
[0] https://wiki.postgresql.org/wiki/Working_with_Git#Using_git_hooks
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-16 00:52 Peter Geoghegan <[email protected]>
parent: Peter Geoghegan <[email protected]>
1 sibling, 2 replies; 142+ messages in thread
From: Peter Geoghegan @ 2023-10-16 00:52 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sat, Aug 12, 2023 at 5:53 PM Peter Geoghegan <[email protected]> wrote:
> Maybe I'm wrong -- maybe the new policy is practicable. It might even
> turn out to be worth the bother. Time will tell.
(Two months pass.)
There were two independent fixup commits to address complaints from
koel just today (from two different committers). Plus there was a
third issue (involving a third committer) this past Wednesday.
This policy isn't working.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 00:45 Tom Lane <[email protected]>
parent: Peter Geoghegan <[email protected]>
1 sibling, 3 replies; 142+ messages in thread
From: Tom Lane @ 2023-10-17 00:45 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Peter Geoghegan <[email protected]> writes:
> There were two independent fixup commits to address complaints from
> koel just today (from two different committers). Plus there was a
> third issue (involving a third committer) this past Wednesday.
> This policy isn't working.
Two thoughts about that:
1. We should not commit indent fixups on behalf of somebody else's
misindented commit. Peer pressure on committers who aren't being
careful about this is the only way to improve matters; so complain
to the person at fault until they fix it.
2. We could raise awareness of this issue by adding indent verification
to CI testing. I hesitate to suggest that, though, for a couple of
reasons:
2a. It seems fairly expensive, though I might be misjudging.
2b. It's often pretty handy to submit patches that aren't fully
indent-clean; I have such a patch in flight right now at [1].
2b could be ameliorated by making the indent check be a separate
test process that doesn't obscure the results of other testing.
regards, tom lane
[1] https://www.postgresql.org/message-id/2617358.1697501956%40sss.pgh.pa.us
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 01:22 Peter Geoghegan <[email protected]>
parent: Tom Lane <[email protected]>
2 siblings, 0 replies; 142+ messages in thread
From: Peter Geoghegan @ 2023-10-17 01:22 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Mon, Oct 16, 2023 at 5:45 PM Tom Lane <[email protected]> wrote:
> Two thoughts about that:
>
> 1. We should not commit indent fixups on behalf of somebody else's
> misindented commit. Peer pressure on committers who aren't being
> careful about this is the only way to improve matters; so complain
> to the person at fault until they fix it.
Thomas Munro's recent commit 01529c704008 was added to
.git-blame-ignore-revs by Michael Paquier, despite the fact that
Munro's commit technically isn't just a pure indentation fix (it also
fixed some typos). It's hard to judge Michael too harshly for this,
since in general it's harder to commit things when koel is already
complaining about existing misindetation -- I get why he'd prefer to
take care of that first.
> 2. We could raise awareness of this issue by adding indent verification
> to CI testing. I hesitate to suggest that, though, for a couple of
> reasons:
> 2a. It seems fairly expensive, though I might be misjudging.
> 2b. It's often pretty handy to submit patches that aren't fully
> indent-clean; I have such a patch in flight right now at [1].
It's also often handy to make a minor change to a comment or something
at the last minute, without necessarily having the comment indented
perfectly.
> 2b could be ameliorated by making the indent check be a separate
> test process that doesn't obscure the results of other testing.
I was hoping that "go back to the old status quo" would also appear as
an option.
My main objection to the new policy is that it's not quite clear what
process I should go through in order to be 100% confident that koel
won't start whining (short of waiting around for koel to whine). I
know how to run pgindent, of course, and haven't had any problems so
far...but it still seems quite haphazard. If we're going to make this
a hard rule, enforced on every commit, it should be dead easy to
comply with the rule.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 02:57 Michael Paquier <[email protected]>
parent: Tom Lane <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: Michael Paquier @ 2023-10-17 02:57 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Mon, Oct 16, 2023 at 08:45:00PM -0400, Tom Lane wrote:
> 2. We could raise awareness of this issue by adding indent verification
> to CI testing. I hesitate to suggest that, though, for a couple of
> reasons:
> 2a. It seems fairly expensive, though I might be misjudging.
> 2b. It's often pretty handy to submit patches that aren't fully
> indent-clean; I have such a patch in flight right now at [1].
>
> 2b could be ameliorated by making the indent check be a separate
> test process that doesn't obscure the results of other testing.
I see an extra reason with not doing that: this increases the
difficulty when it comes to send and maintain patches to the lists and
newcomers would need to learn more tooling. I don't think that we
should make that more complicated for code-formatting reasons.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 10:49 Jelte Fennema <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Jelte Fennema @ 2023-10-17 10:49 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Geoghegan <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, 17 Oct 2023 at 04:57, Michael Paquier <[email protected]> wrote:
> I see an extra reason with not doing that: this increases the
> difficulty when it comes to send and maintain patches to the lists and
> newcomers would need to learn more tooling. I don't think that we
> should make that more complicated for code-formatting reasons.
Honestly, I don't think it's a huge hurdle for newcomers. Most open
source projects have a CI job that runs automatic code formatting, so
it's a pretty common thing for contributors to deal with. And as long
as we keep it a separate CI job from the normal tests, committers can
even choose to commit the patch if the formatting job fails, after
running pgindent themselves.
And personally as a contributor it's a much nicer experience to see
quickly in CI that I messed up the code style, then to hear it a
week/month later in an email when someone took the time to review and
mentions the styling is way off all over the place.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 12:45 Robert Haas <[email protected]>
parent: Peter Geoghegan <[email protected]>
1 sibling, 3 replies; 142+ messages in thread
From: Robert Haas @ 2023-10-17 12:45 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Tom Lane <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Sun, Oct 15, 2023 at 8:52 PM Peter Geoghegan <[email protected]> wrote:
> On Sat, Aug 12, 2023 at 5:53 PM Peter Geoghegan <[email protected]> wrote:
> > Maybe I'm wrong -- maybe the new policy is practicable. It might even
> > turn out to be worth the bother. Time will tell.
>
> (Two months pass.)
>
> There were two independent fixup commits to address complaints from
> koel just today (from two different committers). Plus there was a
> third issue (involving a third committer) this past Wednesday.
>
> This policy isn't working.
+1. I think this is more annoying than the status quo ante.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 14:03 Robert Haas <[email protected]>
parent: Robert Haas <[email protected]>
2 siblings, 0 replies; 142+ messages in thread
From: Robert Haas @ 2023-10-17 14:03 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Tom Lane <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, Oct 17, 2023 at 8:45 AM Robert Haas <[email protected]> wrote:
> > This policy isn't working.
>
> +1. I think this is more annoying than the status quo ante.
Although ... I do think it's spared me some rebasing pain, and that
does have some real value. I wonder if we could think of other
alternatives. For example, maybe we could have a bot. If you push a
commit that's not indented properly, the bot reindents the tree,
updates git-blame-ignore-revs, and sends you an email admonishing you
for your error. Or we could have a server-side hook that will refuse
the misindented commit, with some kind of override for emergency
situations. What I really dislike about the current situation is that
it's doubling down on the idea that committers have to be perfect and
get everything right every time. Turns out, that's hard to do. If not,
why do people keep screwing things up? Somebody could theorize - and
this seems to be Tom and Jelte's theory, though perhaps I'm
misinterpreting their comments - that the people who have made
mistakes here are just lazy, and what they need to do is up their
game.
But I don't buy that. First, I think that most of our committers are
pretty intelligent and hard-working people who are trying to do the
right thing. We can't all be Tom Lane, no matter how hard we may try.
Second, even if it were true that the offending committers are "just
lazy," all of our contributors and many senior non-committer
contributors are people who have put thousands, if not tens of
thousands, of hours into the project. Making them feel bad serves us
poorly. At the end of the day, it doesn't matter whether it's too much
of a pain for the perfect committers we'd like to have. It matters
whether it's too much of a pain for the human committers that we do
have.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 14:23 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
2 siblings, 4 replies; 142+ messages in thread
From: Tom Lane @ 2023-10-17 14:23 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Robert Haas <[email protected]> writes:
> On Tue, Oct 17, 2023 at 8:45 AM Robert Haas <[email protected]> wrote:
>> +1. I think this is more annoying than the status quo ante.
> Although ... I do think it's spared me some rebasing pain, and that
> does have some real value. I wonder if we could think of other
> alternatives.
An alternative I was thinking about after reading your earlier email
was going back to the status quo ante, but doing the manual tree-wide
reindents significantly more often than once a year. Adding one at
the conclusion of each commitfest would be a natural thing to do,
for instance. It's hard to say what frequency would lead to the
least rebasing pain, but we know once-a-year isn't ideal.
> For example, maybe we could have a bot. If you push a
> commit that's not indented properly, the bot reindents the tree,
> updates git-blame-ignore-revs, and sends you an email admonishing you
> for your error.
I'm absolutely not in favor of completely-automated reindents.
pgindent is a pretty stupid tool and it will sometimes do stupid
things, which you have to correct for by tweaking the input
formatting. The combination of the tool and human supervision
generally produces pretty good results, but the tool alone
not so much.
> Or we could have a server-side hook that will refuse
> the misindented commit, with some kind of override for emergency
> situations.
Even though I'm in the camp that would like the tree correctly
indented at all times, I remain very much against a commit hook.
I think that'd be significantly more annoying than the current
situation, which you're already unhappy about the annoying-ness of.
The bottom line here, I think, is that there's a subset of committers
that would like perfectly mechanically-indented code at all times,
and there's another subset that just doesn't care that much.
We don't (and shouldn't IMO) have a mechanism to let one set force
their views on the other set. The current approach is clearly
insufficient for that, and I don't think trying to institute stronger
enforcement is going to make anybody happy.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 14:55 Peter Geoghegan <[email protected]>
parent: Tom Lane <[email protected]>
3 siblings, 0 replies; 142+ messages in thread
From: Peter Geoghegan @ 2023-10-17 14:55 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, Oct 17, 2023 at 7:23 AM Tom Lane <[email protected]> wrote:
> Robert Haas <[email protected]> writes:
> > On Tue, Oct 17, 2023 at 8:45 AM Robert Haas <[email protected]> wrote:
> >> +1. I think this is more annoying than the status quo ante.
>
> > Although ... I do think it's spared me some rebasing pain, and that
> > does have some real value. I wonder if we could think of other
> > alternatives.
>
> An alternative I was thinking about after reading your earlier email
> was going back to the status quo ante, but doing the manual tree-wide
> reindents significantly more often than once a year. Adding one at
> the conclusion of each commitfest would be a natural thing to do,
> for instance. It's hard to say what frequency would lead to the
> least rebasing pain, but we know once-a-year isn't ideal.
That seems like the best alternative we have. The old status quo did
occasionally allow code with indentation that *clearly* wasn't up to
project standards to slip in. It could stay that way for quite a few
months at a time. That wasn't great either.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 15:00 Jelte Fennema <[email protected]>
parent: Tom Lane <[email protected]>
3 siblings, 0 replies; 142+ messages in thread
From: Jelte Fennema @ 2023-10-17 15:00 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Peter Geoghegan <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, 17 Oct 2023 at 16:23, Tom Lane <[email protected]> wrote:
> > Or we could have a server-side hook that will refuse
> > the misindented commit, with some kind of override for emergency
> > situations.
>
> Even though I'm in the camp that would like the tree correctly
> indented at all times, I remain very much against a commit hook.
> I think that'd be significantly more annoying than the current
> situation, which you're already unhappy about the annoying-ness of.
Why do you think that would be significantly more annoying than the
current situation? Instead of getting delayed feedback you get instant
feedback when you push.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 15:01 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
3 siblings, 2 replies; 142+ messages in thread
From: Robert Haas @ 2023-10-17 15:01 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, Oct 17, 2023 at 10:23 AM Tom Lane <[email protected]> wrote:
> An alternative I was thinking about after reading your earlier email
> was going back to the status quo ante, but doing the manual tree-wide
> reindents significantly more often than once a year. Adding one at
> the conclusion of each commitfest would be a natural thing to do,
> for instance. It's hard to say what frequency would lead to the
> least rebasing pain, but we know once-a-year isn't ideal.
Yes. I suspect once a commitfest still wouldn't be often enough. Maybe
once a month or something would be. But I'm not sure. You might rebase
once over the misindented commit and then have to rebase again over
the indent that fixed it. There's not really anything that quite
substitutes for doing it right on every commit.
> The bottom line here, I think, is that there's a subset of committers
> that would like perfectly mechanically-indented code at all times,
> and there's another subset that just doesn't care that much.
> We don't (and shouldn't IMO) have a mechanism to let one set force
> their views on the other set. The current approach is clearly
> insufficient for that, and I don't think trying to institute stronger
> enforcement is going to make anybody happy.
I mean, I think we DO have such a mechanism. Everyone agrees that the
buildfarm has to stay green, and we have a buildfarm member that
checks pgindent, so that means everyone has to pgindent. We could
decide to kill that buildfarm member, in which case we go back to
people not having to pgindent, but right now they do.
And if it's going to remain the policy, it's better to enforce that
policy earlier rather than later. I mean, what is the point of having
a system where we let people do the wrong thing and then publicly
embarrass them afterwards? How is that better than preventing them
from doing the wrong thing in the first place? Even if they don't
subjectively feel embarrassed, nobody likes having to log back on in
the evening or the weekend and clean up after something they thought
they were done with.
In fact, that particular experience is one of the worst things about
being a committer. It actively discourages me, at least, from trying
to get other people's patches committed. This particular problem is
minor, but the overall experience of trying to get things committed is
that you have to check 300 things for every patch and if you get every
one of them right then nothing happens and if you get one of them
wrong then you get a bunch of irritated emails criticizing your
laziness, sloppiness, or whatever, and you have to drop everything to
go fix it immediately. What a deal! I'm sure this isn't the only
reason why we have such a huge backlog of patches needing committer
attention, but it sure doesn't help. And there is absolutely zero need
for this to be yet another thing that you can find out you did wrong
in the 1-24 hour period AFTER you type 'git push'.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 15:15 Peter Geoghegan <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Peter Geoghegan @ 2023-10-17 15:15 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, Oct 17, 2023 at 8:01 AM Robert Haas <[email protected]> wrote:
> In fact, that particular experience is one of the worst things about
> being a committer. It actively discourages me, at least, from trying
> to get other people's patches committed. This particular problem is
> minor, but the overall experience of trying to get things committed is
> that you have to check 300 things for every patch and if you get every
> one of them right then nothing happens and if you get one of them
> wrong then you get a bunch of irritated emails criticizing your
> laziness, sloppiness, or whatever, and you have to drop everything to
> go fix it immediately. What a deal!
Yep. Enforcing perfect indentation on koel necessitates rechecking
indentation after each and every last-minute fixup affecting C code --
the interactions makes it quite a bit harder to get everything right
on the first push. For example, if I spot an issue with a comment
during final pre-commit review, and fixup that commit, I have to run
pgindent again. On a long enough timeline, I'm going to forget to do
that.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 15:23 Robert Haas <[email protected]>
parent: Peter Geoghegan <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Robert Haas @ 2023-10-17 15:23 UTC (permalink / raw)
To: Peter Geoghegan <[email protected]>; +Cc: Tom Lane <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, Oct 17, 2023 at 11:16 AM Peter Geoghegan <[email protected]> wrote:
> Yep. Enforcing perfect indentation on koel necessitates rechecking
> indentation after each and every last-minute fixup affecting C code --
> the interactions makes it quite a bit harder to get everything right
> on the first push. For example, if I spot an issue with a comment
> during final pre-commit review, and fixup that commit, I have to run
> pgindent again. On a long enough timeline, I'm going to forget to do
> that.
I also just discovered that my pre-commit hook doesn't work if I pull
commits into master by cherry-picking. I had thought that I could have
my hook just check my commits to master and not all of my local dev
branches where I really don't want to mess with this when I'm just
banging out a rough draft of something. But now I see that I'm going
to need to work harder on this if I actually want it to catch all the
ways I might screw this up.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-17 15:47 Peter Geoghegan <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Peter Geoghegan @ 2023-10-17 15:47 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, Oct 17, 2023 at 8:24 AM Robert Haas <[email protected]> wrote:
> I also just discovered that my pre-commit hook doesn't work if I pull
> commits into master by cherry-picking. I had thought that I could have
> my hook just check my commits to master and not all of my local dev
> branches where I really don't want to mess with this when I'm just
> banging out a rough draft of something. But now I see that I'm going
> to need to work harder on this if I actually want it to catch all the
> ways I might screw this up.
Once you figure all that out, you're still obligated to hand-polish
typedefs.list to be consistent with whatever Bruce's machine's copy of
objdump does (or is it Tom's?). You need to sort the entries so they
kinda look like they originated from the same source as existing
entries, since my Debian machine seems to produce somewhat different
results to RHEL, for whatever reason. It's hard to imagine a worse use
of committer time.
I think that something like this new policy could work if the
underlying tooling was very easy to use and gave perfectly consistent
results on everybody's development machine. Obviously, that just isn't
the case.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-18 04:40 David Rowley <[email protected]>
parent: Robert Haas <[email protected]>
2 siblings, 2 replies; 142+ messages in thread
From: David Rowley @ 2023-10-18 04:40 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Wed, 18 Oct 2023 at 01:47, Robert Haas <[email protected]> wrote:
> On Sat, Aug 12, 2023 at 5:53 PM Peter Geoghegan <[email protected]> wrote:
> > This policy isn't working.
>
> +1. I think this is more annoying than the status quo ante.
Maybe there are two camps of committers here; ones who care about
committing correctly indented code and ones who do not.
I don't mean that in a bad way, but if a committer just does not care
about correctly pgindented code then he/she likely didn't suffer
enough pain from how things used to be... having to unindent all the
unrelated indent fixes that were committed since the last pgindent run
I personally found slow/annoying/error-prone.
What I do now seems significantly easier. Assuming it's just 1 commit, just:
perl src/tools/pgindent/pgindent --commit HEAD
git diff # manual check to see if everything looks sane.
git commit -a --fixup HEAD
git rebase -i HEAD~2
If we were to go back to how it was before, then why should I go to
the trouble of unindenting all the unrelated indents from code changed
by other committers since the last pgindent run when those committers
are not bothering to and making my job harder each time they commit
incorrectly indented code.
How many of the committers who have broken koel are repeat offenders?
What is their opinion on this?
Did they just forget once or do they hate the process and want to go back?
I'm personally very grateful for all the work that was done to improve
pgindent and set out the new process. I'd really rather not go back to
how things were.
I agree that it's not nice to add yet another way of breaking the
buildfarm and even more so when the committer did make check-world
before committing. We have --enable-tap-tests, we could have
--enable-indent-checks and have pgindent check the code is correctly
indented during make check-world. Then just not have
--enable-indent-checks in CI.
I think many of us have scripts we use instead of typing out all the
configure options we want. It's likely pretty easy to add
--enable-indent-checks to those.
David
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-18 07:20 Peter Eisentraut <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Peter Eisentraut @ 2023-10-18 07:20 UTC (permalink / raw)
To: David Rowley <[email protected]>; Robert Haas <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 18.10.23 06:40, David Rowley wrote:
> I agree that it's not nice to add yet another way of breaking the
> buildfarm and even more so when the committer did make check-world
> before committing. We have --enable-tap-tests, we could have
> --enable-indent-checks and have pgindent check the code is correctly
> indented during make check-world. Then just not have
> --enable-indent-checks in CI.
This approach seems like a good improvement, even independent of
everything else we might do about this. Making it easier to use and
less likely to be forgotten. Also, this way, non-committer contributors
can opt-in, if they want to earn bonus points.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-18 09:07 Jelte Fennema <[email protected]>
parent: David Rowley <[email protected]>
1 sibling, 2 replies; 142+ messages in thread
From: Jelte Fennema @ 2023-10-18 09:07 UTC (permalink / raw)
To: David Rowley <[email protected]>; Peter Eisentraut <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; +Cc: Robert Haas <[email protected]>; Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Wed, 18 Oct 2023 at 06:40, David Rowley <[email protected]> wrote:
> How many of the committers who have broken koel are repeat offenders?
I just checked the commits and there don't seem to be real repeat
offenders. The maximum number of times someone broke koel since its
inception is two. That was the case for only two people. The other 8
people only caused one breakage.
> What is their opinion on this?
> Did they just forget once or do they hate the process and want to go back?
The commiters that broke koel since its inception are:
- Alexander Korotkov
- Amit Kapila
- Amit Langote
- Andres Freund
- Etsuro Fujita
- Jeff Davis
- Michael Paquier
- Peter Eisentraut
- Tatsuo Ishii
- Tomas Vondra
I included all of them in the To field of this message, in the hope
that they share their viewpoint. Because otherwise it stays guessing
what they think.
But based on the contents of the fixup commits a commonality seems to
be that the fixup only fixes a few lines, quite often touching only
comments. So it seems like the main reason for breaking koel is
forgetting to re-run pgindent after some final cleanup/wording
changes/typo fixes. And that seems like an expected flaw of being
human instead of a robot, which can only be worked around with better
automation.
> I agree that it's not nice to add yet another way of breaking the
> buildfarm and even more so when the committer did make check-world
> before committing. We have --enable-tap-tests, we could have
> --enable-indent-checks and have pgindent check the code is correctly
> indented during make check-world. Then just not have
> --enable-indent-checks in CI.
I think --enable-indent-checks sounds like a good improvement to the
status quo. But I'm not confident that it will help remove the cases
where only a comment needs to be re-indented. Do commiters really
always run check-world again when only changing a typo in a comment? I
know I probably wouldn't (or at least not always).
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-18 09:34 David Rowley <[email protected]>
parent: Jelte Fennema <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: David Rowley @ 2023-10-18 09:34 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; Robert Haas <[email protected]>; Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Wed, 18 Oct 2023 at 22:07, Jelte Fennema <[email protected]> wrote:
> But based on the contents of the fixup commits a commonality seems to
> be that the fixup only fixes a few lines, quite often touching only
> comments. So it seems like the main reason for breaking koel is
> forgetting to re-run pgindent after some final cleanup/wording
> changes/typo fixes. And that seems like an expected flaw of being
> human instead of a robot, which can only be worked around with better
> automation.
I wonder if you might just be assuming these were caused by
last-minute comment adjustments. I may have missed something on the
thread, but it could be that, or it could be due to the fact that
pgindent just simply does more adjustments to comments than it does
with code lines.
On Wed, 18 Oct 2023 at 06:40, David Rowley <[email protected]> wrote:
> > I agree that it's not nice to add yet another way of breaking the
> > buildfarm and even more so when the committer did make check-world
> > before committing. We have --enable-tap-tests, we could have
> > --enable-indent-checks and have pgindent check the code is correctly
> > indented during make check-world. Then just not have
> > --enable-indent-checks in CI.
>
> I think --enable-indent-checks sounds like a good improvement to the
> status quo. But I'm not confident that it will help remove the cases
> where only a comment needs to be re-indented. Do commiters really
> always run check-world again when only changing a typo in a comment? I
> know I probably wouldn't (or at least not always).
I can't speak for others, but I always make edits in a dev branch and
do "make check-world" before doing "git format-patch" before I "git
am" that patch into a clean repo. Before I push, I'll always run
"make check-world" again as sometimes master might have moved on a few
commits from where the dev branch was taken (perhaps I need to update
the expected output of some newly added EXPLAIN tests if say doing a
planner adjustment). I personally never adjust any code or comments
after the git am. I only sometimes adjust the commit message.
So, in theory at least, if --enable-indent-checks existed and I used
it, I shouldn't break koel... let's see if I just jinxed myself.
It would be good to learn how many of the committers out of the ones
you listed that --enable-indent-checks would have saved from breaking
koel.
David
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-18 13:04 Vik Fearing <[email protected]>
parent: Tom Lane <[email protected]>
3 siblings, 0 replies; 142+ messages in thread
From: Vik Fearing @ 2023-10-18 13:04 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Robert Haas <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 10/17/23 16:23, Tom Lane wrote:
> An alternative I was thinking about after reading your earlier email was
> going back to the status quo ante, but doing the manual tree-wide
> reindents significantly more often than once a year. Adding one at the
> conclusion of each commitfest would be a natural thing to do, for
> instance. It's hard to say what frequency would lead to the least
> rebasing pain, but we know once-a-year isn't ideal.
This is basically how the SQL Committee functions. The Change Proposals
(patches) submitted every meeting (commitfest) are always against the
documents as they exist after the application of papers (commits) from
the previous meeting.
One major difference is that Change Proposals are against the text, and
patches are against the code. It is not dissimilar to people saying
what our documentation should say, and then someone implementing that
change.
So I am in favor of a pgindent run *at least* at the end of each
commitfest, giving a full month for patch authors to rebase before the
next fest.
--
Vik Fearing
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-18 14:07 Robert Haas <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Robert Haas @ 2023-10-18 14:07 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: David Rowley <[email protected]>; Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Wed, Oct 18, 2023 at 3:21 AM Peter Eisentraut <[email protected]> wrote:
> On 18.10.23 06:40, David Rowley wrote:
> > I agree that it's not nice to add yet another way of breaking the
> > buildfarm and even more so when the committer did make check-world
> > before committing. We have --enable-tap-tests, we could have
> > --enable-indent-checks and have pgindent check the code is correctly
> > indented during make check-world. Then just not have
> > --enable-indent-checks in CI.
>
> This approach seems like a good improvement, even independent of
> everything else we might do about this. Making it easier to use and
> less likely to be forgotten. Also, this way, non-committer contributors
> can opt-in, if they want to earn bonus points.
Yeah. I'm not going to push anything that doesn't pass make
check-world, so this is appealing in that something that I'm already
doing would (or could be configured to) catch this problem also.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-18 16:45 Bruce Momjian <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Bruce Momjian @ 2023-10-18 16:45 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Peter Geoghegan <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On Tue, Oct 17, 2023 at 11:01:44AM -0400, Robert Haas wrote:
> In fact, that particular experience is one of the worst things about
> being a committer. It actively discourages me, at least, from trying
> to get other people's patches committed. This particular problem is
> minor, but the overall experience of trying to get things committed is
> that you have to check 300 things for every patch and if you get every
> one of them right then nothing happens and if you get one of them
> wrong then you get a bunch of irritated emails criticizing your
> laziness, sloppiness, or whatever, and you have to drop everything to
> go fix it immediately. What a deal! I'm sure this isn't the only
> reason why we have such a huge backlog of patches needing committer
> attention, but it sure doesn't help. And there is absolutely zero need
> for this to be yet another thing that you can find out you did wrong
> in the 1-24 hour period AFTER you type 'git push'.
This comment resonated with me. I do all my git operations with shell
scripts so I can check for all the mistakes I have made in the past and
generate errors. Even with all of that, committing is an
anxiety-producing activity because any small mistake is quickly revealed
to the world. There aren't many things I do in a day where mistakes are
so impactful.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
Only you can decide what is important to you.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-18 19:15 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
2 siblings, 1 reply; 142+ messages in thread
From: Andres Freund @ 2023-10-18 19:15 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Hi,
On 2023-10-16 20:45:00 -0400, Tom Lane wrote:
> Peter Geoghegan <[email protected]> writes:
> 2. We could raise awareness of this issue by adding indent verification
> to CI testing. I hesitate to suggest that, though, for a couple of
> reasons:
> 2a. It seems fairly expensive, though I might be misjudging.
Compared to other things it's not that expensive. On my workstation, which is
slower on a per-core basis than CI, a whole tree pgindent --silent-diff takes
6.8s. For That's doing things serially, it shouldn't be that hard to parallelize
the per-file processing.
For comparison, the current compiler warnings task takes 6-15min, depending on
the state of the ccache "database". Even when ccache is primed, running
cpluspluscheck or headerscheck is ~30s each. Adding a few more seconds for an
indentation check wouldn't be a problem.
> 2b. It's often pretty handy to submit patches that aren't fully
> indent-clean; I have such a patch in flight right now at [1].
>
> 2b could be ameliorated by making the indent check be a separate
> test process that doesn't obscure the results of other testing.
The compiler warnings task already executes a number of tests even if prior
tests have failed (to be able to find compiler warnings in different compilers
at once). Adding pgindent cleanliness to that would be fairly simple.
I still think that one of the more important things we ought to do is to make
it trivial to check if code is correctly indented and reindent it for the
user. I've posted a preliminary patch to add a 'indent-tree' target a few
months back, at
https://postgr.es/m/20230527184201.2zdorrijg2inqt6v%40alap3.anarazel.de
I've updated that patch, now it has
- indent-tree, reindents the entire tree
- indent-head, which pgindent --commit=HEAD
- indent-check, fails if the tree isn't correctly indented
- indent-diff, like indent-check, but also shows the diff
If we tought pgindent to emit the list of files it processes to a dependency
file, we could make it cheap to call indent-check repeatedly, by teaching
meson/ninja to not reinvoke it if the input files haven't changed. Personally
that'd make it more bearable to script indentation checks to happen
frequently.
I'll look into writing a command to update typedefs.list with all the local
changes.
Greetings,
Andres Freund
Attachments:
[text/x-diff] v2-0001-Add-meson-targets-to-run-pgindent.patch (2.9K, ../../[email protected]/2-v2-0001-Add-meson-targets-to-run-pgindent.patch)
download | inline diff:
From 288c109286fda4dbd40fab345fbaf71d91d2db39 Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Sat, 27 May 2023 11:38:27 -0700
Subject: [PATCH v2] Add meson targets to run pgindent
Author:
Reviewed-By:
Discussion: https://postgr.es/m/
Backpatch:
---
meson.build | 36 ++++++++++++++++++++++++++++++++++++
src/tools/pgindent/pgindent | 12 ++++++++++--
2 files changed, 46 insertions(+), 2 deletions(-)
diff --git a/meson.build b/meson.build
index 862c955453f..7d4d02e7624 100644
--- a/meson.build
+++ b/meson.build
@@ -3013,6 +3013,42 @@ run_target('install-test-files',
+###############################################################
+# Indentation and similar targets
+###############################################################
+
+indent_base = [perl, files('src/tools/pgindent/pgindent'),
+ '--indent', pg_bsd_indent.full_path(),
+ '--sourcetree=@SOURCE_ROOT@']
+
+# reindent the entire tree
+run_target('indent-tree',
+ command: indent_base + ['.'],
+ depends: pg_bsd_indent
+)
+
+# reindent the last commit
+run_target('indent-head',
+ command: indent_base + ['--commit=HEAD'],
+ depends: pg_bsd_indent
+)
+
+# silently check if any file is not corectly indented (fails the build if
+# there are differences)
+run_target('indent-check',
+ command: indent_base + ['--silent-diff', '.'],
+ depends: pg_bsd_indent
+)
+
+# show a diff of all the unindented files (doesn't fail the build if there are
+# differences)
+run_target('indent-diff',
+ command: indent_base + ['--show-diff', '.'],
+ depends: pg_bsd_indent
+)
+
+
+
###############################################################
# Test prep
###############################################################
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index bce63d95daf..08b0c95814f 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -23,7 +23,7 @@ my $devnull = File::Spec->devnull;
my ($typedefs_file, $typedef_str, @excludes,
$indent, $build, $show_diff,
- $silent_diff, $help, @commits,);
+ $silent_diff, $sourcetree, $help, @commits,);
$help = 0;
@@ -35,7 +35,8 @@ my %options = (
"excludes=s" => \@excludes,
"indent=s" => \$indent,
"show-diff" => \$show_diff,
- "silent-diff" => \$silent_diff,);
+ "silent-diff" => \$silent_diff,
+ "sourcetree=s" => \$sourcetree);
GetOptions(%options) || usage("bad command line argument");
usage() if $help;
@@ -53,6 +54,13 @@ $typedefs_file ||= $ENV{PGTYPEDEFS};
# get indent location for environment or default
$indent ||= $ENV{PGINDENT} || $ENV{INDENT} || "pg_bsd_indent";
+# When invoked from the build directory, change into source tree, otherwise
+# the heuristics in locate_sourcedir() don't work.
+if (defined $sourcetree)
+{
+ chdir $sourcetree;
+}
+
my $sourcedir = locate_sourcedir();
# if it's the base of a postgres tree, we will exclude the files
--
2.38.0
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-19 00:56 Andres Freund <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andres Freund @ 2023-10-19 00:56 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Hi,
On 2023-10-18 12:15:51 -0700, Andres Freund wrote:
> I still think that one of the more important things we ought to do is to make
> it trivial to check if code is correctly indented and reindent it for the
> user. I've posted a preliminary patch to add a 'indent-tree' target a few
> months back, at
> https://postgr.es/m/20230527184201.2zdorrijg2inqt6v%40alap3.anarazel.de
>
> I've updated that patch, now it has
> - indent-tree, reindents the entire tree
> - indent-head, which pgindent --commit=HEAD
> - indent-check, fails if the tree isn't correctly indented
> - indent-diff, like indent-check, but also shows the diff
>
> If we tought pgindent to emit the list of files it processes to a dependency
> file, we could make it cheap to call indent-check repeatedly, by teaching
> meson/ninja to not reinvoke it if the input files haven't changed. Personally
> that'd make it more bearable to script indentation checks to happen
> frequently.
>
>
> I'll look into writing a command to update typedefs.list with all the local
> changes.
It turns out that updating the in-tree typedefs.list would be very noisy. On
my local linux system I get
1 file changed, 422 insertions(+), 1 deletion(-)
On a mac mini I get
1 file changed, 351 insertions(+), 1 deletion(-)
We could possibly address that by updating the in-tree typedefs.list a bit
more aggressively. Sure looks like the source systems are on the older side.
But in the attached patch I've implemented this slightly differently. If the
tooling to do so is available, the indent-* targets explained above,
use/depend on src/tools/pgindent/typedefs.list.merged (in the build dir),
which is the combination of a src/tools/pgindent/typedefs.list.local generated
for the local binaries/libraries and the source tree
src/tools/pgindent/typedefs.list.
src/tools/pgindent/typedefs.list.local is generated in fragments, with one
fragment for each build target. That way the whole file doesn't have to be
regenerated all the time, which can save a good bit of time (althoug obviously
less when hacking on the backend).
This makes it quite quick to locally indent, without needing to manually
fiddle around with manually modifying typedefs.list or using a separate
typedefs.list.
In a third commit I added a 'nitpick' configure time option, defaulting to
off, which runs an indentation check. The failure mode of that currently isn't
very helpful though, as it just uses --silent-diff.
All of this currently is meson only, largely because I don't feel like
spending the time messing with the configure build, particularly before there
is any agreement on this being the thing to do.
Greetings,
Andres Freund
Attachments:
[text/x-diff] v3-0001-Add-meson-targets-to-run-pgindent.patch (2.9K, ../../[email protected]/2-v3-0001-Add-meson-targets-to-run-pgindent.patch)
download | inline diff:
From f7520fd87764cddbde9d6c5d25fdfe5314ec5cbc Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Sat, 27 May 2023 11:38:27 -0700
Subject: [PATCH v3 1/3] Add meson targets to run pgindent
Author:
Reviewed-By:
Discussion: https://postgr.es/m/
Backpatch:
---
meson.build | 38 +++++++++++++++++++++++++++++++++++++
src/tools/pgindent/pgindent | 12 ++++++++++--
2 files changed, 48 insertions(+), 2 deletions(-)
diff --git a/meson.build b/meson.build
index 862c955453f..2cfeb60eb07 100644
--- a/meson.build
+++ b/meson.build
@@ -3013,6 +3013,44 @@ run_target('install-test-files',
+###############################################################
+# Indentation and similar targets
+###############################################################
+
+indent_base_cmd = [perl, files('src/tools/pgindent/pgindent'),
+ '--indent', pg_bsd_indent.full_path(),
+ '--sourcetree=@SOURCE_ROOT@']
+indent_depend = [pg_bsd_indent]
+
+# reindent the entire tree
+# Reindent the entire tree
+run_target('indent-tree',
+ command: indent_base_cmd + ['.'],
+ depends: indent_depend
+)
+
+# Reindent the last commit
+run_target('indent-head',
+ command: indent_base_cmd + ['--commit=HEAD'],
+ depends: indent_depend
+)
+
+# Silently check if any file is not corectly indented (fails the build if
+# there are differences)
+run_target('indent-check',
+ command: indent_base_cmd + ['--silent-diff', '.'],
+ depends: indent_depend
+)
+
+# Show a diff of all the unindented files (doesn't fail the build if there are
+# differences)
+run_target('indent-diff',
+ command: indent_base_cmd + ['--show-diff', '.'],
+ depends: indent_depend
+)
+
+
+
###############################################################
# Test prep
###############################################################
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index bce63d95daf..08b0c95814f 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -23,7 +23,7 @@ my $devnull = File::Spec->devnull;
my ($typedefs_file, $typedef_str, @excludes,
$indent, $build, $show_diff,
- $silent_diff, $help, @commits,);
+ $silent_diff, $sourcetree, $help, @commits,);
$help = 0;
@@ -35,7 +35,8 @@ my %options = (
"excludes=s" => \@excludes,
"indent=s" => \$indent,
"show-diff" => \$show_diff,
- "silent-diff" => \$silent_diff,);
+ "silent-diff" => \$silent_diff,
+ "sourcetree=s" => \$sourcetree);
GetOptions(%options) || usage("bad command line argument");
usage() if $help;
@@ -53,6 +54,13 @@ $typedefs_file ||= $ENV{PGTYPEDEFS};
# get indent location for environment or default
$indent ||= $ENV{PGINDENT} || $ENV{INDENT} || "pg_bsd_indent";
+# When invoked from the build directory, change into source tree, otherwise
+# the heuristics in locate_sourcedir() don't work.
+if (defined $sourcetree)
+{
+ chdir $sourcetree;
+}
+
my $sourcedir = locate_sourcedir();
# if it's the base of a postgres tree, we will exclude the files
--
2.38.0
[text/x-diff] v3-0002-heavily-wip-update-typedefs.patch (8.6K, ../../[email protected]/3-v3-0002-heavily-wip-update-typedefs.patch)
download | inline diff:
From b8d31613acfc9d554640b4e292f366d79084bb8e Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 18 Oct 2023 16:18:58 -0700
Subject: [PATCH v3 2/3] heavily wip: update typedefs
---
meson.build | 14 +++++-
src/backend/jit/llvm/meson.build | 2 +-
src/backend/snowball/meson.build | 2 +-
src/timezone/meson.build | 2 +-
src/tools/pgindent/merge_typedefs | 24 +++++++++
src/tools/pgindent/meson.build | 70 +++++++++++++++++++++++++++
src/tools/pgindent/typedefs_from_objs | 42 ++++++++++++++++
7 files changed, 152 insertions(+), 4 deletions(-)
create mode 100755 src/tools/pgindent/merge_typedefs
create mode 100644 src/tools/pgindent/meson.build
create mode 100755 src/tools/pgindent/typedefs_from_objs
diff --git a/meson.build b/meson.build
index 2cfeb60eb07..70f6b680c2c 100644
--- a/meson.build
+++ b/meson.build
@@ -2579,6 +2579,7 @@ add_project_link_arguments(ldflags, language: ['c', 'cpp'])
# list of targets for various alias targets
backend_targets = []
+data_targets = []
bin_targets = []
pl_targets = []
contrib_targets = []
@@ -2894,6 +2895,8 @@ subdir('src/interfaces/ecpg/test')
subdir('doc/src/sgml')
+subdir('src/tools/pgindent')
+
generated_sources_ac += {'': ['GNUmakefile']}
# After processing src/test, add test_install_libs to the testprep_targets
@@ -2982,6 +2985,7 @@ endif
all_built = [
backend_targets,
bin_targets,
+ data_targets,
libpq_st,
pl_targets,
contrib_targets,
@@ -3017,12 +3021,20 @@ run_target('install-test-files',
# Indentation and similar targets
###############################################################
+# If the dependencies for generating a local typedefs.list are fulfilled, we
+# use a combination of a locally built and the source tree's typededefs.list
+# file (see src/tools/pgindent/meson.build) for reindenting. That ensures
+# newly added typedefs are indented correctly.
indent_base_cmd = [perl, files('src/tools/pgindent/pgindent'),
'--indent', pg_bsd_indent.full_path(),
'--sourcetree=@SOURCE_ROOT@']
indent_depend = [pg_bsd_indent]
-# reindent the entire tree
+if typedefs_supported
+ indent_base_cmd += ['--typedefs', typedefs_merged.full_path()]
+ indent_depend += typedefs_merged
+endif
+
# Reindent the entire tree
run_target('indent-tree',
command: indent_base_cmd + ['.'],
diff --git a/src/backend/jit/llvm/meson.build b/src/backend/jit/llvm/meson.build
index 8ffaf414609..7e2f6d5bd5c 100644
--- a/src/backend/jit/llvm/meson.build
+++ b/src/backend/jit/llvm/meson.build
@@ -82,4 +82,4 @@ llvmjit_types = custom_target('llvmjit_types.bc',
install_dir: dir_lib_pkg,
depfile: '@[email protected]',
)
-backend_targets += llvmjit_types
+data_targets += llvmjit_types
diff --git a/src/backend/snowball/meson.build b/src/backend/snowball/meson.build
index 0f669c0bf3c..69e777399fd 100644
--- a/src/backend/snowball/meson.build
+++ b/src/backend/snowball/meson.build
@@ -94,4 +94,4 @@ install_subdir('stopwords',
)
backend_targets += dict_snowball
-backend_targets += snowball_create
+data_targets += snowball_create
diff --git a/src/timezone/meson.build b/src/timezone/meson.build
index 7b85a01c6bd..779a51d85a4 100644
--- a/src/timezone/meson.build
+++ b/src/timezone/meson.build
@@ -50,7 +50,7 @@ if get_option('system_tzdata') == ''
install_dir: dir_data,
)
- bin_targets += tzdata
+ data_targets += tzdata
endif
subdir('tznames')
diff --git a/src/tools/pgindent/merge_typedefs b/src/tools/pgindent/merge_typedefs
new file mode 100755
index 00000000000..c2e1f446c15
--- /dev/null
+++ b/src/tools/pgindent/merge_typedefs
@@ -0,0 +1,24 @@
+#!/usr/bin/env python3
+
+# Tool to merge multiple typedefs files into one
+
+import argparse
+import os
+import subprocess
+import sys
+
+parser = argparse.ArgumentParser(
+ description='merge multiple typedef files')
+
+parser.add_argument('--output', type=argparse.FileType('w'), required=True)
+parser.add_argument('input', type=argparse.FileType('r'), nargs='*')
+
+args = parser.parse_args()
+
+typedefs = set()
+
+for input in args.input:
+ for typedef in input.readlines():
+ typedefs.add(typedef.strip())
+
+print('\n'.join(sorted(typedefs)), file=args.output)
diff --git a/src/tools/pgindent/meson.build b/src/tools/pgindent/meson.build
new file mode 100644
index 00000000000..c0801a6c86c
--- /dev/null
+++ b/src/tools/pgindent/meson.build
@@ -0,0 +1,70 @@
+# Currently the code requires meson 0.63 or upwards. This could likely be
+# lifted (in fact, some older versions actually work, but only some), but for
+# now it's not crucial.
+typedefs_supported = meson.version().version_compare('>=0.63')
+if not typedefs_supported
+ subdir_done()
+endif
+
+typedefs_from_objs = files('typedefs_from_objs')
+typedefs_from_objs_cmd = [typedefs_from_objs, '--host', host_system, '--output', '@OUTPUT@', '@INPUT@']
+merge_typedefs = files('merge_typedefs')
+merge_typedefs_cmd = [merge_typedefs, '--output', '@OUTPUT@', '@INPUT@']
+
+# XXX: This list of targets should likely not be maintained here
+typedef_src_tgts = [
+ backend_targets,
+ bin_targets,
+ [libpq_st],
+ pl_targets,
+ contrib_targets,
+ ecpg_targets,
+]
+
+# We generate partial typedefs files for each binary/library. That makes using
+# this during incremental development much faster.
+#
+# The reason we process the object files instead of executables is that that
+# doesn't work on macos. There doesn't seem to be a reason to target
+# executables directly on other platforms.
+typedef_tgts = []
+foreach tgts : typedef_src_tgts
+ foreach tgt : tgts
+ # can't use tgt.name(), as we have have targets that differ just in suffix
+ name = fs.name(tgt.full_path())
+ tdname = 'typedefs.list.local-@0@'.format(name)
+ objs = tgt.extract_all_objects(recursive: true)
+ typedef_tgts += custom_target(tdname,
+ output: tdname,
+ command: typedefs_from_objs_cmd,
+ input: objs,
+ build_by_default: false,
+ )
+
+ endforeach
+endforeach
+
+# A typedefs.list file for the local build tree. This may not contain typedefs
+# referenced e.g. by platform specific code for other platforms.
+typedefs_local = custom_target('typedefs.list.local', output: 'typedefs.list.local',
+ command: merge_typedefs_cmd,
+ input: typedef_tgts,
+ build_by_default: false,
+)
+
+# The locally generated typedef.list, merge with the source tree version. This
+# is useful for reindenting code, particularly when the local tree has new
+# types added.
+typedefs_merged = custom_target('typedefs.list.merged', output: 'typedefs.list.merged',
+ command: merge_typedefs_cmd,
+ input: [typedefs_local, files('typedefs.list')],
+ build_by_default: false,
+)
+
+# FIXME: As-is this is rarely useful, as the local typedefs file will likely
+# contain too many changes.
+if cp.found()
+ run_target('update-typedefs',
+ command: [cp, typedefs_merged, '@SOURCE_ROOT@/src/tools/pgindent/typedefs.list'],
+ )
+endif
diff --git a/src/tools/pgindent/typedefs_from_objs b/src/tools/pgindent/typedefs_from_objs
new file mode 100755
index 00000000000..821c84cb3d8
--- /dev/null
+++ b/src/tools/pgindent/typedefs_from_objs
@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+
+# Tool to extract typedefs from object files.
+
+import argparse
+import re
+import shutil
+import subprocess
+import sys
+
+parser = argparse.ArgumentParser(
+ description='generate typedefs file for a set of object files')
+
+parser.add_argument('--output', type=argparse.FileType('w'), required=True)
+parser.add_argument('--host', type=str, required=True)
+parser.add_argument('input', nargs='*')
+
+args = parser.parse_args()
+
+if args.host == 'linux':
+ find_td_re = re.compile(r'^[^\n]+TAG_typedef\)\n[^\n]+DW_AT_name\s*:\s*\([^\)]+\): ([^\n]+)$', re.MULTILINE)
+ # FIXME: should probably be set by the caller? Except that the same binary
+ # name behaves very differently on different platforms :/
+ cmd = [shutil.which('objdump'), '-Wi']
+elif args.host == 'darwin':
+ find_td_re = re.compile(r'^[^\n]+TAG_typedef\n\s*DW_AT_type[^\n]+\n\s+DW_AT_name\s*\(\"([^\n]+)\"\)$', re.MULTILINE)
+ cmd = [shutil.which('dwarfdump')]
+else:
+ raise f'unsupported platform: {args.host}'
+
+lcmd = cmd + args.input
+sp = subprocess.run(lcmd, stdout=subprocess.PIPE, universal_newlines=True)
+if sp.returncode != 0:
+ print(f'{lcmd} failed with return code {sp.returncode}', file=sys.stderr)
+ sys.exit(sp.returncode)
+
+fa = find_td_re.findall(sp.stdout)
+
+for typedef in fa:
+ print(typedef, file=args.output)
+
+sys.exit(0)
--
2.38.0
[text/x-diff] v3-0003-WIP-meson-nitpick-mode-test-checking-indentation.patch (2.6K, ../../[email protected]/4-v3-0003-WIP-meson-nitpick-mode-test-checking-indentation.patch)
download | inline diff:
From 6688a89700af997b5adaa55f349c8db53a86290b Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 18 Oct 2023 17:51:21 -0700
Subject: [PATCH v3 3/3] WIP: meson: nitpick mode + test checking indentation
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch:
---
meson.build | 17 +++++++++++++++--
meson_options.txt | 3 +++
.cirrus.tasks.yml | 1 +
3 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/meson.build b/meson.build
index 70f6b680c2c..86a5d0995a1 100644
--- a/meson.build
+++ b/meson.build
@@ -44,6 +44,7 @@ cc = meson.get_compiler('c')
not_found_dep = dependency('', required: false)
thread_dep = dependency('threads')
auto_features = get_option('auto_features')
+nitpick_mode = get_option('nitpick')
@@ -3025,9 +3026,10 @@ run_target('install-test-files',
# use a combination of a locally built and the source tree's typededefs.list
# file (see src/tools/pgindent/meson.build) for reindenting. That ensures
# newly added typedefs are indented correctly.
-indent_base_cmd = [perl, files('src/tools/pgindent/pgindent'),
+indent_base_args = [files('src/tools/pgindent/pgindent'),
'--indent', pg_bsd_indent.full_path(),
- '--sourcetree=@SOURCE_ROOT@']
+ '--sourcetree=@0@'.format(meson.current_source_dir())]
+indent_base_cmd = [perl, indent_base_args]
indent_depend = [pg_bsd_indent]
if typedefs_supported
@@ -3364,6 +3366,17 @@ if meson.version().version_compare('>=0.57')
endif
+# If we were asked to be nitpicky, add a test that fails if sources aren't
+# properly indented
+if nitpick_mode
+ test('indent-check',
+ perl, args: [indent_base_args, '--silent-diff', '.'],
+ depends: indent_depend,
+ suite: 'source',
+ priority: 10)
+endif
+
+
###############################################################
# Pseudo targets
diff --git a/meson_options.txt b/meson_options.txt
index d2f95cfec36..90cb2177a44 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -52,6 +52,9 @@ option('atomics', type: 'boolean', value: true,
option('spinlocks', type: 'boolean', value: true,
description: 'Use spinlocks')
+option('nitpick', type: 'boolean', value: false,
+ description: 'annoy the hell out of you, committers ought to enable this')
+
# Compilation options
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index e137769850d..5dd29bed4cb 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -358,6 +358,7 @@ task:
meson setup \
--buildtype=debug \
-Dcassert=true \
+ -Dnitpick=true \
${LINUX_MESON_FEATURES} \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
build
--
2.38.0
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-19 01:29 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-10-19 01:29 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Andres Freund <[email protected]> writes:
> It turns out that updating the in-tree typedefs.list would be very noisy. On
> my local linux system I get
> 1 file changed, 422 insertions(+), 1 deletion(-)
> On a mac mini I get
> 1 file changed, 351 insertions(+), 1 deletion(-)
That seems like it needs a considerably closer look. What exactly
is getting added/deleted?
> We could possibly address that by updating the in-tree typedefs.list a bit
> more aggressively. Sure looks like the source systems are on the older side.
Really? Per [1] we've currently got contributions from calliphoridae
which is Debian sid, crake which is Fedora 38, indri/sifaka which are
macOS Sonoma. Were you really expecting something newer, and if so what?
> But in the attached patch I've implemented this slightly differently. If the
> tooling to do so is available, the indent-* targets explained above,
> use/depend on src/tools/pgindent/typedefs.list.merged (in the build dir),
> which is the combination of a src/tools/pgindent/typedefs.list.local generated
> for the local binaries/libraries and the source tree
> src/tools/pgindent/typedefs.list.
Hmm ... that allows indenting your C files, but how do you get from that
to a non-noisy patch to commit to typedefs.list?
regards, tom lane
[1] https://buildfarm.postgresql.org/cgi-bin/typedefs.pl?show_list
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-19 02:18 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Andres Freund @ 2023-10-19 02:18 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Hi,
On 2023-10-18 21:29:37 -0400, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
> > It turns out that updating the in-tree typedefs.list would be very noisy. On
> > my local linux system I get
> > 1 file changed, 422 insertions(+), 1 deletion(-)
> > On a mac mini I get
> > 1 file changed, 351 insertions(+), 1 deletion(-)
>
> That seems like it needs a considerably closer look. What exactly
> is getting added/deleted?
Types from bison, openssl, libxml, libxslt, icu and libc, at least. If I
enable LLVM, there are even more.
(I think I figured out what's happening further down)
> > We could possibly address that by updating the in-tree typedefs.list a bit
> > more aggressively. Sure looks like the source systems are on the older side.
>
> Really? Per [1] we've currently got contributions from calliphoridae
> which is Debian sid, crake which is Fedora 38, indri/sifaka which are
> macOS Sonoma. Were you really expecting something newer, and if so what?
It's quite odd, I see plenty more types than those. I can't really explain why
they're not being picked up on those animals.
E.g. for me bison generated files contain typedefs like
typedef int_least8_t yytype_int8;
typedef signed char yytype_int8;
typedef yytype_int8 yy_state_t;
yet they don't show up in the buildfarm typedefs output. It's not a thing of
the binary, I checked that the symbols are present.
I have a hard time parsing the buildfarm code for generating the typedefs
file, tbh.
<stare>
Ah, I see. If I interpret that correctly, the code filters out symbols it
doesn't find in in some .[chly] file in the *source* directory. This code is,
uh, barely readable and massively underdocumented.
I guess I need to reimplement that :/. Don't immediately see how this could
be implemented for in-tree autoconf builds...
> > But in the attached patch I've implemented this slightly differently. If the
> > tooling to do so is available, the indent-* targets explained above,
> > use/depend on src/tools/pgindent/typedefs.list.merged (in the build dir),
> > which is the combination of a src/tools/pgindent/typedefs.list.local generated
> > for the local binaries/libraries and the source tree
> > src/tools/pgindent/typedefs.list.
>
> Hmm ... that allows indenting your C files, but how do you get from that
> to a non-noisy patch to commit to typedefs.list?
It doesn't provide that on its own. Being able to painlessly indent the files
seems pretty worthwhile already. But clearly it'd much better if we can
automatically update typedefs.list.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-19 04:49 Andres Freund <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Andres Freund @ 2023-10-19 04:49 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Andrew Dunstan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Hi,
On 2023-10-18 19:18:13 -0700, Andres Freund wrote:
> On 2023-10-18 21:29:37 -0400, Tom Lane wrote:
> Ah, I see. If I interpret that correctly, the code filters out symbols it
> doesn't find in in some .[chly] file in the *source* directory. This code is,
> uh, barely readable and massively underdocumented.
>
> I guess I need to reimplement that :/. Don't immediately see how this could
> be implemented for in-tree autoconf builds...
>
> > > But in the attached patch I've implemented this slightly differently. If the
> > > tooling to do so is available, the indent-* targets explained above,
> > > use/depend on src/tools/pgindent/typedefs.list.merged (in the build dir),
> > > which is the combination of a src/tools/pgindent/typedefs.list.local generated
> > > for the local binaries/libraries and the source tree
> > > src/tools/pgindent/typedefs.list.
> >
> > Hmm ... that allows indenting your C files, but how do you get from that
> > to a non-noisy patch to commit to typedefs.list?
>
> It doesn't provide that on its own. Being able to painlessly indent the files
> seems pretty worthwhile already. But clearly it'd much better if we can
> automatically update typedefs.list.
With code for that added, things seem to work quite nicely. I added similar
logic to the buildfarm code that builds a list of all tokens in the source
code.
With that, the in-tree typedefs.list can be updated with new tokens found
locally *and* typdefs that aren't used anymore can be removed from the in-tree
typedefs.list (detected by no matching tokens found in the source code).
The only case this approach can't handle is newly referenced typedefs in code
that isn't built locally - which I think isn't particularly common and seems
somewhat fundamental. In those cases typedefs.list still can be updated
manually (and the sorting will still be "fixed" if necessary).
The code is still in a somewhat rough shape and I'll not finish polishing it
today. I've attached the code anyway, don't be too rough :).
The changes from running "ninja update-typedefs indent-tree" on debian and
macos are attached as 0004 - the set of changes looks quite correct to me.
The buildfarm code filtered out a few typedefs manually:
push(@badsyms, 'date', 'interval', 'timestamp', 'ANY');
but I don't really see why? Possibly that was needed with an older
pg_bsd_indent to prevent odd stuff?
Right now building a new unified typedefs.list and copying it to the source
tree are still separate targets, but that probably makes less sense now? Or
perhaps it should be copied to the source tree when reindenting?
I've only handled linux and macos in the typedefs gathering code. But the
remaining OSs should be "just a bit of work" [TM].
Greetings,
Andres Freund
Attachments:
[text/x-diff] v4-0001-Add-meson-targets-to-run-pgindent.patch (2.9K, ../../[email protected]/2-v4-0001-Add-meson-targets-to-run-pgindent.patch)
download | inline diff:
From f7520fd87764cddbde9d6c5d25fdfe5314ec5cbc Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Sat, 27 May 2023 11:38:27 -0700
Subject: [PATCH v4 1/4] Add meson targets to run pgindent
Author:
Reviewed-By:
Discussion: https://postgr.es/m/
Backpatch:
---
meson.build | 38 +++++++++++++++++++++++++++++++++++++
src/tools/pgindent/pgindent | 12 ++++++++++--
2 files changed, 48 insertions(+), 2 deletions(-)
diff --git a/meson.build b/meson.build
index 862c955453f..2cfeb60eb07 100644
--- a/meson.build
+++ b/meson.build
@@ -3013,6 +3013,44 @@ run_target('install-test-files',
+###############################################################
+# Indentation and similar targets
+###############################################################
+
+indent_base_cmd = [perl, files('src/tools/pgindent/pgindent'),
+ '--indent', pg_bsd_indent.full_path(),
+ '--sourcetree=@SOURCE_ROOT@']
+indent_depend = [pg_bsd_indent]
+
+# reindent the entire tree
+# Reindent the entire tree
+run_target('indent-tree',
+ command: indent_base_cmd + ['.'],
+ depends: indent_depend
+)
+
+# Reindent the last commit
+run_target('indent-head',
+ command: indent_base_cmd + ['--commit=HEAD'],
+ depends: indent_depend
+)
+
+# Silently check if any file is not corectly indented (fails the build if
+# there are differences)
+run_target('indent-check',
+ command: indent_base_cmd + ['--silent-diff', '.'],
+ depends: indent_depend
+)
+
+# Show a diff of all the unindented files (doesn't fail the build if there are
+# differences)
+run_target('indent-diff',
+ command: indent_base_cmd + ['--show-diff', '.'],
+ depends: indent_depend
+)
+
+
+
###############################################################
# Test prep
###############################################################
diff --git a/src/tools/pgindent/pgindent b/src/tools/pgindent/pgindent
index bce63d95daf..08b0c95814f 100755
--- a/src/tools/pgindent/pgindent
+++ b/src/tools/pgindent/pgindent
@@ -23,7 +23,7 @@ my $devnull = File::Spec->devnull;
my ($typedefs_file, $typedef_str, @excludes,
$indent, $build, $show_diff,
- $silent_diff, $help, @commits,);
+ $silent_diff, $sourcetree, $help, @commits,);
$help = 0;
@@ -35,7 +35,8 @@ my %options = (
"excludes=s" => \@excludes,
"indent=s" => \$indent,
"show-diff" => \$show_diff,
- "silent-diff" => \$silent_diff,);
+ "silent-diff" => \$silent_diff,
+ "sourcetree=s" => \$sourcetree);
GetOptions(%options) || usage("bad command line argument");
usage() if $help;
@@ -53,6 +54,13 @@ $typedefs_file ||= $ENV{PGTYPEDEFS};
# get indent location for environment or default
$indent ||= $ENV{PGINDENT} || $ENV{INDENT} || "pg_bsd_indent";
+# When invoked from the build directory, change into source tree, otherwise
+# the heuristics in locate_sourcedir() don't work.
+if (defined $sourcetree)
+{
+ chdir $sourcetree;
+}
+
my $sourcedir = locate_sourcedir();
# if it's the base of a postgres tree, we will exclude the files
--
2.38.0
[text/x-diff] v4-0002-heavily-wip-update-typedefs.patch (10.9K, ../../[email protected]/3-v4-0002-heavily-wip-update-typedefs.patch)
download | inline diff:
From dfe5ad9229e812d0f805d6a39cb296b9284e30f2 Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 18 Oct 2023 16:18:58 -0700
Subject: [PATCH v4 2/4] heavily wip: update typedefs
---
meson.build | 14 ++++-
src/backend/jit/llvm/meson.build | 2 +-
src/backend/snowball/meson.build | 2 +-
src/timezone/meson.build | 2 +-
src/tools/pgindent/merge_typedefs | 86 +++++++++++++++++++++++++++
src/tools/pgindent/meson.build | 69 +++++++++++++++++++++
src/tools/pgindent/typedefs_from_objs | 42 +++++++++++++
7 files changed, 213 insertions(+), 4 deletions(-)
create mode 100755 src/tools/pgindent/merge_typedefs
create mode 100644 src/tools/pgindent/meson.build
create mode 100755 src/tools/pgindent/typedefs_from_objs
diff --git a/meson.build b/meson.build
index 2cfeb60eb07..492ce9a48b1 100644
--- a/meson.build
+++ b/meson.build
@@ -2579,6 +2579,7 @@ add_project_link_arguments(ldflags, language: ['c', 'cpp'])
# list of targets for various alias targets
backend_targets = []
+data_targets = []
bin_targets = []
pl_targets = []
contrib_targets = []
@@ -2894,6 +2895,8 @@ subdir('src/interfaces/ecpg/test')
subdir('doc/src/sgml')
+subdir('src/tools/pgindent')
+
generated_sources_ac += {'': ['GNUmakefile']}
# After processing src/test, add test_install_libs to the testprep_targets
@@ -2982,6 +2985,7 @@ endif
all_built = [
backend_targets,
bin_targets,
+ data_targets,
libpq_st,
pl_targets,
contrib_targets,
@@ -3017,12 +3021,20 @@ run_target('install-test-files',
# Indentation and similar targets
###############################################################
+# If the dependencies for generating a local typedefs.list are fulfilled, we
+# use a combination of a locally built and the source tree's typededefs.list
+# file (see src/tools/pgindent/meson.build) for reindenting. That ensures
+# newly added typedefs are indented correctly.
indent_base_cmd = [perl, files('src/tools/pgindent/pgindent'),
'--indent', pg_bsd_indent.full_path(),
'--sourcetree=@SOURCE_ROOT@']
indent_depend = [pg_bsd_indent]
-# reindent the entire tree
+if typedefs_supported
+ indent_base_cmd += ['--typedefs', typedefs_merged.full_path()]
+ indent_depend += typedefs
+endif
+
# Reindent the entire tree
run_target('indent-tree',
command: indent_base_cmd + ['.'],
diff --git a/src/backend/jit/llvm/meson.build b/src/backend/jit/llvm/meson.build
index 8ffaf414609..7e2f6d5bd5c 100644
--- a/src/backend/jit/llvm/meson.build
+++ b/src/backend/jit/llvm/meson.build
@@ -82,4 +82,4 @@ llvmjit_types = custom_target('llvmjit_types.bc',
install_dir: dir_lib_pkg,
depfile: '@[email protected]',
)
-backend_targets += llvmjit_types
+data_targets += llvmjit_types
diff --git a/src/backend/snowball/meson.build b/src/backend/snowball/meson.build
index 0f669c0bf3c..69e777399fd 100644
--- a/src/backend/snowball/meson.build
+++ b/src/backend/snowball/meson.build
@@ -94,4 +94,4 @@ install_subdir('stopwords',
)
backend_targets += dict_snowball
-backend_targets += snowball_create
+data_targets += snowball_create
diff --git a/src/timezone/meson.build b/src/timezone/meson.build
index 7b85a01c6bd..779a51d85a4 100644
--- a/src/timezone/meson.build
+++ b/src/timezone/meson.build
@@ -50,7 +50,7 @@ if get_option('system_tzdata') == ''
install_dir: dir_data,
)
- bin_targets += tzdata
+ data_targets += tzdata
endif
subdir('tznames')
diff --git a/src/tools/pgindent/merge_typedefs b/src/tools/pgindent/merge_typedefs
new file mode 100755
index 00000000000..9826fd1a675
--- /dev/null
+++ b/src/tools/pgindent/merge_typedefs
@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+
+# Tool to build a new typedefs.list file from
+# a) the current typedefs.list file in the source tree
+# b) newly generated typedefs.list fragments for build products
+# c) the source code
+#
+# To avoid littering the typedefs file with typedefs that are not in our source
+# code, the list of typedefs found is filtered with tokens found in source
+# files.
+#
+# If the local build references a typedef that's not present in the source
+# typedefs.list, we can discover such typedefs.
+#
+# If the source typedefs.list references a typedef that is not used anymore, it
+# will not appear in the source tree anymore and thus can be filtered out.
+#
+# However, typedefs in newly written code for a different platform (or a
+# disabled build option), will not be discovered.
+
+import argparse
+import os
+import subprocess
+import sys
+import re
+
+parser = argparse.ArgumentParser(
+ description='build new typedefs files from multiple inputs')
+
+parser.add_argument('--output-local', type=argparse.FileType('w'), required=True)
+parser.add_argument('--output-merged', type=argparse.FileType('w'), required=True)
+parser.add_argument('--current-typedefs', type=argparse.FileType('r'), required=True)
+parser.add_argument('--filter-source', type=str, required=False)
+parser.add_argument('input', type=argparse.FileType('r'), nargs='*')
+
+
+def merge_typedefs(files):
+ typedefs = set()
+ for input in files:
+ for typedef in input.readlines():
+ typedefs.add(typedef.strip())
+ return typedefs
+
+def source_list(srcdir):
+ filelist = []
+ os.chdir(srcdir)
+ accepted_files = re.compile(r'\.(c|h|l|y|.cpp)$')
+ for root, dirs, files in os.walk('.', topdown=True):
+ # don't go into hidden dirs or docs
+ for d in dirs:
+ if d.startswith('.') or d == 'doc':
+ dirs.remove(d)
+ for f in files:
+ if accepted_files.search(f):
+ filelist.append((os.path.join(root, f)))
+ return filelist
+
+def tokenize_sources(files):
+ comment_sub_re = re.compile(r'/\*.*?\*/', flags=re.MULTILINE|re.DOTALL)
+ token_find_re = re.compile(r'\b\w+\b')
+
+ tokens = set()
+ for file in files:
+ with open(file, 'r') as fh:
+ content = fh.read()
+ content = comment_sub_re.sub('', content)
+ tokens.update(token_find_re.findall(content))
+ return tokens
+
+def main(args):
+ local_typedefs = merge_typedefs(args.input)
+ current_typedefs = merge_typedefs([args.current_typedefs])
+
+ if args.filter_source:
+ filelist = source_list(args.filter_source)
+ tokens = tokenize_sources(filelist)
+ local_typedefs.intersection_update(tokens)
+ current_typedefs.intersection_update(tokens)
+
+ current_typedefs.update(local_typedefs)
+ print('\n'.join(sorted(local_typedefs)), file=args.output_local)
+ print('\n'.join(sorted(current_typedefs)), file=args.output_merged)
+
+if __name__ == '__main__':
+ main(parser.parse_args())
+ sys.exit(0)
diff --git a/src/tools/pgindent/meson.build b/src/tools/pgindent/meson.build
new file mode 100644
index 00000000000..de02c676a46
--- /dev/null
+++ b/src/tools/pgindent/meson.build
@@ -0,0 +1,69 @@
+# Currently the code requires meson 0.63 or upwards. This could likely be
+# lifted (in fact, some older versions actually work, but only some), but for
+# now it's not crucial.
+typedefs_supported = meson.version().version_compare('>=0.63')
+if not typedefs_supported
+ subdir_done()
+endif
+
+typedefs_from_objs = files('typedefs_from_objs')
+typedefs_from_objs_cmd = [typedefs_from_objs, '--host', host_system, '--output', '@OUTPUT@', '@INPUT@']
+merge_typedefs = files('merge_typedefs')
+
+# XXX: This list of targets should likely not be maintained here
+typedef_src_tgts = [
+ backend_targets,
+ bin_targets,
+ [libpq_st],
+ pl_targets,
+ contrib_targets,
+ ecpg_targets,
+]
+
+# We generate partial typedefs files for each binary/library. That makes using
+# this during incremental development much faster.
+#
+# The reason we process the object files instead of executables is that that
+# doesn't work on macos. There doesn't seem to be a reason to target
+# executables directly on other platforms.
+typedef_tgts = []
+foreach tgts : typedef_src_tgts
+ foreach tgt : tgts
+ # can't use tgt.name(), as we have have targets that differ just in suffix
+ name = fs.name(tgt.full_path())
+ tdname = 'typedefs.list.local-@0@'.format(name)
+ objs = tgt.extract_all_objects(recursive: true)
+ typedef_tgts += custom_target(tdname,
+ output: tdname,
+ command: typedefs_from_objs_cmd,
+ input: objs,
+ build_by_default: false,
+ )
+
+ endforeach
+endforeach
+
+
+# Build new typedefs.list files. typedefs.list.local will contain only
+# typedefs in the local build, whereas typedefs.list.merged is the combination
+# of typedefs.list.local and the source tree's typedefs.list. However,
+# typedefs that are not used in the code anymore are removed even from
+# typedefs.list.merged.
+typedefs = custom_target('typedefs.list', output: ['typedefs.list.local', 'typedefs.list.merged'],
+ command: [merge_typedefs,
+ '--output-local', '@OUTPUT0@',
+ '--output-merged', '@OUTPUT1@',
+ '--current-typedefs', files('typedefs.list'),
+ '--filter-source', '@SOURCE_ROOT@',
+ '@INPUT@'],
+ input: typedef_tgts,
+ build_by_default: false,
+)
+typedefs_local = typedefs[0]
+typedefs_merged = typedefs[1]
+
+if cp.found()
+ run_target('update-typedefs',
+ command: [cp, typedefs_merged, '@SOURCE_ROOT@/src/tools/pgindent/typedefs.list'],
+ )
+endif
diff --git a/src/tools/pgindent/typedefs_from_objs b/src/tools/pgindent/typedefs_from_objs
new file mode 100755
index 00000000000..821c84cb3d8
--- /dev/null
+++ b/src/tools/pgindent/typedefs_from_objs
@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+
+# Tool to extract typedefs from object files.
+
+import argparse
+import re
+import shutil
+import subprocess
+import sys
+
+parser = argparse.ArgumentParser(
+ description='generate typedefs file for a set of object files')
+
+parser.add_argument('--output', type=argparse.FileType('w'), required=True)
+parser.add_argument('--host', type=str, required=True)
+parser.add_argument('input', nargs='*')
+
+args = parser.parse_args()
+
+if args.host == 'linux':
+ find_td_re = re.compile(r'^[^\n]+TAG_typedef\)\n[^\n]+DW_AT_name\s*:\s*\([^\)]+\): ([^\n]+)$', re.MULTILINE)
+ # FIXME: should probably be set by the caller? Except that the same binary
+ # name behaves very differently on different platforms :/
+ cmd = [shutil.which('objdump'), '-Wi']
+elif args.host == 'darwin':
+ find_td_re = re.compile(r'^[^\n]+TAG_typedef\n\s*DW_AT_type[^\n]+\n\s+DW_AT_name\s*\(\"([^\n]+)\"\)$', re.MULTILINE)
+ cmd = [shutil.which('dwarfdump')]
+else:
+ raise f'unsupported platform: {args.host}'
+
+lcmd = cmd + args.input
+sp = subprocess.run(lcmd, stdout=subprocess.PIPE, universal_newlines=True)
+if sp.returncode != 0:
+ print(f'{lcmd} failed with return code {sp.returncode}', file=sys.stderr)
+ sys.exit(sp.returncode)
+
+fa = find_td_re.findall(sp.stdout)
+
+for typedef in fa:
+ print(typedef, file=args.output)
+
+sys.exit(0)
--
2.38.0
[text/x-diff] v4-0003-WIP-meson-nitpick-mode-test-checking-indentation.patch (2.9K, ../../[email protected]/4-v4-0003-WIP-meson-nitpick-mode-test-checking-indentation.patch)
download | inline diff:
From 017183e75c7a633dedabdbf57aa1ac80ee955f93 Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 18 Oct 2023 21:40:36 -0700
Subject: [PATCH v4 3/4] WIP: meson: nitpick mode + test checking indentation
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch:
---
meson.build | 20 +++++++++++++++++---
meson_options.txt | 3 +++
.cirrus.tasks.yml | 1 +
3 files changed, 21 insertions(+), 3 deletions(-)
diff --git a/meson.build b/meson.build
index 492ce9a48b1..854b7ecf56c 100644
--- a/meson.build
+++ b/meson.build
@@ -44,6 +44,7 @@ cc = meson.get_compiler('c')
not_found_dep = dependency('', required: false)
thread_dep = dependency('threads')
auto_features = get_option('auto_features')
+nitpick_mode = get_option('nitpick')
@@ -3025,16 +3026,18 @@ run_target('install-test-files',
# use a combination of a locally built and the source tree's typededefs.list
# file (see src/tools/pgindent/meson.build) for reindenting. That ensures
# newly added typedefs are indented correctly.
-indent_base_cmd = [perl, files('src/tools/pgindent/pgindent'),
+indent_base_args = [files('src/tools/pgindent/pgindent'),
'--indent', pg_bsd_indent.full_path(),
- '--sourcetree=@SOURCE_ROOT@']
+ '--sourcetree=@0@'.format(meson.current_source_dir())]
indent_depend = [pg_bsd_indent]
if typedefs_supported
- indent_base_cmd += ['--typedefs', typedefs_merged.full_path()]
+ indent_base_args += ['--typedefs', typedefs_merged.full_path()]
indent_depend += typedefs
endif
+indent_base_cmd = [perl, indent_base_args]
+
# Reindent the entire tree
run_target('indent-tree',
command: indent_base_cmd + ['.'],
@@ -3364,6 +3367,17 @@ if meson.version().version_compare('>=0.57')
endif
+# If we were asked to be nitpicky, add a test that fails if sources aren't
+# properly indented
+if nitpick_mode
+ test('indent-check',
+ perl, args: [indent_base_args, '--silent-diff', '.'],
+ depends: indent_depend,
+ suite: 'source',
+ priority: 10)
+endif
+
+
###############################################################
# Pseudo targets
diff --git a/meson_options.txt b/meson_options.txt
index d2f95cfec36..90cb2177a44 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -52,6 +52,9 @@ option('atomics', type: 'boolean', value: true,
option('spinlocks', type: 'boolean', value: true,
description: 'Use spinlocks')
+option('nitpick', type: 'boolean', value: false,
+ description: 'annoy the hell out of you, committers ought to enable this')
+
# Compilation options
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index e137769850d..5dd29bed4cb 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -358,6 +358,7 @@ task:
meson setup \
--buildtype=debug \
-Dcassert=true \
+ -Dnitpick=true \
${LINUX_MESON_FEATURES} \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
build
--
2.38.0
[text/x-diff] v4-0004-Update-typedefs-reindent-using-the-new-build-targ.patch (3.7K, ../../[email protected]/5-v4-0004-Update-typedefs-reindent-using-the-new-build-targ.patch)
download | inline diff:
From 7e06ca28697b365561371b2843d88c4caaccd347 Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Wed, 18 Oct 2023 21:31:22 -0700
Subject: [PATCH v4 4/4] Update typedefs, reindent using the new build targets
From debian sid and macos ventura.
---
contrib/sepgsql/hooks.c | 2 +-
contrib/sepgsql/label.c | 2 +-
contrib/sepgsql/uavc.c | 2 +-
src/tools/pgindent/typedefs.list | 14 +++++++++++---
4 files changed, 14 insertions(+), 6 deletions(-)
diff --git a/contrib/sepgsql/hooks.c b/contrib/sepgsql/hooks.c
index fa734763602..443e30be994 100644
--- a/contrib/sepgsql/hooks.c
+++ b/contrib/sepgsql/hooks.c
@@ -50,7 +50,7 @@ typedef struct
* command. Elsewhere (including the case of default) NULL.
*/
const char *createdb_dtemplate;
-} sepgsql_context_info_t;
+} sepgsql_context_info_t;
static sepgsql_context_info_t sepgsql_context_info;
diff --git a/contrib/sepgsql/label.c b/contrib/sepgsql/label.c
index 38ff4068ff2..e13640dc4d9 100644
--- a/contrib/sepgsql/label.c
+++ b/contrib/sepgsql/label.c
@@ -67,7 +67,7 @@ typedef struct
{
SubTransactionId subid;
char *label;
-} pending_label;
+} pending_label;
/*
* sepgsql_get_client_label
diff --git a/contrib/sepgsql/uavc.c b/contrib/sepgsql/uavc.c
index 6e3a892da24..6f63a19aa96 100644
--- a/contrib/sepgsql/uavc.c
+++ b/contrib/sepgsql/uavc.c
@@ -44,7 +44,7 @@ typedef struct
/* true, if tcontext is valid */
char *ncontext; /* temporary scontext on execution of trusted
* procedure, or NULL elsewhere */
-} avc_cache;
+} avc_cache;
/*
* Declaration of static variables
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index e69bb671bf8..d6178386252 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2,6 +2,7 @@ ACCESS_ALLOWED_ACE
ACL
ACL_SIZE_INFORMATION
AFFIX
+ANY
ASN1_INTEGER
ASN1_OBJECT
ASN1_OCTET_STRING
@@ -1275,9 +1276,9 @@ JsonManifestWALRangeField
JsonObjectAgg
JsonObjectConstructor
JsonOutput
-JsonParseExpr
JsonParseContext
JsonParseErrorType
+JsonParseExpr
JsonPath
JsonPathBool
JsonPathExecContext
@@ -1913,7 +1914,6 @@ ParallelHashJoinBatch
ParallelHashJoinBatchAccessor
ParallelHashJoinState
ParallelIndexScanDesc
-ParallelReadyList
ParallelSlot
ParallelSlotArray
ParallelSlotResultHandler
@@ -3135,6 +3135,7 @@ _ino_t
_locale_t
_resultmap
_stringlist
+access_vector_t
acquireLocksOnSubLinks_context
add_nulling_relids_context
adjust_appendrel_attrs_context
@@ -3167,6 +3168,7 @@ assign_collations_context
auth_password_hook_typ
autovac_table
av_relation
+avc_cache
avl_dbase
avl_node
avl_tree
@@ -3247,6 +3249,7 @@ crosstab_HashEnt
crosstab_cat_desc
datapagemap_iterator_t
datapagemap_t
+date
dateKEY
datetkn
dce_uuid_t
@@ -3402,6 +3405,7 @@ indexed_tlist
inet
inetKEY
inet_struct
+initRowMethod
init_function
inline_cte_walker_context
inline_error_callback_arg
@@ -3418,6 +3422,7 @@ int64
int64KEY
int8
internalPQconninfoOption
+interval
intptr_t
intset_internal_node
intset_leaf_node
@@ -3519,6 +3524,7 @@ parse_error_callback_arg
parser_context
partition_method_t
pendingPosition
+pending_label
pgParameterStatus
pg_atomic_flag
pg_atomic_uint32
@@ -3710,7 +3716,9 @@ saophash_hash
save_buffer
scram_state
scram_state_enum
+security_class_t
sem_t
+sepgsql_context_info_t
sequence_magic
set_join_pathlist_hook_type
set_rel_pathlist_hook_type
@@ -3792,6 +3800,7 @@ time_t
timeout_handler_proc
timeout_params
timerCA
+timestamp
tlist_vinfo
toast_compress_header
tokenize_error_callback_arg
@@ -3869,7 +3878,6 @@ wchar2mb_with_len_converter
wchar_t
win32_deadchild_waitinfo
wint_t
-worker_spi_state
worker_state
worktable
wrap
--
2.38.0
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-24 00:50 Jeff Davis <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Jeff Davis @ 2023-10-24 00:50 UTC (permalink / raw)
To: David Rowley <[email protected]>; Jelte Fennema <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; Robert Haas <[email protected]>; Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Wed, 2023-10-18 at 22:34 +1300, David Rowley wrote:
> It would be good to learn how many of the committers out of the ones
> you listed that --enable-indent-checks would have saved from breaking
> koel.
I'd find that a useful option.
Regards,
Jeff Davis
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-24 01:16 Amit Langote <[email protected]>
parent: Jeff Davis <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Amit Langote @ 2023-10-24 01:16 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Bruce Momjian <[email protected]>; David Rowley <[email protected]>; Jelte Fennema <[email protected]>; Jesse Zhang <[email protected]>; Justin Pryzby <[email protected]>; Magnus Hagander <[email protected]>; Michael Paquier <[email protected]>; Noah Misch <[email protected]>; Peter Eisentraut <[email protected]>; Peter Geoghegan <[email protected]>; Robert Haas <[email protected]>; Stephen Frost <[email protected]>; Tom Lane <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected] <[email protected]>; [email protected]
On Tue, Oct 24, 2023 at 9:51 Jeff Davis <[email protected]> wrote:
> On Wed, 2023-10-18 at 22:34 +1300, David Rowley wrote:
> > It would be good to learn how many of the committers out of the ones
> > you listed that --enable-indent-checks would have saved from breaking
> > koel.
>
> I'd find that a useful option.
+1. While I’ve made it part of routine to keep my local work pgindented
since breaking Joel once, an option like this would still be useful.
>
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-24 04:31 Michael Paquier <[email protected]>
parent: Amit Langote <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Michael Paquier @ 2023-10-24 04:31 UTC (permalink / raw)
To: Amit Langote <[email protected]>; +Cc: Jeff Davis <[email protected]>; Alvaro Herrera <[email protected]>; Andres Freund <[email protected]>; Andrew Dunstan <[email protected]>; Bruce Momjian <[email protected]>; David Rowley <[email protected]>; Jelte Fennema <[email protected]>; Jesse Zhang <[email protected]>; Justin Pryzby <[email protected]>; Magnus Hagander <[email protected]>; Noah Misch <[email protected]>; Peter Eisentraut <[email protected]>; Peter Geoghegan <[email protected]>; Robert Haas <[email protected]>; Stephen Frost <[email protected]>; Tom Lane <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected] <[email protected]>; [email protected]
On Tue, Oct 24, 2023 at 10:16:55AM +0900, Amit Langote wrote:
> +1. While I’ve made it part of routine to keep my local work pgindented
> since breaking Joel once, an option like this would still be useful.
I'd be OK with an option like that. It is one of these things to type
once in a script, then forget about it.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-24 13:53 Andrew Dunstan <[email protected]>
parent: Jelte Fennema <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-10-24 13:53 UTC (permalink / raw)
To: Jelte Fennema <[email protected]>; David Rowley <[email protected]>; Peter Eisentraut <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; +Cc: Robert Haas <[email protected]>; Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On 2023-10-18 We 05:07, Jelte Fennema wrote:
> I think --enable-indent-checks sounds like a good improvement to the
> status quo. But I'm not confident that it will help remove the cases
> where only a comment needs to be re-indented. Do commiters really
> always run check-world again when only changing a typo in a comment? I
> know I probably wouldn't (or at least not always).
Yeah. In fact I'm betting that a lot of the offending commits we've seen
come into this category. You build, you check, then you do some final
polish. That's where a pre-commit hook can save you.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-25 09:42 Amit Kapila <[email protected]>
parent: Jeff Davis <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Amit Kapila @ 2023-10-25 09:42 UTC (permalink / raw)
To: Jeff Davis <[email protected]>; +Cc: David Rowley <[email protected]>; Jelte Fennema <[email protected]>; Peter Eisentraut <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Andres Freund <[email protected]>; Michael Paquier <[email protected]>; Robert Haas <[email protected]>; Peter Geoghegan <[email protected]>; Tom Lane <[email protected]>; Andrew Dunstan <[email protected]>; [email protected] <[email protected]>; Justin Pryzby <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers
On Tue, Oct 24, 2023 at 6:21 AM Jeff Davis <[email protected]> wrote:
>
> On Wed, 2023-10-18 at 22:34 +1300, David Rowley wrote:
> > It would be good to learn how many of the committers out of the ones
> > you listed that --enable-indent-checks would have saved from breaking
> > koel.
>
> I'd find that a useful option.
>
+1.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-28 15:47 Andrew Dunstan <[email protected]>
parent: Andrew Dunstan <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-10-28 15:47 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Peter Geoghegan <[email protected]>; +Cc: Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-08-12 Sa 11:57, Andrew Dunstan wrote:
>
>
> On 2023-08-11 Fr 19:17, Tom Lane wrote:
>> Peter Geoghegan<[email protected]> writes:
>>> I'm starting to have doubts about this policy. There have now been
>>> quite a few follow-up "fixes" to indentation issues that koel
>>> complained about. None of these fixups have been included in
>>> .git-blame-ignore-revs. If things continue like this then "git blame"
>>> is bound to become much less usable over time.
>> FWIW, I'm much more optimistic than that. I think what we're seeing
>> is just the predictable result of not all committers having yet
>> incorporated "pgindent it before committing" into their workflow.
>> The need for followup fixes should diminish as people start doing
>> that. If you want to hurry things along, peer pressure on committers
>> who clearly aren't bothering is the solution.
>
>
> Yeah, part of the point of creating koel was to give committers a bit
> of a nudge in that direction.
>
> With a git pre-commit hook it's pretty painless.
>
>
Based on recent experience, where a lot koel's recent complaints seem to
be about comments, I'd like to suggest a modest adjustment.
First, we should provide a mode of pgindent that doesn't reflow
comments. pg_bsd_indent has a flag for this (-nfcb), so this should be
relatively simple. Second, koel could use that mode, so that it
wouldn't complain about comments it thinks need to be reflowed. Of
course, we'd fix these up with our regular pgindent runs.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-28 16:09 Tom Lane <[email protected]>
parent: Andrew Dunstan <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Tom Lane @ 2023-10-28 16:09 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
Andrew Dunstan <[email protected]> writes:
> Based on recent experience, where a lot koel's recent complaints seem to
> be about comments, I'd like to suggest a modest adjustment.
> First, we should provide a mode of pgindent that doesn't reflow
> comments. pg_bsd_indent has a flag for this (-nfcb), so this should be
> relatively simple. Second, koel could use that mode, so that it
> wouldn't complain about comments it thinks need to be reflowed. Of
> course, we'd fix these up with our regular pgindent runs.
Seems like a bit of a kluge. Maybe it's the right thing to do, but
I don't think we have enough data points yet to be confident that
it'd meaningfully reduce the number of breakages.
On a more abstract level: the point of trying to maintain indent
cleanliness is so that if you modify a file and then want to run
pgindent on your own changes, you don't get incidental changes
elsewhere in the file. This solution would break that, so I'm
not sure it isn't throwing the baby out with the bathwater.
regards, tom lane
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: run pgindent on a regular basis / scripted manner
@ 2023-10-29 14:22 Andrew Dunstan <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Andrew Dunstan @ 2023-10-29 14:22 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Peter Geoghegan <[email protected]>; Jelte Fennema <[email protected]>; Michael Paquier <[email protected]>; [email protected] <[email protected]>; Robert Haas <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Noah Misch <[email protected]>; Bruce Momjian <[email protected]>; Magnus Hagander <[email protected]>; Alvaro Herrera <[email protected]>; Stephen Frost <[email protected]>; Jesse Zhang <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>
On 2023-10-28 Sa 12:09, Tom Lane wrote:
> Andrew Dunstan <[email protected]> writes:
>> Based on recent experience, where a lot koel's recent complaints seem to
>> be about comments, I'd like to suggest a modest adjustment.
>> First, we should provide a mode of pgindent that doesn't reflow
>> comments. pg_bsd_indent has a flag for this (-nfcb), so this should be
>> relatively simple. Second, koel could use that mode, so that it
>> wouldn't complain about comments it thinks need to be reflowed. Of
>> course, we'd fix these up with our regular pgindent runs.
> Seems like a bit of a kluge. Maybe it's the right thing to do, but
> I don't think we have enough data points yet to be confident that
> it'd meaningfully reduce the number of breakages.
>
> On a more abstract level: the point of trying to maintain indent
> cleanliness is so that if you modify a file and then want to run
> pgindent on your own changes, you don't get incidental changes
> elsewhere in the file. This solution would break that, so I'm
> not sure it isn't throwing the baby out with the bathwater.
Yeah, could be.
cheers
andrew.
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 142+ messages in thread
end of thread, other threads:[~2023-10-29 14:22 UTC | newest]
Thread overview: 142+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-03-12 03:07 [PATCH 1/7] All grouping sets do their own sorting Pengzhou Tang <[email protected]>
2023-02-04 11:34 Re: run pgindent on a regular basis / scripted manner Andres Freund <[email protected]>
2023-02-04 14:20 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-05 14:29 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-04-28 09:25 ` Re: run pgindent on a regular basis / scripted manner Alvaro Herrera <[email protected]>
2023-02-04 16:07 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-02-04 17:11 ` Re: run pgindent on a regular basis / scripted manner Justin Pryzby <[email protected]>
2023-02-04 17:37 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-02-06 14:40 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-02-06 15:16 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-02-06 15:35 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-02-06 15:21 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-06 15:36 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-02-06 17:03 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-06 17:53 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-07 17:21 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-02-08 12:41 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-08 13:27 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-08 17:06 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-02-08 22:09 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-09 02:29 ` RE: run pgindent on a regular basis / scripted manner Shinoda, Noriyoshi (PN Japan FSIP) <[email protected]>
2023-02-09 18:34 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-10 02:37 ` RE: run pgindent on a regular basis / scripted manner [email protected] <[email protected]>
2023-02-10 09:25 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-02-10 15:21 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-12 14:16 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-13 14:02 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-02-13 16:46 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-13 18:29 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-02-12 16:24 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-02-12 20:41 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-12 21:13 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-02-13 13:27 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-12 20:59 ` Re: run pgindent on a regular basis / scripted manner Justin Pryzby <[email protected]>
2023-02-13 12:57 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-15 18:45 ` Re: run pgindent on a regular basis / scripted manner Justin Pryzby <[email protected]>
2023-02-15 21:00 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-02-16 08:26 ` RE: run pgindent on a regular basis / scripted manner [email protected] <[email protected]>
2023-02-16 16:44 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-04-21 07:58 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-04-22 08:50 ` Re: run pgindent on a regular basis / scripted manner Michael Paquier <[email protected]>
2023-04-22 11:42 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-04-22 12:10 ` Re: run pgindent on a regular basis / scripted manner Michael Paquier <[email protected]>
2023-04-22 12:47 ` Re: run pgindent on a regular basis / scripted manner Magnus Hagander <[email protected]>
2023-04-22 14:12 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-04-23 21:52 ` Re: run pgindent on a regular basis / scripted manner Magnus Hagander <[email protected]>
2023-04-22 14:24 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-06-15 15:26 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-06-15 16:12 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-06-17 14:08 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-06-19 21:07 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-06-20 12:04 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-06-20 13:08 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-06-20 13:21 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-06-20 02:09 ` Re: run pgindent on a regular basis / scripted manner Michael Paquier <[email protected]>
2023-08-11 20:59 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-08-11 21:25 ` Re: run pgindent on a regular basis / scripted manner Andres Freund <[email protected]>
2023-08-11 21:48 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-08-11 22:30 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-08-11 22:46 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-08-11 23:02 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-08-12 21:03 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-08-12 21:14 ` Re: run pgindent on a regular basis / scripted manner Andres Freund <[email protected]>
2023-08-12 22:46 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-08-13 00:13 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-08-13 00:20 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-08-13 00:53 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-08-13 14:33 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-08-13 23:33 ` Re: run pgindent on a regular basis / scripted manner Michael Paquier <[email protected]>
2023-10-16 00:52 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-10-17 00:45 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-10-17 01:22 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-10-17 02:57 ` Re: run pgindent on a regular basis / scripted manner Michael Paquier <[email protected]>
2023-10-17 10:49 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-10-18 19:15 ` Re: run pgindent on a regular basis / scripted manner Andres Freund <[email protected]>
2023-10-19 00:56 ` Re: run pgindent on a regular basis / scripted manner Andres Freund <[email protected]>
2023-10-19 01:29 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-10-19 02:18 ` Re: run pgindent on a regular basis / scripted manner Andres Freund <[email protected]>
2023-10-19 04:49 ` Re: run pgindent on a regular basis / scripted manner Andres Freund <[email protected]>
2023-10-17 12:45 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-10-17 14:03 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-10-17 14:23 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-10-17 14:55 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-10-17 15:00 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-10-17 15:01 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-10-17 15:15 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-10-17 15:23 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-10-17 15:47 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-10-18 16:45 ` Re: run pgindent on a regular basis / scripted manner Bruce Momjian <[email protected]>
2023-10-18 13:04 ` Re: run pgindent on a regular basis / scripted manner Vik Fearing <[email protected]>
2023-10-18 04:40 ` Re: run pgindent on a regular basis / scripted manner David Rowley <[email protected]>
2023-10-18 07:20 ` Re: run pgindent on a regular basis / scripted manner Peter Eisentraut <[email protected]>
2023-10-18 14:07 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-10-18 09:07 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-10-18 09:34 ` Re: run pgindent on a regular basis / scripted manner David Rowley <[email protected]>
2023-10-24 00:50 ` Re: run pgindent on a regular basis / scripted manner Jeff Davis <[email protected]>
2023-10-24 01:16 ` Re: run pgindent on a regular basis / scripted manner Amit Langote <[email protected]>
2023-10-24 04:31 ` Re: run pgindent on a regular basis / scripted manner Michael Paquier <[email protected]>
2023-10-25 09:42 ` Re: run pgindent on a regular basis / scripted manner Amit Kapila <[email protected]>
2023-10-24 13:53 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-08-14 14:04 ` Re: run pgindent on a regular basis / scripted manner Peter Eisentraut <[email protected]>
2023-08-14 19:58 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-08-11 23:20 ` Re: run pgindent on a regular basis / scripted manner Andres Freund <[email protected]>
2023-08-12 00:11 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-08-12 00:46 ` Re: run pgindent on a regular basis / scripted manner Andres Freund <[email protected]>
2023-08-14 14:08 ` Re: run pgindent on a regular basis / scripted manner Peter Eisentraut <[email protected]>
2023-08-11 23:17 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-08-12 15:57 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-10-28 15:47 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-10-28 16:09 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-10-29 14:22 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-08-11 23:18 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-08-15 20:31 ` Re: run pgindent on a regular basis / scripted manner Nathan Bossart <[email protected]>
2023-08-16 20:15 ` Re: run pgindent on a regular basis / scripted manner Peter Geoghegan <[email protected]>
2023-08-17 14:40 ` Re: run pgindent on a regular basis / scripted manner Nathan Bossart <[email protected]>
2023-04-22 14:39 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-04-22 15:10 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-04-22 15:37 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-04-22 19:52 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-04-22 19:58 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-04-23 15:04 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-04-23 15:16 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-04-23 15:29 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-04-24 14:09 ` Re: run pgindent on a regular basis / scripted manner Peter Eisentraut <[email protected]>
2023-04-24 14:14 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-04-26 07:38 ` Re: run pgindent on a regular basis / scripted manner Peter Eisentraut <[email protected]>
2023-04-26 13:27 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-04-26 19:44 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-04-28 18:08 ` Re: run pgindent on a regular basis / scripted manner Bruce Momjian <[email protected]>
2023-04-30 13:02 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-04-30 14:32 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-04-26 20:05 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-05-17 21:10 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-05-18 12:59 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-07 13:17 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-07 16:10 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-02-07 16:32 ` Re: run pgindent on a regular basis / scripted manner Jelte Fennema <[email protected]>
2023-02-07 16:57 ` Re: run pgindent on a regular basis / scripted manner Robert Haas <[email protected]>
2023-02-10 14:26 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
2023-02-07 15:25 ` Re: run pgindent on a regular basis / scripted manner Justin Pryzby <[email protected]>
2023-02-07 15:46 ` Re: run pgindent on a regular basis / scripted manner Tom Lane <[email protected]>
2023-02-07 15:46 ` Re: run pgindent on a regular basis / scripted manner Andrew Dunstan <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox