Message-ID: <27a573c0-70f0-4d5c-b9ca-d8c4e2b999f6@dunslane.net>
Date: Wed, 8 Jan 2025 05:42:09 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Coccinelle for PostgreSQL development [1/N]: coccicheck.py
To: Mats Kindahl <mats@timescale.com>,
 PostgreSQL Development <pgsql-hackers@postgresql.org>
References: <CA+14426e8dbmMjGLu8jO8CQAb9-FKiM-CQhvQHUB=3OnJwWpzQ@mail.gmail.com>
From: Andrew Dunstan <andrew@dunslane.net>
Content-Language: en-US
Autocrypt: addr=andrew@dunslane.net; keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNJEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PsLAmwQTAQgARQIbAwIX
 gAIZAQULCQgHAgMiAgEGFQoJCAsCBBYCAwECHgcWIQTkPlhGHfx8v0RpFaWZ+n/LWfw7gQUC
 ZFlxxwUJGVGAbgAKCRCZ+n/LWfw7gXikB/9ZdcUy6CTBFIIuL/bVsc1eLEW/gJBjJBF6HxNY
 xgEkAgXAp4Lg4A5U+QB9GouFr7+GYxF0BU4hzoGhNPUWltxnHdMWP8nC/38LAqgMi8L/bbsm
 HW5YPBdWYaAZAPJQVfOAgjTbRUb26KSprpyrrJKW0ZmrZfjhNPcQ72jpWzoPLQqx2X6B0fru
 1jq+cBh8lb6r1mJTim1T3JIn+F/v5VpdQS+EL8xqsHkfzKjIPsW3CIXpkypSk6saA55Rkkbl
 26AW8ftPVB0Q6Lnn6FLt9CP0MGNixBQ55yq8r1K+nCBvCCjvQjM8RDm0UUum0WNl+ifQgTLO
 E8TWEnwVtkBf+3QWzsBNBE7KWFkBCADRnOM0FCzsYW6jtncg+dWIagjUZpvaClmqn/sJluLa
 Q3v1VXMQJzYs3eC1gh386W+XBwLRpDj3jzH81lX+p73Re3d3oJW7X+ffsxuzu5ZVdMUkqBYo
 nkAbKxr6gyJ12F/+JkUVzLcoTN+d/7YsQvUVi7NaKH8mJgjz112O4fUe3p9wfAaFa0RXHc5S
 GPzRTYRRlv/XZBIho4J2tkZOnteZJZ+GbxQVlINt6fd8P6al3MWOvpP/ExJPguEfjOsO6Njy
 xjo3WfpD4lHMOR/Oc3/8mScEF84rF2jXbsFgelWnbPWAvXY+pD0dXOFRkagGmC/viwBDqq5b
 5tk76kKmUbZxABEBAAHCwHwEGAEIACYCGwwWIQTkPlhGHfx8v0RpFaWZ+n/LWfw7gQUCZFlx
 5wUJGVGAjgAKCRCZ+n/LWfw7gf+iB/4g8CPY5jihf5r/8EsoIGe2H+dpVmpPF8YGBzTIvCz/
 fQoOq8AX/pE76QEuFnFZWfjw+wgBXgCVmkox2Eflkk6z4ND3pcwGZ6CfCxTQCDk/dij+2DQ4
 6bmDCy/sBgcbz9mTpoLC11HLoPae6YN9nBNQRZDcEFEu54OaVOqlIdbA6m+POIBCXZdHOFc0
 WoDTgxHRzC1jgQNidyd6tKqcsVJs0dzF0oKTmFFmUAqTdJO12LBuNA1rlqrR3EtpYk8B/wtS
 5dIMD7Q8hwQpL+4C6GNpb6ZKnPkLi47pDOLhz2qBrqN+rqUEsT3YnExYpzj5yOBi+FlmV1Hw
 49QYe1sn2ZPs
In-Reply-To: <CA+14426e8dbmMjGLu8jO8CQAb9-FKiM-CQhvQHUB=3OnJwWpzQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/27a573c0-70f0-4d5c-b9ca-d8c4e2b999f6%40dunslane.net>
Precedence: bulk


On 2025-01-07 Tu 2:44 PM, Mats Kindahl wrote:
> I got some time over during the holidays, so I spent some of it 
> doing something I've been thinking about for a while.
>
> For those of you that are not aware of it: Coccinelle is a tool for 
> pattern matching and text transformation for C code and can be used 
> for detection of problematic programming patterns and to make complex, 
> tree-wide patches easy. It is aware of the structure of C code and is 
> better suited to make complicated changes than what is possible using 
> normal text substitution tools like Sed and Perl.
>
> Coccinelle have been successfully been used in the Linux project since 
> 2008 and is now an established tool for Linux development and a large 
> number of semantic patches have been added to the source tree to 
> capture everything from generic issues (like eliminating the redundant 
> A in expressions like "!A || (A && B)") to more Linux-specific 
> problems like adding a missing call to kfree().
>
> Although PostgreSQL is nowhere the size of the Linux kernel, it is 
> nevertheless of a significant size and would benefit from 
> incorporating Coccinelle into the development. I noticed it's been 
> used in a few cases way back (like 10 years back) to fix issues in the 
> PostgreSQL code, but I thought it might be useful to make it part of 
> normal development practice to, among other things:
>
> - Identify and correct bugs in the source code both during development 
> and review.
> - Make large-scale changes to the source tree to improve the code 
> based on new insights.
> - Encode and enforce APIs by ensuring that function calls are used 
> correctly.
> - Use improved coding patterns for more efficient code.
> - Allow extensions to automatically update code for later PostgreSQL 
> versions.
>
> To that end, I created a series of patches to show how it could be 
> used in the PostgreSQL tree. It is a lot easier to discuss concrete 
> code and I split it up into separate messages since that makes it 
> easier to discuss each individual patch. The series contains code to 
> make it easy to work with Coccinelle during development and reviews, 
> as well as examples of semantic patches that capture problems, 
> demonstrate how to make large-scale changes, how to enforce APIs, and 
> also improve some coding patterns.
>
> This first patch contains the coccicheck.py script, which is a 
> re-implementation of the coccicheck script that the Linux kernel uses. 
> We cannot immediately use the coccicheck script since it is quite 
> closely tied to the Linux source code tree and we need to have 
> something that both supports autoconf and Meson. Since Python seems to 
> be used more and more in the tree, it seems to be the most natural 
> choice. (I have no strong opinion on what language to use, but think 
> it would be good to have something that is as platform-independent as 
> possible.)
>
> The intention is that we should be able to use the Linux semantic 
> patches directly, so it supports the "Requires" and "Options" 
> keywords, which can be used to require a specific version of spatch(1) 
> and add options to the execution of that semantic patch, respectively.
>


Please don't start multiple threads like this. If you want to submit a 
set of patches for a single feature, send them all as attachments in a 
single email. Otherwise this just makes life hard for threading email 
readers.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com