public inbox for [email protected]
help / color / mirror / Atom feedBest practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary
4+ messages / 3 participants
[nested] [flat]
* Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary
@ 2020-11-03 21:38 Francois-Guillaume Ribreau <[email protected]>
2020-11-03 21:47 ` Re: Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary Tom Lane <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Francois-Guillaume Ribreau @ 2020-11-03 21:38 UTC (permalink / raw)
To: [email protected]
Hello!
(PostgreSQL rocks)
I wonder what is the easiest way to extract and (ab)use the raw_parser
function out of postgresql codebase, as a library, so I can use it from my
own code in Rust.
My C is rusty (pun intended) so I tried this:
cd src/backend/parser
gcc -I../../include -fpic -c parser.c
so raw_parser is present (yey):
nm parser.o | grep raw_parser
0000000000000000 T _raw_parser
but I missed lots of other implementation files (doh) ^^
nm parser.o | grep U
U _ScanKeywordTokens
U _ScanKeywords
U _base_yyparse
U _cancel_scanner_errposition_callback
U _core_yylex
U _errcode
U _errfinish
U _errhint
U _errmsg
U _errmsg_internal
U _errstart
U _isxdigit
U _palloc
U _parser_init
U _pg_unicode_to_server
U _repalloc
U _scanner_errposition
U _scanner_finish
U _scanner_init
U _scanner_isspace
U _scanner_yyerror
U _setup_scanner_errposition_callback
U _strlen
U _truncate_identifier
I started to include them one by one but the task is tedious and I'm pretty
sure there is an easier way :)
So I did take a look at various makefile in contrib/ folder but I'm not
sure adapting them will do want I want, I do not want to make an extension
for postgresql but instead generate an .a library that I can access from
Rust through FFI.
Does anyone tried this before?
PS: If you are interested, here is the repository:
https://github.com/FGRibreau/poc-pgsql-parser
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary
2020-11-03 21:38 Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary Francois-Guillaume Ribreau <[email protected]>
@ 2020-11-03 21:47 ` Tom Lane <[email protected]>
2020-11-03 22:14 ` Re: Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary Francois-Guillaume Ribreau <[email protected]>
2021-01-06 13:40 ` Re: Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary Pól Ua Laoínecháin <[email protected]>
0 siblings, 2 replies; 4+ messages in thread
From: Tom Lane @ 2020-11-03 21:47 UTC (permalink / raw)
To: Francois-Guillaume Ribreau <[email protected]>; +Cc: [email protected]
Francois-Guillaume Ribreau <[email protected]> writes:
> I wonder what is the easiest way to extract and (ab)use the raw_parser
> function out of postgresql codebase, as a library, so I can use it from my
> own code in Rust.
You're not the first to have thought of that. I'm failing to locate
any relevant threads in our archives, but I distinctly recall having
heard of somebody who'd made a standalone version of our lexer+grammar.
You might try searching on github.
(I make no warranties about how up-to-date any such project may be.)
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary
2020-11-03 21:38 Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary Francois-Guillaume Ribreau <[email protected]>
2020-11-03 21:47 ` Re: Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary Tom Lane <[email protected]>
@ 2020-11-03 22:14 ` Francois-Guillaume Ribreau <[email protected]>
1 sibling, 0 replies; 4+ messages in thread
From: Francois-Guillaume Ribreau @ 2020-11-03 22:14 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: [email protected]
wow thanks, found it (I think) https://github.com/lfittl/libpg_query !
On Tue, Nov 3, 2020 at 10:47 PM Tom Lane <[email protected]> wrote:
> Francois-Guillaume Ribreau <[email protected]> writes:
> > I wonder what is the easiest way to extract and (ab)use the raw_parser
> > function out of postgresql codebase, as a library, so I can use it from
> my
> > own code in Rust.
>
> You're not the first to have thought of that. I'm failing to locate
> any relevant threads in our archives, but I distinctly recall having
> heard of somebody who'd made a standalone version of our lexer+grammar.
> You might try searching on github.
>
> (I make no warranties about how up-to-date any such project may be.)
>
> regards, tom lane
>
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary
2020-11-03 21:38 Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary Francois-Guillaume Ribreau <[email protected]>
2020-11-03 21:47 ` Re: Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary Tom Lane <[email protected]>
@ 2021-01-06 13:40 ` Pól Ua Laoínecháin <[email protected]>
1 sibling, 0 replies; 4+ messages in thread
From: Pól Ua Laoínecháin @ 2021-01-06 13:40 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Francois-Guillaume Ribreau <[email protected]>; [email protected]
Hi all,
> You're not the first to have thought of that. I'm failing to locate
> any relevant threads in our archives, but I distinctly recall having
> heard of somebody who'd made a standalone version of our lexer+grammar.
> You might try searching on github.
Funnily enough, I was only reading about this yesterday - sometimes
wandering through the interweb has its benefits! :-)
The project is DuckDB https://duckdb.org/.
Specifically, this page: https://duckdb.org/docs/why_duckdb.html#duckdbissimple
"SQL Parser: We use the PostgreSQL parser that was repackaged as a
stand-alone library. The translation to our own parse tree is inspired
by Peloton."
The stand-alone library they use is from here (linked in text above):
https://github.com/lfittl/libpg_query
> (I make no warranties about how up-to-date any such project may be.)
Seems interesting and active - DuckDB's last GitHub update 18 days
ago! Last update for the libpg_query is 3 years, however DuckDB appear
to be maintaining their own fork, available here:
https://github.com/cwida/duckdb/tree/master/third_party/libpg_query -
last update 26 days ago!
HTH,
Pól...
> regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2021-01-06 13:40 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-11-03 21:38 Best practice: call an internal postgresql function (e.g. raw_parser) from another C/Rust binary Francois-Guillaume Ribreau <[email protected]>
2020-11-03 21:47 ` Tom Lane <[email protected]>
2020-11-03 22:14 ` Francois-Guillaume Ribreau <[email protected]>
2021-01-06 13:40 ` Pól Ua Laoínecháin <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox