pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedHow to insert from linux to postgreSQL
6+ messages / 6 participants
[nested] [flat]
* How to insert from linux to postgreSQL
@ 2019-04-08 19:03 Campbell, Lance <lance@illinois.edu>
2019-04-08 19:18 ` Re: How to insert from linux to postgreSQL Ron <ronljohnsonjr@gmail.com>
2019-04-08 19:33 ` Re: How to insert from linux to postgreSQL Rui DeSousa <rui@crazybean.net>
2019-04-09 16:22 ` Re: How to insert from linux to postgreSQL Frank Alberto Rodriguez <franknigth@gmail.com>
0 siblings, 3 replies; 6+ messages in thread
From: Campbell, Lance @ 2019-04-08 19:03 UTC (permalink / raw)
To: pgsql-admin
PostgreSQL 10.x
Use case:
I want to read a log file on my linux server and write each line out to a row in a postgreSQL table. When I do inserts however I want to make sure I am using prepared statements to avoid SQL injections. I know how to do this is Java but I have not found an example using a linux bash scripts.
Does anyone have an example they could point me to?
Thanks,
Lance
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: How to insert from linux to postgreSQL
2019-04-08 19:03 How to insert from linux to postgreSQL Campbell, Lance <lance@illinois.edu>
@ 2019-04-08 19:18 ` Ron <ronljohnsonjr@gmail.com>
2019-04-08 19:20 ` Re: How to insert from linux to postgreSQL Scott Whitney <scott@journyx.com>
2 siblings, 1 reply; 6+ messages in thread
From: Ron @ 2019-04-08 19:18 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
On 4/8/19 2:03 PM, Campbell, Lance wrote:
>
> PostgreSQL 10.x
>
> Use case:
>
> I want to read a log file on my linux server and write each line out to a
> row in a postgreSQL table.
>
Presumably some manipulations need to be performed so that you can't just do
COPY FROM STDIN?
> When I do inserts however I want to make sure I am using prepared
> statements to avoid SQL injections. I know how to do this is Java but I
> have not found an example using a linux bash scripts.
>
> Does anyone have an example they could point me to?
>
I think the problem is that you need an open connection to use prepared
statements, and you can't do that in bash. What about using Python? (After
all, that's a scripting language, too.)
--
Angular momentum makes the world go 'round.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: How to insert from linux to postgreSQL
2019-04-08 19:03 How to insert from linux to postgreSQL Campbell, Lance <lance@illinois.edu>
2019-04-08 19:18 ` Re: How to insert from linux to postgreSQL Ron <ronljohnsonjr@gmail.com>
@ 2019-04-08 19:20 ` Scott Whitney <scott@journyx.com>
2019-04-09 16:07 ` Re: How to insert from linux to postgreSQL Peter M. Groen <peter@osdev.nl>
0 siblings, 1 reply; 6+ messages in thread
From: Scott Whitney @ 2019-04-08 19:20 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>; Ron <ronljohnsonjr@gmail.com>
Is the log file "ready" for INSERT, line by line? If not, you need to preprocess it. The 2 solutions I would use are psycopg for Python or sed + INSERT depending.
________________________________
From: Ron <ronljohnsonjr@gmail.com>
Sent: Monday, April 8, 2019 2:18:18 PM
To: pgsql-admin@lists.postgresql.org
Subject: Re: How to insert from linux to postgreSQL
On 4/8/19 2:03 PM, Campbell, Lance wrote:
PostgreSQL 10.x
Use case:
I want to read a log file on my linux server and write each line out to a row in a postgreSQL table.
Presumably some manipulations need to be performed so that you can't just do COPY FROM STDIN?
When I do inserts however I want to make sure I am using prepared statements to avoid SQL injections. I know how to do this is Java but I have not found an example using a linux bash scripts.
Does anyone have an example they could point me to?
I think the problem is that you need an open connection to use prepared statements, and you can't do that in bash. What about using Python? (After all, that's a scripting language, too.)
--
Angular momentum makes the world go 'round.
[http://journyx.com/emailImages/ScottWhitney.png]
[http://journyx.com/emailImages/linkedin.png]<https://www.linkedin.com/company/journyx-inc/; [http://journyx.com/emailImages/twitter.png] <https://www.twitter.com/journyxinc;
[http://journyx.com/emailImages/bluebar.jpg]
Scott Whitney
Director of IT
[http://journyx.com/emailImages/journyx.jpg]
scott@journyx.com<mailto:scott@journyx.com> | +1 (800) 755-9878
7600 Burnet Road, Ste. 300
Austin, TX 78757
Visit us at www.journyx.com<https://www.journyx.com/;
To unsubscribe from Journyx promotional emails, click here <http://journyx.com/communication-preferences; or visit http://journyx.com/communication-preferences.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: How to insert from linux to postgreSQL
2019-04-08 19:03 How to insert from linux to postgreSQL Campbell, Lance <lance@illinois.edu>
2019-04-08 19:18 ` Re: How to insert from linux to postgreSQL Ron <ronljohnsonjr@gmail.com>
2019-04-08 19:20 ` Re: How to insert from linux to postgreSQL Scott Whitney <scott@journyx.com>
@ 2019-04-09 16:07 ` Peter M. Groen <peter@osdev.nl>
0 siblings, 0 replies; 6+ messages in thread
From: Peter M. Groen @ 2019-04-09 16:07 UTC (permalink / raw)
To: Scott Whitney <scott@journyx.com>; +Cc: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>; Ron <ronljohnsonjr@gmail.com>
Or just use PERL as it was designed to (pre)process large amounts of
texts... psql support is pretty great....
--
Open Systems Development B.V.
Peter M. Groen
Rooseveltstraat 18P
2321 BM Leiden
Telef :
Mobile : +31 6 811 0 3537
Email : pgroen@osdev.nl
On Mon, 8 Apr 2019 19:20:03 +0000, Scott Whitney wrote :
>Is the log file "ready" for INSERT, line by line? If not, you need to
>preprocess it. The 2 solutions I would use are psycopg for Python or
>sed + INSERT depending. ________________________________ From: Ron
><ronljohnsonjr@gmail.com> Sent: Monday, April 8, 2019 2:18:18 PM
>To: pgsql-admin@lists.postgresql.org
>Subject: Re: How to insert from linux to postgreSQL
>
>On 4/8/19 2:03 PM, Campbell, Lance wrote:
>PostgreSQL 10.x
>
>Use case:
>I want to read a log file on my linux server and write each line out
>to a row in a postgreSQL table.
>
>Presumably some manipulations need to be performed so that you can't
>just do COPY FROM STDIN?
>
>When I do inserts however I want to make sure I am using prepared
>statements to avoid SQL injections. I know how to do this is Java but
>I have not found an example using a linux bash scripts.
>
>Does anyone have an example they could point me to?
>
>I think the problem is that you need an open connection to use
>prepared statements, and you can't do that in bash. What about using
>Python? (After all, that's a scripting language, too.)
>
>
>--
>Angular momentum makes the world go 'round.
>
>
>
>[http://journyx.com/emailImages/ScottWhitney.png]
>
>[http://journyx.com/emailImages/linkedin.png]<https://www.linkedin.com/company/journyx-inc/;
>[http://journyx.com/emailImages/twitter.png]
><https://www.twitter.com/journyxinc;
> [http://journyx.com/emailImages/bluebar.jpg]
>
>Scott Whitney
>Director of IT
>
>[http://journyx.com/emailImages/journyx.jpg]
>
>scott@journyx.com<mailto:scott@journyx.com> | +1 (800) 755-9878
>
>7600 Burnet Road, Ste. 300
>Austin, TX 78757
>
>Visit us at www.journyx.com<https://www.journyx.com/;
>
>
>
>To unsubscribe from Journyx promotional emails, click here
><http://journyx.com/communication-preferences; or visit
>http://journyx.com/communication-preferences.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: How to insert from linux to postgreSQL
2019-04-08 19:03 How to insert from linux to postgreSQL Campbell, Lance <lance@illinois.edu>
@ 2019-04-08 19:33 ` Rui DeSousa <rui@crazybean.net>
2 siblings, 0 replies; 6+ messages in thread
From: Rui DeSousa @ 2019-04-08 19:33 UTC (permalink / raw)
To: Campbell, Lance <lance@illinois.edu>; +Cc: pgsql-admin
I would use psql \copy command within a shell script or just not use it.
Personally, I would choose Perl or GoLang if it requires parsing the data before inserting.
Perl: https://www.perl.com/pub/1999/10/DBI.html/ <https://www.perl.com/pub/1999/10/DBI.html/;
Go: https://golang.org/pkg/database/sql/#DB.Prepare <https://golang.org/pkg/database/sql/#DB.Prepare;
> On Apr 8, 2019, at 3:03 PM, Campbell, Lance <lance@illinois.edu> wrote:
>
> PostgreSQL 10.x
>
> Use case:
> I want to read a log file on my linux server and write each line out to a row in a postgreSQL table. When I do inserts however I want to make sure I am using prepared statements to avoid SQL injections. I know how to do this is Java but I have not found an example using a linux bash scripts.
>
> Does anyone have an example they could point me to?
>
> Thanks,
>
> Lance
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: How to insert from linux to postgreSQL
2019-04-08 19:03 How to insert from linux to postgreSQL Campbell, Lance <lance@illinois.edu>
@ 2019-04-09 16:22 ` Frank Alberto Rodriguez <franknigth@gmail.com>
2 siblings, 0 replies; 6+ messages in thread
From: Frank Alberto Rodriguez @ 2019-04-09 16:22 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
Then why not using java? Just exec the .jar with the terminal params
and do de job in java, or read a configuration file.Or you could use
PL/java.
https://tada.github.io/pljava/ Greetings
On Mon, 2019-04-08 at 19:03 +0000, Campbell, Lance wrote:
> PostgreSQL 10.x
>
> Use case:
> I want to read a log file on my linux server and write each line out
> to a row in a postgreSQL table. When I do inserts however I want to
> make sure I am using prepared statements to avoid SQL injections.
> I know how to do this is Java but I have not found an example using
> a linux bash scripts.
>
>
> Does anyone have an example they could point me to?
>
> Thanks,
>
> Lance
>
>
>
>
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2019-04-09 16:22 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-04-08 19:03 How to insert from linux to postgreSQL Campbell, Lance <lance@illinois.edu>
2019-04-08 19:18 ` Ron <ronljohnsonjr@gmail.com>
2019-04-08 19:20 ` Scott Whitney <scott@journyx.com>
2019-04-09 16:07 ` Peter M. Groen <peter@osdev.nl>
2019-04-08 19:33 ` Rui DeSousa <rui@crazybean.net>
2019-04-09 16:22 ` Frank Alberto Rodriguez <franknigth@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox