pg.ddx.io pgsql-sql@postgresql.org mailing list archive
help / color / mirror / Atom feedhow to store more than 3 MB of character data in Postgres Varchar field
4+ messages / 3 participants
[nested] [flat]
* how to store more than 3 MB of character data in Postgres Varchar field
@ 2005-06-06 10:17 Vadivel Subramaniam <vadivel.subramaniam@flextronicssoftware.com>
0 siblings, 2 replies; 4+ messages in thread
From: Vadivel Subramaniam @ 2005-06-06 10:17 UTC (permalink / raw)
To: pgsql-sql
Hi,
We have a requirement wherein we have to store around 3 MB of data in
Postgres database.
We had gone through the postgres website
http://www.postgresql.org/docs/7.4/interactive/datatype-character.html#DATATYPE-CHARACTER-SPECIAL-TA...
The above link says "varchar" can store upto 1 GB of data. But when i try
to insert using ODBC API's the varchar field takes
only 65484 characters. But i could insert more characters using INSERT
SQL command. Even in this case when we fetch the data it reads only 65540
characters. Remaining data is not read from the varchar field.
We have tried in 2 ways
1. table is created without any limit specified for varchar.
create table details (name varchar, data varchar);
2. create table utilntmlscripts (name character varying, data character
varying(10485770));
ERROR: length for type 'varchar' cannot exceed 10485760
It's not allowing more than 10 MB of size during table creation.
Is there anything needs to be configured to make the varchar field to
support upto 1 GB.
Even if it does not support 1 GB, can we store & retrieve atleast 10 MB of
character data
in Varchar field?
Please send the details of how this can be acheived.
Postgres Version 7.4.2
Accessing using UnixODBC driver from C++ application.
thanks,
Vadivel.
*********************** FSS-Private ***********************
"DISCLAIMER: This message is proprietary to Flextronics Software Systems
Limited (FSS) and is intended solely for the use of the
individual to whom it is addressed. It may contain privileged or
confidential information and should not be circulated or used for
any purpose other than for what it is intended. If you have received this
message in error, please notify the originator immediately.
If you are not the intended recipient, you are notified that you are
strictly prohibited from using, copying, altering, or disclosing
the contents of this message. FSS accepts no responsibility for loss or
damage arising from the use of the information transmitted
by this email including damage from virus."
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: how to store more than 3 MB of character data in Postgres
@ 2005-06-06 10:48 Richard Huxton <dev@archonet.com>
parent: Vadivel Subramaniam <vadivel.subramaniam@flextronicssoftware.com>
1 sibling, 0 replies; 4+ messages in thread
From: Richard Huxton @ 2005-06-06 10:48 UTC (permalink / raw)
To: Vadivel Subramaniam <vadivel.subramaniam@flextronicssoftware.com>; +Cc: pgsql-sql
Vadivel Subramaniam wrote:
>
> Hi,
>
> We have a requirement wherein we have to store around 3 MB of data
> in Postgres database.
>
> We had gone through the postgres website
> http://www.postgresql.org/docs/7.4/interactive/datatype-character.html#DATATYPE-CHARACTER-SPECIAL-TA...
>
> The above link says "varchar" can store upto 1 GB of data.
I don't think it does, it says the "longest possible character string"
which covers type "text".
> But when
> i try to insert using ODBC API's the varchar field takes only 65484
> characters. But i could insert more characters using INSERT SQL
> command. Even in this case when we fetch the data it reads only
> 65540 characters. Remaining data is not read from the varchar field.
So, your problem is with the ODBC interface rather than actually
inserting the data.
> We have tried in 2 ways
>
> 1. table is created without any limit specified for varchar. create
> table details (name varchar, data varchar);
>
> 2. create table utilntmlscripts (name character varying, data
> character varying(10485770)); ERROR: length for type 'varchar'
> cannot exceed 10485760 It's not allowing more than 10 MB of size
> during table creation.
>
>
> Is there anything needs to be configured to make the varchar field to
> support upto 1 GB. Even if it does not support 1 GB, can we store &
> retrieve atleast 10 MB of character data in Varchar field?
Check your ODBC settings and try type "text" instead of "varchar". I
think the problem is with ODBC restrictions rather than anything else.
You should be able to get type "text" to map to a "Memo" type or
similar, which should hold more than 64k IIRC.
Failing that, try internal type "bytea" and large-object on the ODBC end.
--
Richard Huxton
Archonet Ltd
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: how to store more than 3 MB of character data in Postgres
@ 2005-06-06 11:46 Richard Huxton <dev@archonet.com>
0 siblings, 0 replies; 4+ messages in thread
From: Richard Huxton @ 2005-06-06 11:46 UTC (permalink / raw)
To: Vadivel Subramaniam <vadivel.subramaniam@flextronicssoftware.com>; pgsql-sql
Vadivel Subramaniam wrote:
>
> I assume, it could not be a problem with ODBC. I am able to store 2.5 MB of
> data into Oracle using the same ODBC APIs.
Well, it certainly isn't to do with PG itself:
$ cat bigtest.pl
#!/usr/bin/perl
print "CREATE TABLE foo (a int, b varchar);\n";
print "INSERT INTO foo VALUES (1, '".('a' x 5000000)."');\n";
$ perl bigtest.pl | psql -Urichardh richardh
CREATE TABLE
INSERT 3634376 1
$ psql -Urichardh richardh
richardh=> SELECT a,length(b) FROM foo;
a | length
---+---------
1 | 5000000
(1 row)
> Only difference is that Oracle supports CLOB datatype, that is mapped to
> LONG VARCHAR in ODBC.
> The input parameter during insert is bound using the below ODBC API
> retStatus = SQLBindParameter(dStmtHandle, 1, SQL_PARAM_INPUT,
> SQL_C_CHAR, SQL_LONGVARCHAR, dLen, 1, pData, 0, &dLen);
>
> For PostGres also the same API is used. Here varchar/text at the Postgres
> schema is mapped to LONGVARCHAR at ODBC.
> We can not go for bytea etc, since the data is in character format.
>
> I tried using "text" also at the postgres schema, still the impact is same
> (only stores around 64K).
> Could you please provide some example, how the large character data ( more
> than 1 MB ) can be inserted to text/varchar (in PostGres) using ODBC?
Hmm - looking back at your original email, I see you're using the
UnixODBC driver. Might be worth trying the PG version:
http://gborg.postgresql.org/project/psqlodbc/genpage.php?downloads
First step though, should be to check whether there are any
configuration settings that apply to longvarchar, because it sounds like
the odbc driver should be mapping to that.
I can't help you with finding those settings, since I have only ever
used the "official" PG driver on Windows.
--
Richard Huxton
Archonet Ltd
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: how to store more than 3 MB of character data in Postgres Varchar field
@ 2005-06-06 14:31 Tom Lane <tgl@sss.pgh.pa.us>
parent: Vadivel Subramaniam <vadivel.subramaniam@flextronicssoftware.com>
1 sibling, 0 replies; 4+ messages in thread
From: Tom Lane @ 2005-06-06 14:31 UTC (permalink / raw)
To: Vadivel Subramaniam <vadivel.subramaniam@flextronicssoftware.com>; +Cc: pgsql-sql
Vadivel Subramaniam <vadivel.subramaniam@flextronicssoftware.com> writes:
> 2. create table utilntmlscripts (name character varying, data character
> varying(10485770));
> ERROR: length for type 'varchar' cannot exceed 10485760
> It's not allowing more than 10 MB of size during table creation.
Use type "text", or equivalently varchar with no length specification.
The 1-MB limit on what you can write as a length spec is somewhat
arbitrary, but certainly an enforced length spec larger than that
would be a completely arbitrary number as well ...
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2005-06-06 14:31 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-06-06 10:17 how to store more than 3 MB of character data in Postgres Varchar field Vadivel Subramaniam <vadivel.subramaniam@flextronicssoftware.com>
2005-06-06 10:48 ` Re: how to store more than 3 MB of character data in Postgres Richard Huxton <dev@archonet.com>
2005-06-06 14:31 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-06-06 11:46 Re: how to store more than 3 MB of character data in Postgres Richard Huxton <dev@archonet.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox