pg.ddx.io pgsql-sql@postgresql.org mailing list archive
help / color / mirror / Atom feedMax column number.
19+ messages / 6 participants
[nested] [flat]
* Max column number.
@ 2011-07-12 17:48 Miguel Angel Conte <diafano@gmail.com>
0 siblings, 2 replies; 19+ messages in thread
From: Miguel Angel Conte @ 2011-07-12 17:48 UTC (permalink / raw)
To: pgsql-sql
Hi,
I'm using postgresql 9 and I'd like to know if there is a way to "ask" if
when I'm going to add a column, I'm exceeding the max number of columns.
I've found that the max number of columns is 1600 and It's depends of the
data types.
I've made a test adding 1600 columns using different data types and I didn't
have any problem. The problem was when I tried to insert data.
ERROR: row is too big: size 8168, maximum size 8160
I know that the row limit size is 8k by default, but I need a way to prevent
this exception.
For example, my table have 600 columns and I insert a row with values for
each column and the row size is 8000. So, in this case If I want to prevent
errors, I can't add more columns, but to do that I need to know about this
limit situation.
Something like:
"If I'm not going to exceed the size limit, then I can add a new column"
Thanks in advance,
Miguel.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-12 17:52 Reinoud van Leeuwen <reinoud.v@n.leeuwen.net>
parent: Miguel Angel Conte <diafano@gmail.com>
1 sibling, 1 reply; 19+ messages in thread
From: Reinoud van Leeuwen @ 2011-07-12 17:52 UTC (permalink / raw)
To: pgsql-sql
On Tue, Jul 12, 2011 at 02:48:26PM -0300, Miguel Angel Conte wrote:
> Something like:
> "If I'm not going to exceed the size limit, then I can add a new column"
You want to add columns in your application? Are you sure you have the
right datamodel?
Reinoud
--
__________________________________________________
"Nothing is as subjective as reality"
Reinoud van Leeuwen reinoud.v@n.leeuwen.net
http://reinoud.van.leeuwen.net kvk 27320762
__________________________________________________
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-12 18:08 Miguel Angel Conte <diafano@gmail.com>
parent: Reinoud van Leeuwen <reinoud.v@n.leeuwen.net>
0 siblings, 3 replies; 19+ messages in thread
From: Miguel Angel Conte @ 2011-07-12 18:08 UTC (permalink / raw)
To: pgsql-sql
Unfortunately It's an inherited data model and I can't make any change for
now...
Thanks for your answer!
On Tue, Jul 12, 2011 at 2:52 PM, Reinoud van Leeuwen <
reinoud.v@n.leeuwen.net> wrote:
> On Tue, Jul 12, 2011 at 02:48:26PM -0300, Miguel Angel Conte wrote:
>
> > Something like:
> > "If I'm not going to exceed the size limit, then I can add a new column"
>
> You want to add columns in your application? Are you sure you have the
> right datamodel?
>
> Reinoud
> --
> __________________________________________________
> "Nothing is as subjective as reality"
> Reinoud van Leeuwen reinoud.v@n.leeuwen.net
> http://reinoud.van.leeuwen.net kvk 27320762
> __________________________________________________
>
> --
> Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-sql
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-12 18:11 ktm@rice.edu <ktm@rice.edu>
parent: Miguel Angel Conte <diafano@gmail.com>
2 siblings, 1 reply; 19+ messages in thread
From: ktm@rice.edu @ 2011-07-12 18:11 UTC (permalink / raw)
To: Miguel Angel Conte <diafano@gmail.com>; +Cc: pgsql-sql
On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
> Unfortunately It's an inherited data model and I can't make any change for
> now...
> Thanks for your answer!
>
> On Tue, Jul 12, 2011 at 2:52 PM, Reinoud van Leeuwen <
> reinoud.v@n.leeuwen.net> wrote:
>
> > On Tue, Jul 12, 2011 at 02:48:26PM -0300, Miguel Angel Conte wrote:
> >
> > > Something like:
> > > "If I'm not going to exceed the size limit, then I can add a new column"
> >
> > You want to add columns in your application? Are you sure you have the
> > right datamodel?
> >
> > Reinoud
> > --
Hi Reinoud,
I think you will need to write a function to grovel around in the database
to determine what the size of the row would be given a definition. Tedious
but straight-forward.
Regards,
Ken
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-12 18:25 Miguel Angel Conte <diafano@gmail.com>
parent: ktm@rice.edu <ktm@rice.edu>
0 siblings, 0 replies; 19+ messages in thread
From: Miguel Angel Conte @ 2011-07-12 18:25 UTC (permalink / raw)
To: pgsql-sql
>
> Hi Ken,
>
> Do you know a good way to get the max row size in a table?
> Or maybe I'll have to get this information from the metadata....
>
> thanks!
>
>
> On Tue, Jul 12, 2011 at 3:11 PM, ktm@rice.edu <ktm@rice.edu> wrote:
>
>> On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
>> > Unfortunately It's an inherited data model and I can't make any change
>> for
>> > now...
>> > Thanks for your answer!
>> >
>> > On Tue, Jul 12, 2011 at 2:52 PM, Reinoud van Leeuwen <
>> > reinoud.v@n.leeuwen.net> wrote:
>> >
>> > > On Tue, Jul 12, 2011 at 02:48:26PM -0300, Miguel Angel Conte wrote:
>> > >
>> > > > Something like:
>> > > > "If I'm not going to exceed the size limit, then I can add a new
>> column"
>> > >
>> > > You want to add columns in your application? Are you sure you have the
>> > > right datamodel?
>> > >
>> > > Reinoud
>> > > --
>> Hi Reinoud,
>>
>> I think you will need to write a function to grovel around in the database
>> to determine what the size of the row would be given a definition. Tedious
>> but straight-forward.
>>
>> Regards,
>> Ken
>>
>
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-12 20:16 Viktor Bojović <viktor.bojovic@gmail.com>
parent: Miguel Angel Conte <diafano@gmail.com>
1 sibling, 0 replies; 19+ messages in thread
From: Viktor Bojović @ 2011-07-12 20:16 UTC (permalink / raw)
To: Miguel Angel Conte <diafano@gmail.com>; +Cc: pgsql-sql
Hi Miguel,
maybe you can split table to two tables with one-to-one connection.
The another way is to create dynamic-attribute-tables which means to store
data in columns, not in rows.
On Tue, Jul 12, 2011 at 7:48 PM, Miguel Angel Conte <diafano@gmail.com>wrote:
> Hi,
>
> I'm using postgresql 9 and I'd like to know if there is a way to "ask" if
> when I'm going to add a column, I'm exceeding the max number of columns.
> I've found that the max number of columns is 1600 and It's depends of the
> data types.
> I've made a test adding 1600 columns using different data types and I
> didn't have any problem. The problem was when I tried to insert data.
>
> ERROR: row is too big: size 8168, maximum size 8160
>
> I know that the row limit size is 8k by default, but I need a way to
> prevent this exception.
>
> For example, my table have 600 columns and I insert a row with values for
> each column and the row size is 8000. So, in this case If I want to prevent
> errors, I can't add more columns, but to do that I need to know about this
> limit situation.
> Something like:
> "If I'm not going to exceed the size limit, then I can add a new column"
>
> Thanks in advance,
> Miguel.
>
--
---------------------------------------
Viktor Bojović
---------------------------------------
Wherever I go, Murphy goes with me
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-12 20:45 Scott Marlowe <scott.marlowe@gmail.com>
parent: Miguel Angel Conte <diafano@gmail.com>
2 siblings, 0 replies; 19+ messages in thread
From: Scott Marlowe @ 2011-07-12 20:45 UTC (permalink / raw)
To: Miguel Angel Conte <diafano@gmail.com>; +Cc: pgsql-sql
On Tue, Jul 12, 2011 at 12:08 PM, Miguel Angel Conte <diafano@gmail.com> wrote:
> Unfortunately It's an inherited data model and I can't make any change for
> now...
> Thanks for your answer!
when you can change it, look at hstore
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-12 21:12 Reinoud van Leeuwen <reinoud.v@n.leeuwen.net>
parent: Miguel Angel Conte <diafano@gmail.com>
2 siblings, 1 reply; 19+ messages in thread
From: Reinoud van Leeuwen @ 2011-07-12 21:12 UTC (permalink / raw)
To: pgsql-sql
On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
> Unfortunately It's an inherited data model and I can't make any change for
> now...
but by adding columns you *are* making changes to it...
Reinoud
--
__________________________________________________
"Nothing is as subjective as reality"
Reinoud van Leeuwen reinoud.v@n.leeuwen.net
http://reinoud.van.leeuwen.net kvk 27320762
__________________________________________________
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-12 23:29 Miguel Angel Conte <diafano@gmail.com>
parent: Reinoud van Leeuwen <reinoud.v@n.leeuwen.net>
0 siblings, 2 replies; 19+ messages in thread
From: Miguel Angel Conte @ 2011-07-12 23:29 UTC (permalink / raw)
To: pgsql-sql
Yes, sure. I mean, I can't change the whole process which creates columns
dynamically.
On Tue, Jul 12, 2011 at 6:12 PM, Reinoud van Leeuwen <
reinoud.v@n.leeuwen.net> wrote:
> On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
> > Unfortunately It's an inherited data model and I can't make any change
> for
> > now...
>
> but by adding columns you *are* making changes to it...
>
> Reinoud
> --
> __________________________________________________
> "Nothing is as subjective as reality"
> Reinoud van Leeuwen reinoud.v@n.leeuwen.net
> http://reinoud.van.leeuwen.net kvk 27320762
> __________________________________________________
>
> --
> Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-sql
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 05:56 Kevin Crain <kevin.crain1@gmail.com>
parent: Miguel Angel Conte <diafano@gmail.com>
1 sibling, 0 replies; 19+ messages in thread
From: Kevin Crain @ 2011-07-13 05:56 UTC (permalink / raw)
To: pgsql-sql
This is an unfortunate situation, you shouldn't be required to do
this, the people generating your requirements need to be more
informed. I would make damn sure you notify the stakeholders in this
project that the data model is screwed and needs a redesign. I agree
that you should split this table and do a join if you have no option
of redesigning this.
>
> On Tue, Jul 12, 2011 at 4:29 PM, Miguel Angel Conte <diafano@gmail.com> wrote:
>> Yes, sure. I mean, I can't change the whole process which creates columns
>> dynamically.
>>
>> On Tue, Jul 12, 2011 at 6:12 PM, Reinoud van Leeuwen
>> <reinoud.v@n.leeuwen.net> wrote:
>>>
>>> On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
>>> > Unfortunately It's an inherited data model and I can't make any change
>>> > for
>>> > now...
>>>
>>> but by adding columns you *are* making changes to it...
>>>
>>> Reinoud
>>> --
>>> __________________________________________________
>>> "Nothing is as subjective as reality"
>>> Reinoud van Leeuwen reinoud.v@n.leeuwen.net
>>> http://reinoud.van.leeuwen.net kvk 27320762
>>> __________________________________________________
>>>
>>> --
>>> Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
>>> To make changes to your subscription:
>>> http://www.postgresql.org/mailpref/pgsql-sql
>>
>>
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 06:28 Kevin Crain <kevin.crain1@gmail.com>
parent: Miguel Angel Conte <diafano@gmail.com>
1 sibling, 1 reply; 19+ messages in thread
From: Kevin Crain @ 2011-07-13 06:28 UTC (permalink / raw)
To: Miguel Angel Conte <diafano@gmail.com>; +Cc: pgsql-sql
I still can't imagine why you'd ever need this...could you explain
what this does? I'm just curious now....
On Tue, Jul 12, 2011 at 10:55 PM, Kevin Crain <kevin.crain1@gmail.com> wrote:
> This is an unfortunate situation, you shouldn't be required to do
> this, the people generating your requirements need to be more
> informed. I would make damn sure you notify the stakeholders in this
> project that the data model is screwed and needs a redesign. I agree
> that you should split this table and do a join if you have no option
> of redesigning this.
>
> On Tue, Jul 12, 2011 at 4:29 PM, Miguel Angel Conte <diafano@gmail.com> wrote:
>> Yes, sure. I mean, I can't change the whole process which creates columns
>> dynamically.
>>
>> On Tue, Jul 12, 2011 at 6:12 PM, Reinoud van Leeuwen
>> <reinoud.v@n.leeuwen.net> wrote:
>>>
>>> On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
>>> > Unfortunately It's an inherited data model and I can't make any change
>>> > for
>>> > now...
>>>
>>> but by adding columns you *are* making changes to it...
>>>
>>> Reinoud
>>> --
>>> __________________________________________________
>>> "Nothing is as subjective as reality"
>>> Reinoud van Leeuwen reinoud.v@n.leeuwen.net
>>> http://reinoud.van.leeuwen.net kvk 27320762
>>> __________________________________________________
>>>
>>> --
>>> Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
>>> To make changes to your subscription:
>>> http://www.postgresql.org/mailpref/pgsql-sql
>>
>>
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 15:45 Miguel Angel Conte <diafano@gmail.com>
parent: Kevin Crain <kevin.crain1@gmail.com>
0 siblings, 3 replies; 19+ messages in thread
From: Miguel Angel Conte @ 2011-07-13 15:45 UTC (permalink / raw)
To: Kevin Crain <kevin.crain1@gmail.com>; +Cc: pgsql-sql
Hi,
Thanks for your interest. This app load scv files which change every day
(sometimes the columns too). The sizes of these files are in avg 15MB. So,
We load something like 100MB each day. We tried to find a better solution
but we couldn't, becouse one of the our requirement is not to use a lot of
space. Also, the app is used to consult these information, and for our
particular type of select's queries, we get the best performance if the
information is all into a same row.
On Wed, Jul 13, 2011 at 3:28 AM, Kevin Crain <kevin.crain1@gmail.com> wrote:
> I still can't imagine why you'd ever need this...could you explain
> what this does? I'm just curious now....
>
> On Tue, Jul 12, 2011 at 10:55 PM, Kevin Crain <kevin.crain1@gmail.com>
> wrote:
> > This is an unfortunate situation, you shouldn't be required to do
> > this, the people generating your requirements need to be more
> > informed. I would make damn sure you notify the stakeholders in this
> > project that the data model is screwed and needs a redesign. I agree
> > that you should split this table and do a join if you have no option
> > of redesigning this.
> >
> > On Tue, Jul 12, 2011 at 4:29 PM, Miguel Angel Conte <diafano@gmail.com>
> wrote:
> >> Yes, sure. I mean, I can't change the whole process which creates
> columns
> >> dynamically.
> >>
> >> On Tue, Jul 12, 2011 at 6:12 PM, Reinoud van Leeuwen
> >> <reinoud.v@n.leeuwen.net> wrote:
> >>>
> >>> On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
> >>> > Unfortunately It's an inherited data model and I can't make any
> change
> >>> > for
> >>> > now...
> >>>
> >>> but by adding columns you *are* making changes to it...
> >>>
> >>> Reinoud
> >>> --
> >>> __________________________________________________
> >>> "Nothing is as subjective as reality"
> >>> Reinoud van Leeuwen reinoud.v@n.leeuwen.net
> >>> http://reinoud.van.leeuwen.net kvk 27320762
> >>> __________________________________________________
> >>>
> >>> --
> >>> Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
> >>> To make changes to your subscription:
> >>> http://www.postgresql.org/mailpref/pgsql-sql
> >>
> >>
> >
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 15:53 Reinoud van Leeuwen <reinoud.v@n.leeuwen.net>
parent: Miguel Angel Conte <diafano@gmail.com>
2 siblings, 0 replies; 19+ messages in thread
From: Reinoud van Leeuwen @ 2011-07-13 15:53 UTC (permalink / raw)
To: pgsql-sql
On Wed, Jul 13, 2011 at 12:45:45PM -0300, Miguel Angel Conte wrote:
> Hi,
>
> Thanks for your interest. This app load scv files which change every day
> (sometimes the columns too). The sizes of these files are in avg 15MB. So,
> We load something like 100MB each day. We tried to find a better solution
> but we couldn't, becouse one of the our requirement is not to use a lot of
> space. Also, the app is used to consult these information, and for our
> particular type of select's queries, we get the best performance if the
> information is all into a same row.
Still, you could normalize the data in a better way, and present it to
users or an application with views or stored procedures...
Reinoud
--
__________________________________________________
"Nothing is as subjective as reality"
Reinoud van Leeuwen reinoud.v@n.leeuwen.net
http://reinoud.van.leeuwen.net kvk 27320762
__________________________________________________
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 16:51 Scott Marlowe <scott.marlowe@gmail.com>
parent: Miguel Angel Conte <diafano@gmail.com>
2 siblings, 1 reply; 19+ messages in thread
From: Scott Marlowe @ 2011-07-13 16:51 UTC (permalink / raw)
To: Miguel Angel Conte <diafano@gmail.com>; +Cc: Kevin Crain <kevin.crain1@gmail.com>; pgsql-sql
On Wed, Jul 13, 2011 at 9:45 AM, Miguel Angel Conte <diafano@gmail.com> wrote:
> Hi,
> Thanks for your interest. This app load scv files which change every day
> (sometimes the columns too). The sizes of these files are in avg 15MB. So,
> We load something like 100MB each day. We tried to find a better solution
> but we couldn't, becouse one of the our requirement is not to use a lot of
> space. Also, the app is used to consult these information, and for our
> particular type of select's queries, we get the best performance if the
> information is all into a same row.
This definitely sounds like a job for something like hstore.
Also, can you drop and recreate the table every so often? That would
certainly help.
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 17:57 Miguel Angel Conte <diafano@gmail.com>
parent: Scott Marlowe <scott.marlowe@gmail.com>
0 siblings, 0 replies; 19+ messages in thread
From: Miguel Angel Conte @ 2011-07-13 17:57 UTC (permalink / raw)
To: Scott Marlowe <scott.marlowe@gmail.com>; +Cc: Kevin Crain <kevin.crain1@gmail.com>; pgsql-sql
I can't drop the table. I have to add as many columns as posible and when I
exceed the limit I have to create another table.
I've tried normalizing but then the join's cost is too big. I always need to
use all columns, so getting a all information into a single row it's the
most efficient solution.
On Wed, Jul 13, 2011 at 1:51 PM, Scott Marlowe <scott.marlowe@gmail.com>wrote:
> On Wed, Jul 13, 2011 at 9:45 AM, Miguel Angel Conte <diafano@gmail.com>
> wrote:
> > Hi,
> > Thanks for your interest. This app load scv files which change every day
> > (sometimes the columns too). The sizes of these files are in avg 15MB.
> So,
> > We load something like 100MB each day. We tried to find a better solution
> > but we couldn't, becouse one of the our requirement is not to use a lot
> of
> > space. Also, the app is used to consult these information, and for our
> > particular type of select's queries, we get the best performance if the
> > information is all into a same row.
>
> This definitely sounds like a job for something like hstore.
>
> Also, can you drop and recreate the table every so often? That would
> certainly help.
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 18:00 Kevin Crain <kevin.crain1@gmail.com>
parent: Miguel Angel Conte <diafano@gmail.com>
2 siblings, 1 reply; 19+ messages in thread
From: Kevin Crain @ 2011-07-13 18:00 UTC (permalink / raw)
To: Miguel Angel Conte <diafano@gmail.com>; +Cc: pgsql-sql
How are you determining the data types for these columns?
On Wed, Jul 13, 2011 at 8:45 AM, Miguel Angel Conte <diafano@gmail.com> wrote:
> Hi,
> Thanks for your interest. This app load scv files which change every day
> (sometimes the columns too). The sizes of these files are in avg 15MB. So,
> We load something like 100MB each day. We tried to find a better solution
> but we couldn't, becouse one of the our requirement is not to use a lot of
> space. Also, the app is used to consult these information, and for our
> particular type of select's queries, we get the best performance if the
> information is all into a same row.
>
> On Wed, Jul 13, 2011 at 3:28 AM, Kevin Crain <kevin.crain1@gmail.com> wrote:
>>
>> I still can't imagine why you'd ever need this...could you explain
>> what this does? I'm just curious now....
>>
>> On Tue, Jul 12, 2011 at 10:55 PM, Kevin Crain <kevin.crain1@gmail.com>
>> wrote:
>> > This is an unfortunate situation, you shouldn't be required to do
>> > this, the people generating your requirements need to be more
>> > informed. I would make damn sure you notify the stakeholders in this
>> > project that the data model is screwed and needs a redesign. I agree
>> > that you should split this table and do a join if you have no option
>> > of redesigning this.
>> >
>> > On Tue, Jul 12, 2011 at 4:29 PM, Miguel Angel Conte <diafano@gmail.com>
>> > wrote:
>> >> Yes, sure. I mean, I can't change the whole process which creates
>> >> columns
>> >> dynamically.
>> >>
>> >> On Tue, Jul 12, 2011 at 6:12 PM, Reinoud van Leeuwen
>> >> <reinoud.v@n.leeuwen.net> wrote:
>> >>>
>> >>> On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
>> >>> > Unfortunately It's an inherited data model and I can't make any
>> >>> > change
>> >>> > for
>> >>> > now...
>> >>>
>> >>> but by adding columns you *are* making changes to it...
>> >>>
>> >>> Reinoud
>> >>> --
>> >>> __________________________________________________
>> >>> "Nothing is as subjective as reality"
>> >>> Reinoud van Leeuwen reinoud.v@n.leeuwen.net
>> >>> http://reinoud.van.leeuwen.net kvk 27320762
>> >>> __________________________________________________
>> >>>
>> >>> --
>> >>> Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
>> >>> To make changes to your subscription:
>> >>> http://www.postgresql.org/mailpref/pgsql-sql
>> >>
>> >>
>> >
>
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 18:03 Miguel Angel Conte <diafano@gmail.com>
parent: Kevin Crain <kevin.crain1@gmail.com>
0 siblings, 2 replies; 19+ messages in thread
From: Miguel Angel Conte @ 2011-07-13 18:03 UTC (permalink / raw)
To: Kevin Crain <kevin.crain1@gmail.com>; +Cc: pgsql-sql
I have the metadata in the same csv.
On Wed, Jul 13, 2011 at 3:00 PM, Kevin Crain <kevin.crain1@gmail.com> wrote:
> How are you determining the data types for these columns?
>
> On Wed, Jul 13, 2011 at 8:45 AM, Miguel Angel Conte <diafano@gmail.com>
> wrote:
> > Hi,
> > Thanks for your interest. This app load scv files which change every day
> > (sometimes the columns too). The sizes of these files are in avg 15MB.
> So,
> > We load something like 100MB each day. We tried to find a better solution
> > but we couldn't, becouse one of the our requirement is not to use a lot
> of
> > space. Also, the app is used to consult these information, and for our
> > particular type of select's queries, we get the best performance if the
> > information is all into a same row.
> >
> > On Wed, Jul 13, 2011 at 3:28 AM, Kevin Crain <kevin.crain1@gmail.com>
> wrote:
> >>
> >> I still can't imagine why you'd ever need this...could you explain
> >> what this does? I'm just curious now....
> >>
> >> On Tue, Jul 12, 2011 at 10:55 PM, Kevin Crain <kevin.crain1@gmail.com>
> >> wrote:
> >> > This is an unfortunate situation, you shouldn't be required to do
> >> > this, the people generating your requirements need to be more
> >> > informed. I would make damn sure you notify the stakeholders in this
> >> > project that the data model is screwed and needs a redesign. I agree
> >> > that you should split this table and do a join if you have no option
> >> > of redesigning this.
> >> >
> >> > On Tue, Jul 12, 2011 at 4:29 PM, Miguel Angel Conte <
> diafano@gmail.com>
> >> > wrote:
> >> >> Yes, sure. I mean, I can't change the whole process which creates
> >> >> columns
> >> >> dynamically.
> >> >>
> >> >> On Tue, Jul 12, 2011 at 6:12 PM, Reinoud van Leeuwen
> >> >> <reinoud.v@n.leeuwen.net> wrote:
> >> >>>
> >> >>> On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
> >> >>> > Unfortunately It's an inherited data model and I can't make any
> >> >>> > change
> >> >>> > for
> >> >>> > now...
> >> >>>
> >> >>> but by adding columns you *are* making changes to it...
> >> >>>
> >> >>> Reinoud
> >> >>> --
> >> >>> __________________________________________________
> >> >>> "Nothing is as subjective as reality"
> >> >>> Reinoud van Leeuwen reinoud.v@n.leeuwen.net
> >> >>> http://reinoud.van.leeuwen.net kvk 27320762
> >> >>> __________________________________________________
> >> >>>
> >> >>> --
> >> >>> Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
> >> >>> To make changes to your subscription:
> >> >>> http://www.postgresql.org/mailpref/pgsql-sql
> >> >>
> >> >>
> >> >
> >
> >
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 18:07 Kevin Crain <kevin.crain1@gmail.com>
parent: Miguel Angel Conte <diafano@gmail.com>
1 sibling, 0 replies; 19+ messages in thread
From: Kevin Crain @ 2011-07-13 18:07 UTC (permalink / raw)
To: pgsql-sql
Is there any room for improvement in the data types?
On Wed, Jul 13, 2011 at 11:03 AM, Miguel Angel Conte <diafano@gmail.com> wrote:
> I have the metadata in the same csv.
>
> On Wed, Jul 13, 2011 at 3:00 PM, Kevin Crain <kevin.crain1@gmail.com> wrote:
>>
>> How are you determining the data types for these columns?
>>
>> On Wed, Jul 13, 2011 at 8:45 AM, Miguel Angel Conte <diafano@gmail.com>
>> wrote:
>> > Hi,
>> > Thanks for your interest. This app load scv files which change every day
>> > (sometimes the columns too). The sizes of these files are in avg 15MB.
>> > So,
>> > We load something like 100MB each day. We tried to find a better
>> > solution
>> > but we couldn't, becouse one of the our requirement is not to use a lot
>> > of
>> > space. Also, the app is used to consult these information, and for our
>> > particular type of select's queries, we get the best performance if the
>> > information is all into a same row.
>> >
>> > On Wed, Jul 13, 2011 at 3:28 AM, Kevin Crain <kevin.crain1@gmail.com>
>> > wrote:
>> >>
>> >> I still can't imagine why you'd ever need this...could you explain
>> >> what this does? I'm just curious now....
>> >>
>> >> On Tue, Jul 12, 2011 at 10:55 PM, Kevin Crain <kevin.crain1@gmail.com>
>> >> wrote:
>> >> > This is an unfortunate situation, you shouldn't be required to do
>> >> > this, the people generating your requirements need to be more
>> >> > informed. I would make damn sure you notify the stakeholders in this
>> >> > project that the data model is screwed and needs a redesign. I agree
>> >> > that you should split this table and do a join if you have no option
>> >> > of redesigning this.
>> >> >
>> >> > On Tue, Jul 12, 2011 at 4:29 PM, Miguel Angel Conte
>> >> > <diafano@gmail.com>
>> >> > wrote:
>> >> >> Yes, sure. I mean, I can't change the whole process which creates
>> >> >> columns
>> >> >> dynamically.
>> >> >>
>> >> >> On Tue, Jul 12, 2011 at 6:12 PM, Reinoud van Leeuwen
>> >> >> <reinoud.v@n.leeuwen.net> wrote:
>> >> >>>
>> >> >>> On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
>> >> >>> > Unfortunately It's an inherited data model and I can't make any
>> >> >>> > change
>> >> >>> > for
>> >> >>> > now...
>> >> >>>
>> >> >>> but by adding columns you *are* making changes to it...
>> >> >>>
>> >> >>> Reinoud
>> >> >>> --
>> >> >>> __________________________________________________
>> >> >>> "Nothing is as subjective as reality"
>> >> >>> Reinoud van Leeuwen reinoud.v@n.leeuwen.net
>> >> >>> http://reinoud.van.leeuwen.net kvk 27320762
>> >> >>> __________________________________________________
>> >> >>>
>> >> >>> --
>> >> >>> Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
>> >> >>> To make changes to your subscription:
>> >> >>> http://www.postgresql.org/mailpref/pgsql-sql
>> >> >>
>> >> >>
>> >> >
>> >
>> >
>
>
^ permalink raw reply [nested|flat] 19+ messages in thread
* Re: Max column number.
@ 2011-07-13 18:20 Kevin Crain <kevin.crain1@gmail.com>
parent: Miguel Angel Conte <diafano@gmail.com>
1 sibling, 0 replies; 19+ messages in thread
From: Kevin Crain @ 2011-07-13 18:20 UTC (permalink / raw)
To: Miguel Angel Conte <diafano@gmail.com>; +Cc: pgsql-sql
Have you tried changing the block size?
http://wiki.postgresql.org/wiki/FAQ#What_is_the_maximum_size_for_a_row.2C_a_table.2C_and_a_database....
On Wed, Jul 13, 2011 at 11:03 AM, Miguel Angel Conte <diafano@gmail.com> wrote:
> I have the metadata in the same csv.
>
> On Wed, Jul 13, 2011 at 3:00 PM, Kevin Crain <kevin.crain1@gmail.com> wrote:
>>
>> How are you determining the data types for these columns?
>>
>> On Wed, Jul 13, 2011 at 8:45 AM, Miguel Angel Conte <diafano@gmail.com>
>> wrote:
>> > Hi,
>> > Thanks for your interest. This app load scv files which change every day
>> > (sometimes the columns too). The sizes of these files are in avg 15MB.
>> > So,
>> > We load something like 100MB each day. We tried to find a better
>> > solution
>> > but we couldn't, becouse one of the our requirement is not to use a lot
>> > of
>> > space. Also, the app is used to consult these information, and for our
>> > particular type of select's queries, we get the best performance if the
>> > information is all into a same row.
>> >
>> > On Wed, Jul 13, 2011 at 3:28 AM, Kevin Crain <kevin.crain1@gmail.com>
>> > wrote:
>> >>
>> >> I still can't imagine why you'd ever need this...could you explain
>> >> what this does? I'm just curious now....
>> >>
>> >> On Tue, Jul 12, 2011 at 10:55 PM, Kevin Crain <kevin.crain1@gmail.com>
>> >> wrote:
>> >> > This is an unfortunate situation, you shouldn't be required to do
>> >> > this, the people generating your requirements need to be more
>> >> > informed. I would make damn sure you notify the stakeholders in this
>> >> > project that the data model is screwed and needs a redesign. I agree
>> >> > that you should split this table and do a join if you have no option
>> >> > of redesigning this.
>> >> >
>> >> > On Tue, Jul 12, 2011 at 4:29 PM, Miguel Angel Conte
>> >> > <diafano@gmail.com>
>> >> > wrote:
>> >> >> Yes, sure. I mean, I can't change the whole process which creates
>> >> >> columns
>> >> >> dynamically.
>> >> >>
>> >> >> On Tue, Jul 12, 2011 at 6:12 PM, Reinoud van Leeuwen
>> >> >> <reinoud.v@n.leeuwen.net> wrote:
>> >> >>>
>> >> >>> On Tue, Jul 12, 2011 at 03:08:36PM -0300, Miguel Angel Conte wrote:
>> >> >>> > Unfortunately It's an inherited data model and I can't make any
>> >> >>> > change
>> >> >>> > for
>> >> >>> > now...
>> >> >>>
>> >> >>> but by adding columns you *are* making changes to it...
>> >> >>>
>> >> >>> Reinoud
>> >> >>> --
>> >> >>> __________________________________________________
>> >> >>> "Nothing is as subjective as reality"
>> >> >>> Reinoud van Leeuwen reinoud.v@n.leeuwen.net
>> >> >>> http://reinoud.van.leeuwen.net kvk 27320762
>> >> >>> __________________________________________________
>> >> >>>
>> >> >>> --
>> >> >>> Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
>> >> >>> To make changes to your subscription:
>> >> >>> http://www.postgresql.org/mailpref/pgsql-sql
>> >> >>
>> >> >>
>> >> >
>> >
>> >
>
>
^ permalink raw reply [nested|flat] 19+ messages in thread
end of thread, other threads:[~2011-07-13 18:20 UTC | newest]
Thread overview: 19+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2011-07-12 17:48 Max column number. Miguel Angel Conte <diafano@gmail.com>
2011-07-12 17:52 ` Reinoud van Leeuwen <reinoud.v@n.leeuwen.net>
2011-07-12 18:08 ` Miguel Angel Conte <diafano@gmail.com>
2011-07-12 18:11 ` ktm@rice.edu <ktm@rice.edu>
2011-07-12 18:25 ` Miguel Angel Conte <diafano@gmail.com>
2011-07-12 20:45 ` Scott Marlowe <scott.marlowe@gmail.com>
2011-07-12 21:12 ` Reinoud van Leeuwen <reinoud.v@n.leeuwen.net>
2011-07-12 23:29 ` Miguel Angel Conte <diafano@gmail.com>
2011-07-13 05:56 ` Kevin Crain <kevin.crain1@gmail.com>
2011-07-13 06:28 ` Kevin Crain <kevin.crain1@gmail.com>
2011-07-13 15:45 ` Miguel Angel Conte <diafano@gmail.com>
2011-07-13 15:53 ` Reinoud van Leeuwen <reinoud.v@n.leeuwen.net>
2011-07-13 16:51 ` Scott Marlowe <scott.marlowe@gmail.com>
2011-07-13 17:57 ` Miguel Angel Conte <diafano@gmail.com>
2011-07-13 18:00 ` Kevin Crain <kevin.crain1@gmail.com>
2011-07-13 18:03 ` Miguel Angel Conte <diafano@gmail.com>
2011-07-13 18:07 ` Kevin Crain <kevin.crain1@gmail.com>
2011-07-13 18:20 ` Kevin Crain <kevin.crain1@gmail.com>
2011-07-12 20:16 ` Viktor Bojović <viktor.bojovic@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox