[Ерохин Денис Владимирович <erohin-d@datagile.ru>]

Hello,

 

There is a problem on receiving large tsvector from binary format with
getting error "invalid tsvector: maximum total lexeme length exceeded".
Required simple steps to reproduce problem:

- Make table with one column of type 'tsvector'

- Add row with large tsvector (900 Kb will be enougth)

- Save table to file with "copy to with binary"

- Restore table from file with "copy from with binary"

 

At last step we can't restore a legal tsvector because there is error of
required memory size calculation for tsvector during reading of its
binary.

 

It seems function "tsvectorrecv" can be improved:

v1-0001-Fix-receiving-of-big-tsvector-from-binary - patch fixes problem of
required size calculation for tsvector where wrong type is used (WordEntry
instead of WordEntryPos). Size of wrong type is bigger than type of needed
type therefore total size turns out larger than needed. So next test of
that size fails during maximum size check.

v1-0002-Replace-magic-one-in-tsvector-binary-receiving - patch removes
magic ones from code. Those ones are used to add "sizeof(uint16)" to
required size where number of WordEntryPos'es will be stored. Now it works
only because size of WordEntryPos is equal to size of uint16. But it is
not obviously during code reading and causes question "what for this magic
one is?"

v1-0003-Optimize-memory-allocation-during-tsvector-binary - patch makes
memory allocation more accuracy and rarely. It seems memory to store
tsvector's data is allocated redundant and reallocations happen more often
than necessary.

 

Best regards,

Denis Erokhin

Jatoba

 

Attachments:

  [application/octet-stream] v1-0001-Fix-receiving-of-big-tsvector-from-binary.patch (0B, ../009801d9f2d9$f29730c0$d7c59240$@datagile.ru/3-v1-0001-Fix-receiving-of-big-tsvector-from-binary.patch)
  download