Bug 10258 - Unexpected large database file
Summary: Unexpected large database file
Alias: None
Product: TDB
Classification: Unclassified
Component: libtdb (show other bugs)
Version: unspecified
Hardware: All All
: P5 normal
Target Milestone: ---
Assignee: Samba QA Contact
QA Contact: Samba QA Contact
Depends on:
Reported: 2013-11-09 01:51 UTC by Michael
Modified: 2013-11-09 23:13 UTC (History)
1 user (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Michael 2013-11-09 01:51:50 UTC
I am trying to use tdb as a key/value database (unrelated to samba). I was surprised to observe that setting values of the order of MB expands the database file to several GB. For example using the python bindings:

In [89]: fd = tdb.open("/tmp/test.tdb", tdb_flags=tdb.NOMMAP,flags=os.O_CREAT|os.O_RDWR)

In [90]: fd.store("hello", 10 * 1024 * 1024 * "A")

In [91]: fd.close()

In [93]: ls -lh /tmp/test.tdb
-rw------- ---- --- 1.3G Nov  9 02:38 /tmp/test.tdb

$ tdbtool /tmp/test.tdb 
tdb> info
Size of file/data: 1310728192/10485767
Number of records: 2
Smallest/average/largest keys: 1/3/5
Smallest/average/largest data: 1/5242880/10485760
Smallest/average/largest padding: 6/1310726/2621447
Number of dead records: 0
Smallest/average/largest dead records: 0/0/0
Number of free records: 2
Smallest/average/largest free records: 3344/648810090/1297616836
Number of hash chains: 131
Smallest/average/largest hash chains: 0/0/1
Number of uncoalesced records: 0
Smallest/average/largest uncoalesced runs: 0/0/0
Percentage keys/data/padding/free/dead/rechdrs&tailers/hashes: 0/1/0/99/0/0/0

Is this behaviour expected?

Comment 1 Volker Lendecke 2013-11-09 15:20:24 UTC
Which version is this? There have been significant fixes in that area in the last 12-18 months
Comment 2 Michael 2013-11-09 23:13:52 UTC
Ah I am sorry - I should have tried the latest version. This problem is solved with tdb-1.2.12.tar.gz released 04-Jun-2013. Repeating the above gives a tdb file of roughly twice the size of the data (about 25mb).