Bug 13657 - tdb lock error when adding users to groups in large amount
Summary: tdb lock error when adding users to groups in large amount
Status: NEW
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: AD: LDB/DSDB/SAMDB (show other bugs)
Version: 4.9.0rc1
Hardware: x64 Linux
: P5 normal (vote)
Target Milestone: ---
Assignee: Andrew Bartlett
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-18 03:48 UTC by Joe Guo (mail address dead)
Modified: 2018-10-18 03:48 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Joe Guo (mail address dead) 2018-10-18 03:48:08 UTC
I am trying to generate 50k users, 5k groups, and add each user to about 10 groups which produce 500k memberships with `script/traffic_replay`. 

store backend is tdb. Server is a openstack instance with 4 CPUs and 4G mem.

The creation of users and groups are working fine, but adding users to groups will fail with lock error. 

The error is stable, I reproduced the error when I tried it again for a few times.

mdb is working fine.

Error message:

INFO 2018-10-17 20:42:36,428 pid:11227 bin/python/samba/emulate/traffic.py #1718: 1539762156.428459 0 create  group 0.037660  True  
INFO 2018-10-17 20:42:36,466 pid:11227 bin/python/samba/emulate/traffic.py #1718: 1539762156.466589 0 create  group 0.037734  True  
Generating groups took 344.71756196 secs
Assigning users to groups
Assigning users took 5.06887698174 secs
Adding users to groups
ltdb: tdb(/usr/local/samba/private/sam.ldb.d/DC=SAMDOM,DC=EXAMPLE,DC=COM.ldb): tdb_transaction_prepare_commit: failed to upgrade hash locks: Locking error

ltdb: tdb(/usr/local/samba/private/sam.ldb.d/DC=SAMDOM,DC=EXAMPLE,DC=COM.ldb): ../lib/tdb/common/tdb.c:1098 Failed to commit

Traceback (most recent call last):
  File "./script/traffic_replay", line 399, in <module>
    main()
  File "./script/traffic_replay", line 332, in main
    opts.group_memberships)
  File "bin/python/samba/emulate/traffic.py", line 1813, in generate_users_and_groups
    add_users_to_groups(ldb, instance_id, assignments)
  File "bin/python/samba/emulate/traffic.py", line 1959, in add_users_to_groups
    db.modify(m)
_ldb.LdbError: (51, 'end_trans error on DC=samdom,DC=example,DC=com: Failure during tdb_transaction_commit(): Locking error -> Busy')