Bug 16152 - CTDB can run nested elections, in rare circumstances
Summary: CTDB can run nested elections, in rare circumstances
Status: ASSIGNED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: CTDB (show other bugs)
Version: 4.24.2
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Martin Schwenke
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2026-07-01 04:37 UTC by Martin Schwenke
Modified: 2026-07-10 00:36 UTC (History)
1 user (show)

See Also:

Attachments
Patch for v4-24-test, v4-23-test (6.04 KB, patch)
2026-07-10 00:36 UTC, Martin Schwenke 		martins: review? (amitay)
 Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Schwenke 2026-07-01 04:37:19 UTC 
Here's an example:

      2026-05-12T12:15:51.127992+05:30 nsds1 ctdb-recoverd[826]: Leader broadcast timeout
      2026-05-12T12:15:51.128019+05:30 nsds1 ctdb-recoverd[826]: Start election
      2026-05-12T12:15:51.128192+05:30 nsds1 ctdb-recoverd[826]: Attempting to take cluster lock (!/usr/libexec/ctdb/ctdb_mutex_fcntl_helper /mnt/lustre/mac167/nsds_conf/.nsds_config/cluster_lock 5 20)
      2026-05-12T12:15:51.135578+05:30 nsds1 ctdbd[700]: ctdb_mutex_fcntl_helper: Unable to open /mnt/lustre/mac167/nsds_conf/.nsds_config/cluster_lock - (No such file or directory)
      2026-05-12T12:15:51.135590+05:30 nsds1 ctdb-recoverd[826]: Unable to take cluster lock - unknown error
      2026-05-12T12:15:56.130174+05:30 nsds1 ctdb-recoverd[826]: Leader broadcast timeout
      2026-05-12T12:15:56.130206+05:30 nsds1 ctdb-recoverd[826]: Start election
      2026-05-12T12:15:56.130329+05:30 nsds1 ctdb-recoverd[826]: Attempting to take cluster lock (!/usr/libexec/ctdb/ctdb_mutex_fcntl_helper /mnt/lustre/mac167/nsds_conf/.nsds_config/cluster_lock 5 20)
      2026-05-12T12:16:01.131882+05:30 nsds1 ctdb-recoverd[826]: Leader broadcast timeout
      2026-05-12T12:16:01.131918+05:30 nsds1 ctdb-recoverd[826]: Start election
      2026-05-12T12:16:13.094308+05:30 nsds1 ctdbd[700]: ctdb_mutex_fcntl_helper: Unable to open /mnt/lustre/mac167/nsds_conf/.nsds_config/cluster_lock - (No such file or directory)
      2026-05-12T12:16:13.094362+05:30 nsds1 ctdb-recoverd[826]: Unable to take cluster lock - unknown error
      2026-05-12T12:16:13.094582+05:30 nsds1 ctdb-recoverd[826]: do_recovery: Starting do_recovery
      2026-05-12T12:16:13.094591+05:30 nsds1 ctdb-recoverd[826]: Cluster lock not held - abort recovery, ban node
      2026-05-12T12:16:13.094596+05:30 nsds1 ctdb-recoverd[826]: Banning node 0 for 300 seconds
      2026-05-12T12:16:13.094612+05:30 nsds1 ctdbd[700]: Banning this node for 300 seconds

The 2nd election doesn't end before the 3rd one starts.  The 3rd one is triggered by a leader broadcast timeout.

This can result in unexpected behaviour and a memory leak.

It can be fixed by making one check stronger and adding a new check.
Comment 1 Samba QA Contact 2026-07-09 15:25:04 UTC 
This bug was referenced in samba master:

c3c250c9c8e3385f593e386f91e8579b8c89370a
d28f8ed17c867f306ca900bd912e8b75c5c7be8b
Comment 3 Martin Schwenke 2026-07-10 00:36:17 UTC 
Created attachment 19080 [details]
Patch for v4-24-test, v4-23-test

Cherry-picks cleanly into v4-24-test.  Resulting patch applies cleanly to
v4-23-test using git am.

Both branches build and pass a smoke test (local daemons start and
become healthy).