Bug 12616 - After upgrade 4.4.2 to 4.5.5, locking with CTDB fails
After upgrade 4.4.2 to 4.5.5, locking with CTDB fails
Status: NEW
Product: Samba 4.1 and newer
Classification: Unclassified
Component: CTDB
4.5.5
All All
: P5 normal
: ---
Assigned To: Amitay Isaacs
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-03-08 09:51 UTC by acrow
Modified: 2017-03-09 10:05 UTC (History)
3 users (show)

See Also:


Attachments
/var/log/messages (840.95 KB, text/plain)
2017-03-08 09:51 UTC, acrow
no flags Details
/var/log/samba/smbd (12.82 KB, text/plain)
2017-03-08 09:52 UTC, acrow
no flags Details
/var/log/log.ctdb (772.36 KB, text/plain)
2017-03-08 09:52 UTC, acrow
no flags Details
/var/log/samba/<client machine name> (7.88 KB, text/plain)
2017-03-08 09:53 UTC, acrow
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description acrow 2017-03-08 09:51:27 UTC
Created attachment 13028 [details]
/var/log/messages

Hi,

We have a set of fileservers running CTDB on a distributed FS in a Samba 4 AD Domain. These were running OK with 4.4.2, but we decided to upgrade to 4.5.5.

Things looked OK at first, but as more users connected we started to have windows clients throwing up error dialogs saying "file is in use by another user", even for directories and top-level shares. smbd processes were also crashing all over the place.

I've attached log files from /var/log/messages, /var/log/log.ctdb, smbd and a samba log for a windows client.

I'm reticent to perform any upgrade of our fileserver to 4.5 or above because of this issue.
Comment 1 acrow 2017-03-08 09:52:05 UTC
Created attachment 13029 [details]
/var/log/samba/smbd
Comment 2 acrow 2017-03-08 09:52:37 UTC
Created attachment 13030 [details]
/var/log/log.ctdb
Comment 3 acrow 2017-03-08 09:53:54 UTC
Created attachment 13031 [details]
/var/log/samba/<client machine name>
Comment 4 Amitay Isaacs 2017-03-08 22:14:50 UTC
Did you upgrade Samba and CTDB both?  You must run the same version of Samba and CTDB.  It appears from the logs that you are running CTDB 4.4.9.

  2017/02/13 08:58:36.211182 [ 5298]: Starting CTDBD (Version 4.4.9-SerNet-RedHat-37.el7) as PID: 5298
Comment 5 Amitay Isaacs 2017-03-08 22:17:22 UTC
Also, since you are running on Linux, you should enable TDB mutex support for better performance.

In CTDB configuration file, add following:

  CTDB_SET_TDBMutexEnabled=1

In Samba configuration, add following:

  dbwrap_tdb_mutexes:*=true

To do this change,
  - Shutdown CTDB on all the nodes
  - Make the config changes on all the nodes
  - Start CTDB on all the nodes
Comment 6 Amitay Isaacs 2017-03-08 22:20:45 UTC
Looks like you Samba configuration is wrong.

  Feb 13 00:00:06 zearing smbd[26944]: [2017/02/13 00:00:06.390823,  0] ../lib/util/debug.c:951(reopen_logs_internal)
  Feb 13 00:00:06 zearing smbd[26944]:  Unable to open new log file '/var/log/samba/': Is a directory

Check "log file" parameter in your samba configuration (smb.conf).  It should point to a file and NOT to a directory.
Comment 7 acrow 2017-03-09 09:31:11 UTC
(In reply to Amitay Isaacs from comment #6)

My logging is set thusly:

        log file = /var/log/samba/%m

So I'm wondering why %m would be empty in some cases?

This is an unrelated issue, and we still see this in the logs (only since we went Samba 4 AD).
Comment 8 acrow 2017-03-09 09:34:24 UTC
(In reply to Amitay Isaacs from comment #4)

That's really interesting. I had run a:

yum -y update "sernet-samba*"

which should have upgraded both of them, but obviously did not. Maybe sernet missed the ctdb package - I'll look at the repo.

Cheers

Alex
Comment 9 acrow 2017-03-09 09:51:48 UTC
(In reply to Amitay Isaacs from comment #4)

Looks like the sernet repos are OK right now. Something must have gone wrong with the update then.

Cheers

Alex