Bug 7099 - Every Thursday at 11:08-11:15am Windows Client Connections break with Kerberos errors
Summary: Every Thursday at 11:08-11:15am Windows Client Connections break with Kerbero...
Status: RESOLVED FIXED
Alias: None
Product: Samba 3.5
Classification: Unclassified
Component: Winbind (show other bugs)
Version: 3.5.3
Hardware: Other Linux
: P3 normal
Target Milestone: ---
Assignee: Karolin Seeger
QA Contact: Samba QA Contact
URL:
Keywords:
: 7178 7524 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-02-04 15:01 UTC by John H Terpstra (mail address dead(
Modified: 2010-07-13 06:55 UTC (History)
4 users (show)

See Also:


Attachments
Windows Server 2003 R2 IIS Eventviewer error log (793 bytes, text/plain)
2010-02-04 15:03 UTC, John H Terpstra (mail address dead(
no flags Details
Patch for fixing this bug (10.09 KB, patch)
2010-05-21 08:14 UTC, Matthieu Patou
no flags Details
Second fix version (10.14 KB, patch)
2010-05-24 17:04 UTC, Matthieu Patou
no flags Details
Patch for fixing the bug (10.26 KB, patch)
2010-05-31 13:02 UTC, Matthieu Patou
no flags Details
v3-5-test patch (port from master) (11.01 KB, patch)
2010-06-02 08:57 UTC, Guenther Deschner
idra: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description John H Terpstra (mail address dead( 2010-02-04 15:01:04 UTC
A Windows Server 2003 R2 IIS server that is accessing Samba CTDB is generating the following error.  It happens at the same time every week (approx 11:08-11:15am every Thursday) it only happens within this specific time window.

Here is a loglevel 4 snippet.

[2010/02/04 11:13:55,  3] smbd/sesssetup.c:reply_sesssetup_and_X_spnego(1175)
  Doing spnego session setup
[2010/02/04 11:13:55,  3] smbd/sesssetup.c:reply_sesssetup_and_X_spnego(1210)
  NativeOS=[Windows Server 2003 R2 3790 Service Pack 2] NativeLanMan=[]
PrimaryDomain=[Windows Server 2003 R2 5.2]
[2010/02/04 11:13:55,  3] smbd/sesssetup.c:reply_spnego_negotiate(802)
  reply_spnego_negotiate: Got secblob of size 1204
[2010/02/04 11:13:55,  3] libads/kerberos_verify.c:ads_secrets_verify_ticket(296)
  ads_secrets_verify_ticket: enc type [23] failed to decrypt with error
Decrypt integrity check failed
[2010/02/04 11:13:55,  3] libads/kerberos_verify.c:ads_verify_ticket(471)
  ads_verify_ticket: krb5_rd_req with auth failed (Bad encryption type)
[2010/02/04 11:13:55,  1] smbd/sesssetup.c:reply_spnego_kerberos(350)
  Failed to verify incoming ticket with error NT_STATUS_LOGON_FAILURE!
[2010/02/04 11:13:55,  3] smbd/error.c:error_packet_set(61)
  error packet at smbd/sesssetup.c(352) cmd=115 (SMBsesssetupX)
NT_STATUS_LOGON_FAILURE
[2010/02/04 11:13:55,  3] smbd/process.c:smbd_process(1930)
  receive_message_or_smb failed: NT_STATUS_END_OF_FILE, exiting
[2010/02/04 11:13:55,  3] smbd/sec_ctx.c:set_sec_ctx(324)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 0
[2010/02/04 11:13:55,  3] smbd/connection.c:yield_connection(31)
  Yielding connection to
[2010/02/04 11:13:55,  3] smbd/server.c:exit_server_common(971)
  Server exit (normal exit)


The suspect is that the ADS Domain Member trust accounts for the Samba pCIFS cluster is leading to breakage.   Is this trust password change cluster-aware?


Windows clients are reporting errors attached in separate files.
Comment 1 John H Terpstra (mail address dead( 2010-02-04 15:03:54 UTC
Created attachment 5273 [details]
Windows Server 2003 R2 IIS Eventviewer error log
Comment 2 John H Terpstra (mail address dead( 2010-04-12 22:48:45 UTC
Machine password changes have been disabled.  The symptom has resurfaced for 2 months.  This is a bug, but we have a work-around.
Comment 3 John H Terpstra (mail address dead( 2010-04-28 12:17:43 UTC
The work-around is to disable machine password changes.  Not ideal, but it solves the immediate problem.
Comment 4 Stefan Metzmacher 2010-05-12 03:52:02 UTC
*** Bug 7178 has been marked as a duplicate of this bug. ***
Comment 5 Michael Adam 2010-05-18 05:08:32 UTC
This is not cluster-related.
Happened in standalone samba too (bug #7178 - metze said it's a duplicate).
Comment 6 Matthieu Patou 2010-05-21 08:14:08 UTC
Created attachment 5728 [details]
Patch for fixing this bug
Comment 7 Matthieu Patou 2010-05-24 17:04:54 UTC
Created attachment 5735 [details]
Second fix version

This patch has been tested against windows XP. The previous one was failing in case of mutual authentication.
Comment 8 Simo Sorce 2010-05-31 11:29:28 UTC
In order to solve this issue, we need to keep around the old trust password for as long as tickets are valid in the domain.
We also need to make sure we have both old and new password used to accept kerberos credentials from clients.
If we used a keytab all we need to do would be to keep old and new keys in the keytab.
Comment 9 Simo Sorce 2010-05-31 11:35:13 UTC
(In reply to comment #7)
> Created an attachment (id=5735) [details]
> Second fix version
> 
> This patch has been tested against windows XP. The previous one was failing in
> case of mutual authentication.


Patch looks good to me.
Simo.
Comment 10 Matthieu Patou 2010-05-31 13:02:21 UTC
Created attachment 5752 [details]
Patch for fixing the bug

Fix typos in previous patch
Comment 11 Guenther Deschner 2010-06-02 08:57:18 UTC
Created attachment 5758 [details]
v3-5-test patch (port from master)
Comment 12 Guenther Deschner 2010-06-02 09:33:41 UTC
Karolin, please pick for 3.5
Comment 13 Karolin Seeger 2010-06-03 08:34:43 UTC
Pushed to v3-5-test.
Closing out bug report.

Thanks!
Comment 14 Rob V. 2010-07-13 06:55:53 UTC
*** Bug 7524 has been marked as a duplicate of this bug. ***