Bug 6555 - Shares on member server not (allways) accessable on 3.4.0
Summary: Shares on member server not (allways) accessable on 3.4.0
Status: RESOLVED FIXED
Alias: None
Product: Samba 3.4
Classification: Unclassified
Component: Domain Control (show other bugs)
Version: 3.4.1
Hardware: x86 Windows XP
: P3 critical
Target Milestone: ---
Assignee: Guenther Deschner
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-07-16 07:12 UTC by Marc Muehlfeld
Modified: 2010-09-13 14:08 UTC (History)
3 users (show)

See Also:


Attachments
amplicon.log (58.45 KB, text/plain)
2009-07-16 07:13 UTC, Marc Muehlfeld
no flags Details
winbindd.log (5.90 KB, text/plain)
2009-07-16 07:14 UTC, Marc Muehlfeld
no flags Details
log.wb-MUC (9.76 KB, text/plain)
2009-07-16 07:15 UTC, Marc Muehlfeld
no flags Details
nucleus.log (13.16 KB, text/plain)
2009-07-19 08:04 UTC, Marc Muehlfeld
no flags Details
trace.tcpdump (5.59 KB, application/octet-stream)
2009-08-21 08:26 UTC, Marc Muehlfeld
no flags Details
trace2.tcpdump (27.35 KB, application/octet-stream)
2009-08-21 12:35 UTC, Marc Muehlfeld
no flags Details
smb.conf (PDC) (4.89 KB, text/plain)
2009-09-04 03:06 UTC, Marc Muehlfeld
no flags Details
smb.conf (Member Server) (3.16 KB, text/plain)
2009-09-04 03:06 UTC, Marc Muehlfeld
no flags Details
Client level 10 log on the member (66.60 KB, text/plain)
2009-09-24 02:16 UTC, Marc Muehlfeld
no flags Details
Winbind level 10 log on the member (57.69 KB, text/plain)
2009-09-24 02:17 UTC, Marc Muehlfeld
no flags Details
log.wb-MUC (175.50 KB, text/plain)
2009-09-24 03:19 UTC, Marc Muehlfeld
no flags Details
Packet trace on member server (6.11 KB, application/octet-stream)
2009-10-02 01:43 UTC, Marc Muehlfeld
no flags Details
Level 10 debug log on PDC (150.33 KB, text/plain)
2009-10-02 01:46 UTC, Marc Muehlfeld
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Marc Muehlfeld 2009-07-16 07:12:51 UTC
With 3.3.6 on PDC and member server, everything is fine. Also with 3.4.0 on PDC and 3.3.6 on the member server.

But with 3.4.0 on PDC and member server, it's not allways possible to mount a share of the member server. 

Most times if I try to map a share on the member on my XP SP3 clients (don't have others), it fails (using "net use" or the GUI). Sometimes I will be prompted for username/password (what shouldn't.) regardless if I set the new parameter "map untrusted to domain=yes/no". Sometimes I get a "unexpected error" with code 58 or 59 on Windows. And sometimes (in very few cases) the share is mapped without problems, like it allways was with previous version.

It is also mostly possible to access a share, if it was mapped on a client, during I updated samba. But I also had that I could access it for a while and then also got "unexpected error".

This is the share definition on the member:

   [packages]
        comment = Prism Packages
        path = /srv/www/htdocs/packages.mr.lfmg.de
        browsable = no
        force create mode = 0664
        force directory mode = 2775
        guest ok = no
        valid users = +MUC\systemadministration
        invalid users =
        veto files = /.bin/.viminfo/.bash_history/Platform.ini/
Comment 1 Marc Muehlfeld 2009-07-16 07:13:55 UTC
Created attachment 4428 [details]
amplicon.log

XP Workstation Logfile
Comment 2 Marc Muehlfeld 2009-07-16 07:14:20 UTC
Created attachment 4429 [details]
winbindd.log
Comment 3 Marc Muehlfeld 2009-07-16 07:15:14 UTC
Created attachment 4430 [details]
log.wb-MUC

log.wb of the domain
Comment 4 Jeremy Allison 2009-07-16 12:40:29 UTC
Ok, this is what is happening on the member server log.

[2009/07/16 13:51:28, 10] auth/auth_winbind.c:85(check_winbind_security)
  check_winbind_security: wbcAuthenticateUserEx failed: WBC_ERR_AUTH_ERROR
[2009/07/16 13:51:28,  5] auth/auth.c:274(check_ntlm_password)
  check_ntlm_password: winbind authentication for user [muehlfeld] FAILED with error NT_STATUS_INVALID_NETWORK_RESPONSE
[2009/07/16 13:51:28,  2] auth/auth.c:320(check_ntlm_password)
  check_ntlm_password:  Authentication for user [muehlfeld] -> [muehlfeld] FAILED with error NT_STATUS_INVALID_NETWORK_RESPONSE
[2009/07/16 13:51:28,  5] auth/auth_util.c:2114(free_user_info)

I'll investigate the NT_STATUS_INVALID_NETWORK_RESPONSE which I don't understand right now...

Jeremy.
Comment 5 Jeremy Allison 2009-07-16 15:47:46 UTC
Ah, in the winbindd log we have:

[2009/07/16 13:52:03,  5] winbindd/winbindd_async.c:339(lookupname_recv)
  lookupname_recv: unable to determine forest root
[2009/07/16 13:52:03,  2] winbindd/winbindd.c:878(remove_client)
  final write to client failed: Broken pipe

Jeremy.
Comment 6 Jeremy Allison 2009-07-16 15:53:54 UTC
Ah, that's in the parent winbindd, and isn't relevent. Here is the relevent part from the child winbindd.

[2009/07/16 13:51:28, 10] libsmb/async_smb.c:820(validate_smb_crypto)
  Got non-SMB PDU
[2009/07/16 13:51:28, 10] libsmb/async_smb.c:979(handle_incoming_pdu)
  handle_incoming_pdu: Aborting with NT_STATUS_INVALID_NETWORK_RESPONSE

Jeremy.

Can you also post a debug level 10 log from the PDC so we can see what is happening at the other end ? Looks a little like a crypto problem...

Jeremy.
Comment 7 Marc Muehlfeld 2009-07-19 08:04:51 UTC
Created attachment 4446 [details]
nucleus.log

Level 10 logfile of the member server on the PDC (client was asking for username in this case)
Comment 8 Karolin Seeger 2009-08-12 09:34:30 UTC
Jeremy, is that a showstopper for 3.4.1 (scheduled for August 18)?
Comment 9 Volker Lendecke 2009-08-21 07:34:07 UTC
Could this be the same as bug #6649?

Marc, can you try current v3-4-test, there the patch for 6649 is included.

Alternatively, you could try the patch from

http://git.samba.org/?p=samba.git;a=commitdiff;h=ef891070288cd13aff7c730

Thanks,

Volker
Comment 10 Marc Muehlfeld 2009-08-21 08:10:19 UTC
I just want to let you know, if I apply the patch to the member server only, the problem is still there.

Applying it also to the PDC, I can only do at night. I'll do this on the weekend, if the patch was required for the PDC.
Comment 11 Volker Lendecke 2009-08-21 08:15:25 UTC
Ok, thanks. On the DC it won't help.

Looking deeper :-)

Volker
Comment 12 Volker Lendecke 2009-08-21 08:19:12 UTC
If you can reproduce this so quickly, can you upload a network trace of such a failure?

http://wiki.samba.org/index.php/Capture_Packets

Thanks,

Volker
Comment 13 Marc Muehlfeld 2009-08-21 08:26:58 UTC
Created attachment 4583 [details]
trace.tcpdump

The trace file on the member consists:
I tried mapping a share on my member server using "net use". Then I was asked for username/password. I stopped the trace then, because normally (e. g. on 3.3.7) the share is mapped automatically, because I have permission to it.
Comment 14 Volker Lendecke 2009-08-21 08:37:24 UTC
Ok, which part did you patch? You need to patch/restart the winbind authenticating the user, and I would like to see a trace from the start of winbind up to the failure.

Volker
Comment 15 Marc Muehlfeld 2009-08-21 12:35:15 UTC
Created attachment 4584 [details]
trace2.tcpdump

I created a new trace. It contains the start of all samba 3.4.0 services (smbd, nmbd, winbindd) on the memberserver, trying to map a share using "net use" on my windows XP SP3 client until it asks me for username/password (normally this shouldn't be asked. On 3.3.7 I'm authenticated automatically if I'm allowed to access the share).
Comment 16 Guenther Deschner 2009-09-04 02:48:11 UTC
Could not reproduce that, can you please post your pdc and member server smb.conf ?
Comment 17 Marc Muehlfeld 2009-09-04 03:06:23 UTC
Created attachment 4645 [details]
smb.conf (PDC)
Comment 18 Marc Muehlfeld 2009-09-04 03:06:45 UTC
Created attachment 4646 [details]
smb.conf (Member Server)
Comment 19 Guenther Deschner 2009-09-07 07:09:02 UTC
Ok, I can reproduce it (the problem seems to be the "admin users = administrator" in the pdc's smb.conf [global] section.
Comment 20 Marc Muehlfeld 2009-09-07 07:18:57 UTC
Is it a wrong configuration on my side or bug?
Comment 21 Guenther Deschner 2009-09-07 08:00:18 UTC
No, that is a valid configuration.

I am currently testing with the v3-4-test branch (that was planned to become 3.4.1 later this week) which does also exhibit that behavior.

The pattern is really odd:

when I just smbclient against the member server it will mostly work, but quite often return 
NT_STATUS_INVALID_NETWORK_RESPONSE,
NT code 0x1c010002
and NT_STATUS_PIPE_NOT_AVAILABLE

This is another blocker.
Comment 22 Guenther Deschner 2009-09-08 18:37:01 UTC
Ok, this looks like a winbind recursion loop when samba tries to lookup "administrator" (as of "admin users = administrator") on the samba dc.
Comment 23 Guenther Deschner 2009-09-08 18:47:48 UTC
RIght, setting "admin users" to "admin users = DOMAIN\administrator" instead of "admin users = administrator" solved the issue here. Can you please try that ?
Comment 24 Marc Muehlfeld 2009-09-09 03:26:06 UTC
I set
admin users = MUC\Administrator
on my PDC, restarted the services. Also I restarted winbind on the member. But if I try to connect with my account, it still prompts for a password.
Comment 25 Guenther Deschner 2009-09-09 07:09:50 UTC
I am pretty sure this patch
https://bugzilla.samba.org/attachment.cgi?id=4667&action=view
will solve the issue finally.

This patch is part of bug #6646.
Comment 26 Marc Muehlfeld 2009-09-09 07:16:38 UTC
3 of 3 tries of mapping the shares after applying the patch to the member were successfull. Seems to work. I tested it with 3.3.7 on PDC and 3.4.0 on the member.
Comment 27 Guenther Deschner 2009-09-09 07:22:20 UTC
Great,

thanks Marc for all the patience and testing !

This fix will definitely be in the next releases.
Comment 28 Marc Muehlfeld 2009-09-09 07:29:56 UTC
Thanks for fixing. Good work.

This was the last bug that prevented me from switching to 3.4. :)
Comment 29 Guenther Deschner 2009-09-09 07:38:10 UTC
Cool, closing this one as fixed. Please re-open if you still encounter problems with that.
Comment 30 Marc Muehlfeld 2009-09-24 02:02:41 UTC
Was this patch included in 3.4.1? I upgraded from 3.4.0 to 3.4.0 and something similar appears. Maybe it's a new bug, but I'm unsure:

Connecting to the share of the member works fine for several days. But after some days connecting failed and I'm being asked for username/password. The only thing that fixes the issue is to rejoin the domain and restart the services on the member. Then it works for some days again.

Member server is 3.4.1, PDC is 3.3.7.
Comment 31 Karolin Seeger 2009-09-24 02:08:01 UTC
Yes, this patch is included in 3.4.1.
Comment 32 Marc Muehlfeld 2009-09-24 02:09:09 UTC
Didn't saw the bug ID in the change log.

Then it is a new bug?
Comment 33 Karolin Seeger 2009-09-24 02:15:36 UTC
(In reply to comment #32)
> Didn't saw the bug ID in the change log.

Yeah, that's the fix for bug #6646. That one mentioned in the Changelog.

> Then it is a new bug?

I am not sure. Could you provide new logs/sniffs, please?


Comment 34 Marc Muehlfeld 2009-09-24 02:16:58 UTC
Created attachment 4735 [details]
Client level 10 log on the member
Comment 35 Marc Muehlfeld 2009-09-24 02:17:22 UTC
Created attachment 4736 [details]
Winbind level 10 log on the member
Comment 36 Marc Muehlfeld 2009-09-24 02:20:19 UTC
Sorry. I didn't made a packet trace this morning, just a level 10 debug log, before I rejoined and restarted the services, to make our productive server work again.

Until it will appear again, it will take some time again, I guess.
Comment 37 Guenther Deschner 2009-09-24 03:06:35 UTC
(In reply to comment #36)
> Sorry. I didn't made a packet trace this morning, just a level 10 debug log,
> before I rejoined and restarted the services, to make our productive server
> work again.
> 
> Until it will appear again, it will take some time again, I guess.

Do you have the log.wb-PRIMARYDOMAIN logfile ? The one you uploaded seems to be log.winbindd.
Comment 38 Marc Muehlfeld 2009-09-24 03:19:41 UTC
Created attachment 4737 [details]
log.wb-MUC
Comment 39 Guenther Deschner 2009-09-24 09:18:14 UTC
(In reply to comment #38)
> Created an attachment (id=4737) [details]
> log.wb-MUC

weird, the PDC seems to refuse any netr_ServerAuthenticate2 attempts.
sorry, can you also check and upload your pdc log.smbd (that is also 3.4.1, right ?)

Comment 40 Marc Muehlfeld 2009-09-25 02:39:16 UTC
The PDC is 3.3.7, because of an other disturbing bug we have thith 3.4.x.

I didn't made a level 10 debug log on the PDC during the problem. For that I have to wait until it appears again in some days.
Comment 41 Marc Muehlfeld 2009-10-02 01:43:46 UTC
Created attachment 4773 [details]
Packet trace on member server
Comment 42 Marc Muehlfeld 2009-10-02 01:46:34 UTC
Created attachment 4774 [details]
Level 10 debug log on PDC

After waiting some days the problem came up again. Now I did a packet trace on the member server and a level 10 debug log on the PDC also. Both contains the try to map a share with "net use", what worked since the last domain join of the member and restart of the services.

Member: 3.4.1
PDC: 3.3.7
Comment 43 Guenther Deschner 2009-10-02 08:29:42 UTC
(In reply to comment #40)
> The PDC is 3.3.7, because of an other disturbing bug we have thith 3.4.x.

just curious, what bug is holding you off moving to 3.4 on the PDC ?
Comment 44 Marc Muehlfeld 2009-10-02 08:32:56 UTC
Bug 6727 is my blocker for switching to 3.4
Comment 45 Karolin Seeger 2009-10-14 04:43:37 UTC
Is this one a show stopper for 3.4.3?
Comment 46 Jeremy Allison 2009-10-19 16:15:35 UTC
Again, reducing problem to "critical" from blocker. This is a specific problem with a specific setup, not a generic bug as far as I can tell. Guenther please confirm.
Jeremy.
Comment 47 Marc Muehlfeld 2010-02-17 14:47:41 UTC
> This is a specific problem with a specific setup,
> not a generic bug as far as I can tell.

Is it a configuration problem or is it a bug? Is there anything I can do to help fixing this issue? Here it is a showstopper for switching to 3.4 or later.

Please let me know if I can do anything to help you fixing this issue.
Comment 48 Marc Muehlfeld 2010-03-23 04:23:14 UTC
Im running 3.5.0 since 21 days on one of our member servers. With this version, the problem seems to be gone.
Comment 49 Marc Muehlfeld 2010-09-13 14:08:55 UTC
Fixed in 3.5.0 and later