Bug 11275 - Get NT_STATUS_CONNECTION_RESET when use windows server 2008 as client
Summary: Get NT_STATUS_CONNECTION_RESET when use windows server 2008 as client
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services (show other bugs)
Version: 4.1.12
Hardware: x64 Linux
: P5 critical (vote)
Target Milestone: ---
Assignee: Samba QA Contact
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-05-14 15:32 UTC by crashskyshadow
Modified: 2020-12-19 16:28 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description crashskyshadow 2015-05-14 15:32:54 UTC
when copy a lot of files to samba 4.12 server. we get the flow error after write data to disk and then finish the request.


[2015/05/14 10:12:27.100019, 10, pid=2027, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:2499(smbd_smb2_request_done_ex)
  smbd_smb2_request_done_ex: idx[1] status[NT_STATUS_OK] body[16] dyn[yes:0] at ../source3/smbd/smb2_write.c:150
[2015/05/14 10:12:27.100078, 10, pid=2027, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:874(smb2_set_operation_credit)
  smb2_set_operation_credit: requested 1, charge 1, granted 1, current possible/max 258/512, total granted/max/low/range 255/8192/813870/255
[2015/05/14 10:12:27.100150, 10, pid=2027, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:1002(smbd_server_connection_terminate_ex)
  smbd_server_connection_terminate_ex: reason[NT_STATUS_CONNECTION_RESET] at ../source3/smbd/smb2_write.c:152


is It a network error or a bug? it can reproduce on that server.
Comment 1 Ira Cooper 2015-05-14 15:52:45 UTC
My initial thoughts:

A connection reset like what you are seeing can be caused by a SMB2 client detecting what it believes is a protocol violation.  At which point it will disconnect.

I'd suggest two things to you, right off the top of my head.

1. Get us a wireshark trace of this happening, if you can.  We will need a large snapsize to actually find the issue, as a warning.  So please don't use private files. :)
2. Can you please update to 4.1.18 AFTER reproducing the issue, and see if it goes away.  This will make tracking down the bug much easier, and may fix your issue.
Comment 2 crashskyshadow 2015-05-15 00:29:48 UTC
(In reply to Ira Cooper from comment #1)
I need copy about 1.3T image data from a IBM NAS server to my Samba server. we direct connect two network cards. This issue happens when we finish write 40Gb-90Gb files. if we use net work switches  it happened at 1.1T data.
Comment 3 crashskyshadow 2015-05-15 00:33:59 UTC
[global]
log level = 10
load printers = No
max log size = 130000
dos charset = gb18030 
unix charset = gb18030
clustering = yes
idmap backend = tdb2
private dir=/clusterdata/ctdb
fileid:mapping = fsname
use mmap = yes
nt acl support = yes
ea support = yes
#oplocks=no
socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 IPTOS_LOWDELAY TCP_NODELAY
min receivefile size=262144
use sendfile=yes
aio read size=131072
aio write size=131072
block size=4096
Comment 4 crashskyshadow 2015-05-15 08:07:43 UTC
reboot the system and error happened on 600Gb.


Ip:
    44740518 total packets received
    0 forwarded
    0 incoming packets discarded
    44740498 incoming packets delivered
    20083521 requests sent out
    384 outgoing packets dropped
Icmp:
    14620 ICMP messages received
    56 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 1130
        echo requests: 13490
    14601 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 1111
        echo replies: 13490
IcmpMsg:
        InType3: 1130
        InType8: 13490
        OutType0: 13490
        OutType3: 1111
Tcp:
    181 active connections openings
    15 passive connection openings
    87 failed connection attempts
    5 connection resets received
    4 connections established
    44471180 segments received
    20990769 segments send out
    41 segments retransmited
    12 bad segments received.
    87 resets sent
Udp:
    253925 packets received
    796 packets to unknown port received.
    0 packet receive errors
    201048 packets sent
    0 receive buffer errors
    0 send buffer errors
UdpLite:
TcpExt:
    5 invalid SYN cookies received
    1 TCP sockets finished time wait in fast timer
    5 packets rejects in established connections because of timestamp
    9098 delayed acks sent
    46 delayed acks further delayed because of locked socket
    Quick ack mode was activated 332186 times
    39340366 packet headers predicted
    1041031 acknowledgments not containing data payload received
    6602239 predicted acknowledgments
    1 times recovered from packet loss by selective acknowledgements
    Detected reordering 1 times using SACK
    Detected reordering 1 times using time stamp
    1 congestion windows fully recovered without slow start
    9 congestion windows partially recovered using Hoe heuristic
    20 fast retransmits
    1 forward retransmits
    11 other TCP timeouts
    TCPLossProbes: 90
    TCPLossProbeRecovery: 89
    332189 DSACKs sent for old packets
    685643 DSACKs received
    TCPDSACKIgnoredNoUndo: 87928
    TCPSackMerged: 11
    TCPSackShiftFallback: 101
    TCPRcvCoalesce: 13897797
    TCPChallengeACK: 12
    TCPSYNChallenge: 12
    TCPSpuriousRtxHostQueues: 3
IpExt:
    InNoRoutes: 20
    InMcastPkts: 137
    OutMcastPkts: 117
    InBcastPkts: 27
    InOctets: 587819843568
    OutOctets: 3423550891
    InMcastOctets: 21493
    OutMcastOctets: 16993
    InBcastOctets: 6381
Sctp:
    0 Current Associations
    0 Active Associations
    0 Passive Associations
    0 Number of Aborteds 
    0 Number of Graceful Terminations
    0 Number of Out of Blue packets
    0 Number of Packets with invalid Checksum
    0 Number of control chunks sent
    0 Number of ordered chunks sent
    0 Number of Unordered chunks sent
    0 Number of control chunks received
    0 Number of ordered chunks received
    0 Number of Unordered chunks received
    0 Number of messages fragmented
    0 Number of messages reassembled 
    0 Number of SCTP packets sent
    0 Number of SCTP packets received
Comment 5 Björn Jacke 2015-05-15 11:58:32 UTC
which network adapters are involved (brand and type)?
Comment 6 crashskyshadow 2015-05-16 07:24:58 UTC
(In reply to Björn Jacke from comment #5)
The Nas side network type is below:
Ethernet controller: Intel Corporation 82574L Gigabit Network Connection.

other side not sure which type. It is a old Netapp Nas machine.
Comment 7 Björn Jacke 2015-05-18 12:15:57 UTC
This is what I was expecting. This is a dup of bug 9030.

Don't use Intel NICs. They are broken and Intel doesn't even bother to release drivers with workarounds for the broken firmware.

*** This bug has been marked as a duplicate of bug 9030 ***
Comment 8 Ira Cooper 2015-05-18 12:34:59 UTC
(In reply to Björn Jacke from comment #7)

http://lxr.linux.no/#linux+v3.19.1/drivers/net/ethernet/intel/e1000e/hw.h#L46

I disagree with closing this.  The e1000 is a well documented chipset supported on just about every platform.  Support on linux is via the e1000e driver.

http://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/82574-gbe-controller-spec-update.pdf

Thanks,

-Ira
Comment 9 Björn Jacke 2015-05-18 12:55:04 UTC
Ira: but you realize that this is not the right place to discuss that? Feel free to reopen https://bugzilla.kernel.org/show_bug.cgi?id=15384 . Good luck with the ignorance of the Intel developers for existing bugs of their firmware. This is not a Samba problem. This is a Intel/Linux kernel driver issue. I'm curious how much success you will have discussing with the e1000 guys :-)
Comment 10 Ira Cooper 2015-05-18 13:43:56 UTC
Well, if you are closing due to it... it comes here :/

My fedora machine would be fine.  It has tx offload disabled for ipv4 and ipv6.  My guess is modern distros have done the right thing.

crashskyshadow@163.com can you please provide the output of ethtool -k (your nic) please?

Thanks,
Comment 11 crashskyshadow 2015-05-19 00:33:57 UTC
(In reply to Ira Cooper from comment #10)
I test it no problem on the other Intel Corporation 82576 Gigabit Network Connection at my laboratory. Today I will do the test use 82574L direct connection at laboratory. Hope to reproduce.

Well below is 82574L nic output. 
ethtool -k enp7s0
Features for enp7s0:
rx-checksumming: on
tx-checksumming: on
	tx-checksum-ipv4: off [fixed]
	tx-checksum-ip-generic: on
	tx-checksum-ipv6: off [fixed]
	tx-checksum-fcoe-crc: off [fixed]
	tx-checksum-sctp: off [fixed]
scatter-gather: on
	tx-scatter-gather: on
	tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
	tx-tcp-segmentation: on
	tx-tcp-ecn-segmentation: off [fixed]
	tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: on
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
busy-poll: off [fixed]
Comment 12 Ira Cooper 2015-05-19 07:19:27 UTC
Does this happen with many different files... or is there a special file causing the problem... etc..

Thanks,
Comment 13 crashskyshadow 2015-05-19 08:56:34 UTC
two side are 82574L nic now. issue reproduced even I change the nic parameters.
ethtool -K enp6s0 tso off
ethtool -K enp6s0 tx off


samba log is below:
 push_conn_ctx(0) : conn_ctx_stack_ndx = 0
[2015/05/14 16:20:59.981394,  4, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:316(set_sec_ctx)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 1
[2015/05/14 16:20:59.981450,  5, pid=26535, effective(0, 0), real(0, 0)] ../libcli/security/security_token.c:53(security_token_debug)
  Security token: (NULL)
[2015/05/14 16:20:59.981491,  5, pid=26535, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:629(debug_unix_user_token)
  UNIX token of user 0
  Primary group is 0 and contains 0 supplementary groups
[2015/05/14 16:20:59.981560,  4, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:424(pop_sec_ctx)
  pop_sec_ctx (0, 0) - sec_ctx_stack_ndx = 0
[2015/05/14 16:20:59.981605,  5, pid=26535, effective(0, 0), real(0, 0), class=auth] ../source3/auth/auth.c:277(auth_check_ntlm_password)
  check_ntlm_password:  PAM Account for user [root] succeeded
[2015/05/14 16:20:59.981645,  2, pid=26535, effective(0, 0), real(0, 0), class=auth] ../source3/auth/auth.c:290(auth_check_ntlm_password)
  check_ntlm_password:  authentication for user [root] -> [root] -> [root] succeeded
[2015/05/14 16:20:59.981687, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/auth/auth_ntlmssp.c:171(auth3_check_password)
  Got NT session key of length 16
[2015/05/14 16:20:59.981728, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/auth/auth_ntlmssp.c:178(auth3_check_password)
  Got LM session key of length 8
[2015/05/14 16:20:59.981770, 10, pid=26535, effective(0, 0), real(0, 0)] ../auth/ntlmssp/ntlmssp_server.c:539(ntlmssp_server_postauth)
  ntlmssp_server_auth: Using unmodified nt session key.
[2015/05/14 16:20:59.981832,  3, pid=26535, effective(0, 0), real(0, 0)] ../auth/ntlmssp/ntlmssp_sign.c:547(ntlmssp_sign_init)
  NTLMSSP Sign/Seal - Initialising with flags:
[2015/05/14 16:20:59.981875,  3, pid=26535, effective(0, 0), real(0, 0)] ../auth/ntlmssp/ntlmssp_util.c:34(debug_ntlmssp_flags)
  Got NTLMSSP neg_flags=0xe2088215
    NTLMSSP_NEGOTIATE_UNICODE
    NTLMSSP_REQUEST_TARGET
    NTLMSSP_NEGOTIATE_SIGN
    NTLMSSP_NEGOTIATE_NTLM
    NTLMSSP_NEGOTIATE_ALWAYS_SIGN
    NTLMSSP_NEGOTIATE_NTLM2
    NTLMSSP_NEGOTIATE_VERSION
    NTLMSSP_NEGOTIATE_128
    NTLMSSP_NEGOTIATE_KEY_EXCH
    NTLMSSP_NEGOTIATE_56
[2015/05/14 16:20:59.982067,  4, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:216(push_sec_ctx)
  push_sec_ctx(0, 0) : sec_ctx_stack_ndx = 1
[2015/05/14 16:20:59.982115,  4, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/uid.c:485(push_conn_ctx)
  push_conn_ctx(0) : conn_ctx_stack_ndx = 0
[2015/05/14 16:20:59.982156,  4, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:316(set_sec_ctx)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 1
[2015/05/14 16:20:59.982232,  5, pid=26535, effective(0, 0), real(0, 0)] ../libcli/security/security_token.c:53(security_token_debug)
  Security token: (NULL)
[2015/05/14 16:20:59.982274,  5, pid=26535, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:629(debug_unix_user_token)
  UNIX token of user 0
  Primary group is 0 and contains 0 supplementary groups
[2015/05/14 16:20:59.982340,  4, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:424(pop_sec_ctx)
  pop_sec_ctx (0, 0) - sec_ctx_stack_ndx = 0
[2015/05/14 16:20:59.982401, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:224(create_local_nt_token_from_info3)
  Create local NT token for root
[2015/05/14 16:20:59.982509, 10, pid=26535, effective(0, 0), real(0, 0), class=tdb] ../source3/lib/gencache.c:296(gencache_set_data_blob)
  Adding cache entry with key=[IDMAP/SID2XID/S-1-5-21-545351224-1600352317-1800318577-1000] and timeout=[四 1月  1 08时00分00秒 1970 CST] (-1431591659 seconds in the past)
[2015/05/14 16:20:59.996902, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/passdb/lookup_sid.c:1388(sid_to_uid)
  sid S-1-5-21-545351224-1600352317-1800318577-1000 -> uid 4294967295
[2015/05/14 16:20:59.997075,  1, pid=26535, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:430(add_local_groups)
  SID S-1-5-21-545351224-1600352317-1800318577-1000 -> getpwuid(4294967295) failed
[2015/05/14 16:20:59.997133,  3, pid=26535, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:316(create_local_nt_token_from_info3)
  Failed to finalize nt token
[2015/05/14 16:20:59.997175, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/auth/auth_ntlmssp.c:45(auth3_generate_session_info)
  create_local_token failed: NT_STATUS_UNSUCCESSFUL
[2015/05/14 16:20:59.997229,  5, pid=26535, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:187(dbwrap_check_lock_order)
  check lock order 1 for smbXsrv_session_global.tdb
[2015/05/14 16:20:59.997272, 10, pid=26535, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:133(debug_lock_order)
  lock order:  1:smbXsrv_session_global.tdb 2:<none> 3:<none>
[2015/05/14 16:20:59.997319, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/lib/dbwrap/dbwrap_ctdb.c:1057(fetch_locked_internal)
  Locking db 1795596909 key B4A80E1B
[2015/05/14 16:20:59.997377, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/lib/ctdbd_conn.c:868(ctdbd_control)
  ctdbd_control: Sending ctdb packet
[2015/05/14 16:20:59.997655, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/lib/dbwrap/dbwrap_ctdb.c:953(db_ctdb_record_destr)
  Unlocking db 1795596909 key B4A80E1B
[2015/05/14 16:20:59.997771,  5, pid=26535, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:146(dbwrap_lock_order_state_destructor)
  release lock order 1 for smbXsrv_session_global.tdb
[2015/05/14 16:20:59.997820, 10, pid=26535, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:133(debug_lock_order)
  lock order:  1:<none> 2:<none> 3:<none>
[2015/05/14 16:20:59.997900, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:2598(smbd_smb2_request_error_ex)
  smbd_smb2_request_error_ex: idx[1] status[NT_STATUS_UNSUCCESSFUL] || at ../source3/smbd/smb2_sesssetup.c:131
[2015/05/14 16:20:59.997968, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:2499(smbd_smb2_request_done_ex)
  smbd_smb2_request_done_ex: idx[1] status[NT_STATUS_UNSUCCESSFUL] body[8] dyn[yes:1] at ../source3/smbd/smb2_server.c:2651
[2015/05/14 16:20:59.998016, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:874(smb2_set_operation_credit)
  smb2_set_operation_credit: requested 31, charge 1, granted 1, current possible/max 512/512, total granted/max/low/range 1/8192/3/1
[2015/05/14 16:20:59.998203, 10, pid=26535, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:1002(smbd_server_connection_terminate_ex)
  smbd_server_connection_terminate_ex: reason[NT_STATUS_CONNECTION_RESET] at ../source3/smbd/smb2_server.c:3309
Comment 14 crashskyshadow 2015-05-19 09:00:56 UTC
sorry below is the new samba log.
[2015/05/19 13:50:49.762827,  5, pid=28254, effective(0, 0), real(0, 0), class=auth] ../source3/auth/server_info_sam.c:125(make_server_info_sam)
  make_server_info_sam: made server info for user root -> root
[2015/05/19 13:50:49.762882,  4, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:424(pop_sec_ctx)
  pop_sec_ctx (0, 0) - sec_ctx_stack_ndx = 0
[2015/05/19 13:50:49.762957,  3, pid=28254, effective(0, 0), real(0, 0), class=auth] ../source3/auth/auth.c:237(auth_check_ntlm_password)
  check_ntlm_password: sam authentication for user [root] succeeded
[2015/05/19 13:50:49.763014,  4, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:216(push_sec_ctx)
  push_sec_ctx(0, 0) : sec_ctx_stack_ndx = 1
[2015/05/19 13:50:49.763060,  4, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/uid.c:485(push_conn_ctx)
  push_conn_ctx(0) : conn_ctx_stack_ndx = 0
[2015/05/19 13:50:49.763104,  4, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:316(set_sec_ctx)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 1
[2015/05/19 13:50:49.763158,  5, pid=28254, effective(0, 0), real(0, 0)] ../libcli/security/security_token.c:53(security_token_debug)
  Security token: (NULL)
[2015/05/19 13:50:49.763203,  5, pid=28254, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:629(debug_unix_user_token)
  UNIX token of user 0
  Primary group is 0 and contains 0 supplementary groups
[2015/05/19 13:50:49.763275,  4, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:424(pop_sec_ctx)
  pop_sec_ctx (0, 0) - sec_ctx_stack_ndx = 0
[2015/05/19 13:50:49.763324,  5, pid=28254, effective(0, 0), real(0, 0), class=auth] ../source3/auth/auth.c:277(auth_check_ntlm_password)
  check_ntlm_password:  PAM Account for user [root] succeeded
[2015/05/19 13:50:49.763369,  2, pid=28254, effective(0, 0), real(0, 0), class=auth] ../source3/auth/auth.c:290(auth_check_ntlm_password)
  check_ntlm_password:  authentication for user [root] -> [root] -> [root] succeeded
[2015/05/19 13:50:49.763417, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/auth/auth_ntlmssp.c:171(auth3_check_password)
  Got NT session key of length 16
[2015/05/19 13:50:49.763462, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/auth/auth_ntlmssp.c:178(auth3_check_password)
  Got LM session key of length 8
[2015/05/19 13:50:49.763510, 10, pid=28254, effective(0, 0), real(0, 0)] ../auth/ntlmssp/ntlmssp_server.c:539(ntlmssp_server_postauth)
  ntlmssp_server_auth: Using unmodified nt session key.
[2015/05/19 13:50:49.763569,  3, pid=28254, effective(0, 0), real(0, 0)] ../auth/ntlmssp/ntlmssp_sign.c:547(ntlmssp_sign_init)
  NTLMSSP Sign/Seal - Initialising with flags:
[2015/05/19 13:50:49.763614,  3, pid=28254, effective(0, 0), real(0, 0)] ../auth/ntlmssp/ntlmssp_util.c:34(debug_ntlmssp_flags)
  Got NTLMSSP neg_flags=0xe2088215
    NTLMSSP_NEGOTIATE_UNICODE
    NTLMSSP_REQUEST_TARGET
    NTLMSSP_NEGOTIATE_SIGN
    NTLMSSP_NEGOTIATE_NTLM
    NTLMSSP_NEGOTIATE_ALWAYS_SIGN
    NTLMSSP_NEGOTIATE_NTLM2
    NTLMSSP_NEGOTIATE_VERSION
    NTLMSSP_NEGOTIATE_128
    NTLMSSP_NEGOTIATE_KEY_EXCH
    NTLMSSP_NEGOTIATE_56
[2015/05/19 13:50:49.763842,  4, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:216(push_sec_ctx)
  push_sec_ctx(0, 0) : sec_ctx_stack_ndx = 1
[2015/05/19 13:50:49.763905,  4, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/uid.c:485(push_conn_ctx)
  push_conn_ctx(0) : conn_ctx_stack_ndx = 0
[2015/05/19 13:50:49.763959,  4, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:316(set_sec_ctx)
  setting sec ctx (0, 0) - sec_ctx_stack_ndx = 1
[2015/05/19 13:50:49.764003,  5, pid=28254, effective(0, 0), real(0, 0)] ../libcli/security/security_token.c:53(security_token_debug)
  Security token: (NULL)
[2015/05/19 13:50:49.764046,  5, pid=28254, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:629(debug_unix_user_token)
  UNIX token of user 0
  Primary group is 0 and contains 0 supplementary groups
[2015/05/19 13:50:49.764118,  4, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/sec_ctx.c:424(pop_sec_ctx)
  pop_sec_ctx (0, 0) - sec_ctx_stack_ndx = 0
[2015/05/19 13:50:49.764183, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:224(create_local_nt_token_from_info3)
  Create local NT token for root
[2015/05/19 13:50:49.764268, 10, pid=28254, effective(0, 0), real(0, 0), class=tdb] ../source3/lib/gencache.c:296(gencache_set_data_blob)
  Adding cache entry with key=[IDMAP/SID2XID/S-1-5-21-2137076357-3068380017-3353387290-1000] and timeout=[四 1月  1 08时00分00秒 1970 CST] (-1432014649 seconds in the past)
[2015/05/19 13:50:49.777476, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/passdb/lookup_sid.c:1388(sid_to_uid)
  sid S-1-5-21-2137076357-3068380017-3353387290-1000 -> uid 4294967295
[2015/05/19 13:50:49.777647,  1, pid=28254, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:430(add_local_groups)
  SID S-1-5-21-2137076357-3068380017-3353387290-1000 -> getpwuid(4294967295) failed
[2015/05/19 13:50:49.777706,  3, pid=28254, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:316(create_local_nt_token_from_info3)
  Failed to finalize nt token
[2015/05/19 13:50:49.777781, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/auth/auth_ntlmssp.c:45(auth3_generate_session_info)
  create_local_token failed: NT_STATUS_UNSUCCESSFUL
[2015/05/19 13:50:49.777841,  5, pid=28254, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:187(dbwrap_check_lock_order)
  check lock order 1 for smbXsrv_session_global.tdb
[2015/05/19 13:50:49.777910, 10, pid=28254, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:133(debug_lock_order)
  lock order:  1:smbXsrv_session_global.tdb 2:<none> 3:<none>
[2015/05/19 13:50:49.777990, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/lib/dbwrap/dbwrap_ctdb.c:1057(fetch_locked_internal)
  Locking db 1795596909 key 44FC5AD5
[2015/05/19 13:50:49.778057, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/lib/ctdbd_conn.c:868(ctdbd_control)
  ctdbd_control: Sending ctdb packet
[2015/05/19 13:50:49.778321, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/lib/dbwrap/dbwrap_ctdb.c:953(db_ctdb_record_destr)
  Unlocking db 1795596909 key 44FC5AD5
[2015/05/19 13:50:49.778459,  5, pid=28254, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:146(dbwrap_lock_order_state_destructor)
  release lock order 1 for smbXsrv_session_global.tdb
[2015/05/19 13:50:49.778513, 10, pid=28254, effective(0, 0), real(0, 0)] ../lib/dbwrap/dbwrap.c:133(debug_lock_order)
  lock order:  1:<none> 2:<none> 3:<none>
[2015/05/19 13:50:49.778598, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:2598(smbd_smb2_request_error_ex)
  smbd_smb2_request_error_ex: idx[1] status[NT_STATUS_UNSUCCESSFUL] || at ../source3/smbd/smb2_sesssetup.c:131
[2015/05/19 13:50:49.778658, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:2499(smbd_smb2_request_done_ex)
  smbd_smb2_request_done_ex: idx[1] status[NT_STATUS_UNSUCCESSFUL] body[8] dyn[yes:1] at ../source3/smbd/smb2_server.c:2651
[2015/05/19 13:50:49.778709, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:874(smb2_set_operation_credit)
  smb2_set_operation_credit: requested 31, charge 1, granted 1, current possible/max 512/512, total granted/max/low/range 1/8192/3/1
[2015/05/19 13:50:49.778969, 10, pid=28254, effective(0, 0), real(0, 0)] ../source3/smbd/smb2_server.c:1002(smbd_server_connection_terminate_ex)
  smbd_server_connection_terminate_ex: reason[NT_STATUS_CONNECTION_RESET] at ../source3/smbd/smb2_server.c:3309
Comment 15 crashskyshadow 2015-05-19 15:21:50 UTC
I tried the following suggestions..
ethtool -K enp6s0 tso off
ethtool -K enp6s0 tx off

but it not work.
Comment 16 crashskyshadow 2015-05-20 01:45:02 UTC
The reason is I grow my filesystem from 2TB to 6TB.
Could you confirm this is Samba issue or filesystem issue?
Comment 17 Björn Jacke 2015-05-20 06:22:01 UTC
Ira: you objected to closing the e1000 bug here. You should then asign the ticket to you and take care of it also ;-)
Comment 18 crashskyshadow 2015-05-20 06:35:44 UTC
(In reply to Björn Jacke from comment #17)

Hi, do we catch sight of below:

[2015/05/14 16:20:59.997075,  1, pid=26535, effective(0, 0), real(0, 0)] ../source3/auth/token_util.c:430(add_local_groups)
  SID S-1-5-21-545351224-1600352317-1800318577-1000 -> getpwuid(4294967295) failed


the source code are below:
		pass = getpwuid_alloc(tmp_ctx, uid);
		if (pass == NULL) {
			DEBUG(1, ("SID %s -> getpwuid(%u) failed\n",
				sid_string_dbg(&result->sids[0]),
				(unsigned int)uid));
		}


Is there the bug?
Comment 19 Ira Cooper 2015-05-20 06:53:08 UTC
(In reply to Björn Jacke from comment #17)

Bjorn,

Arguing something is valid, and taking it are different.

If you disagree with the analysis I did, that's fine.

But the bug should go through normal triage etc.
Comment 20 crashskyshadow 2015-05-20 07:26:34 UTC
(In reply to Ira Cooper from comment #12)
the file created by vdbench tool.
Comment 21 Stefan Metzmacher 2015-05-24 19:51:48 UTC
(In reply to crashskyshadow from comment #0)

I'd assume this is the same as https://bugzilla.samba.org/show_bug.cgi?id=9702.
Which is fixed in 4.1.18.
Comment 22 crashskyshadow 2015-05-25 03:05:06 UTC
step to reproduce:
1: filesystem great than 6T
2: use synchronous io in samba
3: copy 1.5T data once

samba will get NT_STATUS_CONNECTION_RESET.

There is no issue when use asynchronous io on the same test case ,same environment. 
I set  below parameters:
aio read size=32768
aio write size=32768

I am not familiar with samba source code. So not find issue in source code.
Feel free to close this Bug.