Context: I have a workstation W (running Windows 7 Professional 64 bit) and a server S running Debian GNU/Linux 64 bit). The workstation is a 6-core Xeon, the server is a 4-core i7, both with hyperthreading. The workstation has 12GB of RAM, the server has 16. The workstation has on-board dual Intel gigabit Ethernet controllers (with Jumbo frames enabled, IP checksum offload, TCP checksum offload, ...). The server has dual Marvell SysKonnect gigabit Ethernet controllers (with Jumbo frames enabled, IP checksum offload, TCP checksum offload, ...). They are connected via a Cisco SG300-10 managed gigabit switch with Jumbo frames enabled. On the server, I have installed a fresh copy of Debian wheezy (kernel 3.2.0, samba 3.6.6). On the system hard drive (Samsung 1TB, 7200 RPM) I have created two logical volumes, one formatted with XFS and exported via samba, and the other exported through iSCSI. iperf with 128KB window on the Linux side and 1024KB window and 1024KB length of buffer to send shows the network speed to be ~800mbit/second. Importing the iSCSI partition on Windows, formatting it with NTFS, then copying 11GB (basically tarring up C:\Program Files using 7-zip, no compression) and monitoring the performance with 'dstat 5' on Linux produces a steady stream of network receives and disk writes at 75-79 MBytes/second (attached). Connecting the mounted filesystem that is exported via Samba and writing the same test file produces network receive rates that fluctuate between 21 MBytes/s and 51 MBytes/s (but clustered around 35-43 MBytes/s) The disk writes are also interesting, as they go between 2.5KBytes/s to 136MBytes/s. At the recommendation of a fellow LUG member, I have created a 13GB tmpfs filesystem and exported it via Samba, then run my workload. The 'dstat 5' output is attached as well. Other than the test, the boxes are completely idle.
Created attachment 7721 [details] 1st trace iSCSI
Created attachment 7722 [details] 2nd trace iSCSI
Created attachment 7723 [details] 1st trace Samba
Created attachment 7724 [details] 2nd trace Samba
dtrace is not really informative here. A tunables you might try: max protocol = smb2 aio write size = 1 aio read size = 1 vfs objects = aio_fork
aio_fork seems to not be available in Debian: [2012/07/29 09:46:28.453375, 0] smbd/vfs.c:173(vfs_init_custom) error probing vfs module 'aio_fork': NT_STATUS_UNSUCCESSFUL [2012/07/29 09:46:28.453407, 0] smbd/vfs.c:315(smbd_vfs_init) smbd_vfs_init: vfs_init_custom failed for aio_fork [2012/07/29 09:46:28.453427, 0] smbd/service.c:902(make_connection_snum) vfs_init failed for service space [2012/07/29 09:46:28.454402, 0] smbd/vfs.c:173(vfs_init_custom) error probing vfs module 'aio_fork': NT_STATUS_UNSUCCESSFUL [2012/07/29 09:46:28.454434, 0] smbd/vfs.c:315(smbd_vfs_init) smbd_vfs_init: vfs_init_custom failed for aio_fork [2012/07/29 09:46:28.454454, 0] smbd/service.c:902(make_connection_snum) vfs_init failed for service space [2012/07/29 09:46:28.455098, 0] smbd/vfs.c:173(vfs_init_custom) error probing vfs module 'aio_fork': NT_STATUS_UNSUCCESSFUL [2012/07/29 09:46:28.455129, 0] smbd/vfs.c:315(smbd_vfs_init) smbd_vfs_init: vfs_init_custom failed for aio_fork [2012/07/29 09:46:28.455148, 0] smbd/service.c:902(make_connection_snum) vfs_init failed for service space [2012/07/29 09:46:28.457339, 0] smbd/vfs.c:173(vfs_init_custom) error probing vfs module 'aio_fork': NT_STATUS_UNSUCCESSFUL [2012/07/29 09:46:28.457407, 0] smbd/vfs.c:315(smbd_vfs_init) smbd_vfs_init: vfs_init_custom failed for aio_fork [2012/07/29 09:46:28.457427, 0] smbd/service.c:902(make_connection_snum) vfs_init failed for service space [2012/07/29 09:46:28.458609, 0] smbd/vfs.c:173(vfs_init_custom) error probing vfs module 'aio_fork': NT_STATUS_UNSUCCESSFUL [2012/07/29 09:46:28.458636, 0] smbd/vfs.c:315(smbd_vfs_init) smbd_vfs_init: vfs_init_custom failed for aio_fork [2012/07/29 09:46:28.458655, 0] smbd/service.c:902(make_connection_snum) vfs_init failed for service space
Well, then no sensible aio is available. This leaves you at "max protocol = smb2"
(In reply to comment #7) > Well, then no sensible aio is available. This leaves you at "max protocol = > smb2" As I have used an SSD on the client and a RAM-disk on the server, I believe I was able to take the disk-io out of the picture (and it is only the disk-io on the server might be helped by aio). My main concern right now is the lower network throughput that we see in the dstat log with Samba when compared with the dstat log with iSCSI. With iSCSI we are pushing 75-78MB/s while Samba only manages 26-55MB/s .
Created attachment 7725 [details] SSD to RAM-disk, using max protocol = smb2 The range in network throughput is very interesting: from 2MB/s to 62MB/s.
Well, as I said: dstat output is not really relevant here. We need network traces and your full smb.conf file.
Created attachment 7726 [details] Full smb.conf
(In reply to comment #10) > Well, as I said: dstat output is not really relevant here. We need network > traces and your full smb.conf file. I have attached the smb.conf that I'm using. I do have a managed switch and I can mirror a port - what options should I give to tcpdump so I can get a capture that is useful for you? I suppose I start my test, leave it running for 30 seconds, then start a capture for 30 seconds or so?
See http://wiki.samba.org/index.php/Capture_Packets Please make sure the TCP connection setup is part of the trace. We need to see the options negotiated when the client connects