Something that has changed in Samba 4 causes gvfs to fail with "Software caused connection abort" when copying a file. This has been particularly noticeable since Ubuntu 14.04 changed to libsmbclient 4.1.6. Most of these reports seem to be when using NAS devices. Bugs: [1] https://bugzilla.gnome.org/show_bug.cgi?id=729010 [2] https://bugzilla.gnome.org/show_bug.cgi?id=729298 [3] https://bugs.launchpad.net/ubuntu/+source/nautilus/+bug/1312362 [4] https://bugzilla.redhat.com/show_bug.cgi?id=920536 [5] http://askubuntu.com/questions/450206/how-to-fix-a-software-caused-connection-abort-error-in-ubuntu-14-04 Both [1] and [4] are NAS Samba 3.0.32 servers. There is a log at [1] with some debug output: https://bug729010.bugzilla-attachments.gnome.org/attachment.cgi?id=275743 The interesting bit of the log: """ Queued new job 0x7fa4300054a0 (GVfsJobRead) smbc_read(0x7fa430045700, 65534) --> 65534 job_read send reply, 65534 bytes Queued new job 0x7fa4300055c0 (GVfsJobRead) smbc_read(0x7fa430045700, 65534) --> 65534 job_read send reply, 65534 bytes Queued new job 0x7fa4300054a0 (GVfsJobRead) smbc_read(0x7fa430045700, 65534) --> 65534 job_read send reply, 65534 bytes Queued new job 0x7fa4300055c0 (GVfsJobRead) smbc_read(0x7fa430045700, 65534) --> 65534 job_read send reply, 65534 bytes Queued new job 0x7fa4300054a0 (GVfsJobRead) smbc_read(0x7fa430045700, 65534) ../libcli/smb/smbXcli_base.c:1777: 3 bytes left, expected at least 258 smb1cli_inbuf_parse_chain - NT_STATUS_INVALID_NETWORK_RESPONSE map_errno_from_nt_status: 32 bit codes: code=c000020c smbc_error 1 233 (0xe9) -> 103 job_read send reply, 0 bytes """ I haven't actually been able to reproduce this myself, but hopefully someone here will know what's going on :-) Thanks!
I'm the author of one of the bug reports that Ross mentions above. I can confirm that my NAS is running Samba version 3.0.32 and that I immediately ran into serious trouble when I upgraded to Ubuntu 14.04. I'm attaching a wire trace to this report in case it helps to diagnose the problem. Since this problem is fairly serious for me, I'd be happy to run any tests that might help resolve the issue, so please don't hesitate to ask.
Created attachment 9931 [details] Wireshark capture of the bug
Ok, looking at that trace at packet 139 it's a RST reply from the server, meaning a crash I think. The file it's trying to read is of size 304402 and the client is sending 2 back-to-back readX requests of: Packet 132. read 64512 bytes at offset 0 Packet 133. read 1022 bytes at offset 64512 to which no response is received other than an ACK at the TCP layer from the remote kernel. Then after a long timeout the client sends a close request at packet 137 (it's given up) and then the server crashes. This does look just like a server bug. We do drive servers much harder now in the client code in order to get decent wire speed. Can you update the server on that box ? It's Samba 3.0.x, so you should be able to. The only other option would be to deliberately hobble the libsmbclient code against 3.0.x servers, but I'm loathe to do that just for a specific bug. Can you get a smbd debug level 10 log out of the NAS server ?
While being the owner of such a device, I understand the point of view. Unfortunately, those are closed end-user consumer products. Destined to common mortals. We're not all hackers who can force any Linux-running device into submission. :-) If the product vendor has no more updates, then that's the end of it for many people. I've just checked: no updates. And I'm not expecting any more to come. Wouldn't it be a bit of nonsensical to cut off people from their nas device without offering them a workaround ? Some kind of legacy mode switch ? An optional package to install ? I don't know ... There *must* be something else possible besides asking people to hack into their devices. :-) This is the democratic, common-sense, open-source world we're talking about. Not some kind of bureaucratic, private, closed, commercial operation ! But if it's a choice of replacing my Nas box or my Linux... :-)
It's not intentional :-). I'm not asking you to hack into your device (well, not unless you want to :-), I need some more info on what is going wrong. The latest client libraries completely comply with the SMB1 (and SMB2+SMB3) specs, and work well against most servers. So it's not reasonable to deliberately go back to a 'one-request-at-a-time' mode (which would reduce speed for all users) just to fix an issue with one buggy server. Having said that I'm not averse to adding a switch to allow you to turn that on, I just need more info about what it is we're doing that's causing the server to crash. That's where you need to work with me ! Jeremy.
(In reply to comment #3) > Ok, looking at that trace at packet 139 it's a RST reply from the server, > meaning a crash I think. > > The file it's trying to read is of size 304402 and the client is sending 2 > back-to-back readX requests of: > > Packet 132. read 64512 bytes at offset 0 > Packet 133. read 1022 bytes at offset 64512 > > to which no response is received other than an ACK at the TCP layer from the > remote kernel. Then after a long timeout the client sends a close request at > packet 137 (it's given up) and then the server crashes. Why does the client code send two requests concurrently of those sizes? When copying, gvfs sends requests of size 65534 bytes, not concurrently.
(In reply to comment #5) > It's not intentional :-). I'm not asking you to hack into your device (well, > not unless you want to :-), I need some more info on what is going wrong. > > The latest client libraries completely comply with the SMB1 (and SMB2+SMB3) > specs, and work well against most servers. So it's not reasonable to > deliberately go back to a 'one-request-at-a-time' mode (which would reduce > speed for all users) just to fix an issue with one buggy server. > > Having said that I'm not averse to adding a switch to allow you to turn that > on, I just need more info about what it is we're doing that's causing the > server to crash. That's where you need to work with me ! > > Jeremy. I wouldn't mind to turn the speed down on this machine. Or revert to a lower SMB release... I seem to remember the device has a hack somewhere to allow temporarily ssh access. If that works out, I'll peek around to find out what kind of Linux it is. After that I'll help to enable logging or so.
Could get in - apparently it's a Zyxel device of some kind. I don't know where to look to determine the exact type. I'm no guru either - so to find out what kind of Linux it precisely is... :-) But it seems to be a linux (Arch ?) arm box, with two running smbd processes. ==> uname -a Linux nas-server 2.6.31.14_SMP_820 #4 SMP Mon Oct 22 21:13:08 CST 2012 armv6l GNU/Linux ==> ps | grep smb 1788 root 7040 S N /usr/sbin/smbd -D 1791 root 7040 S N /usr/sbin/smbd -D ==> Setting the log level 10 (1) /etc/init.d/samba.sh stop (2) mv /var/log/samba/log.smbd something-else (3) vi /etc/samba/smb.conf => add "log level = 10" in [global] (4) /etc/init.d/samba.sh start (5) try to copy smb://public/meteo.log from smb => error The file "meteo.log" seems to be mentioned in the log file. So I hope there's something there to explain the error. Weird thing is that the log file doesn't seem to change very fast. Running "tail -f" didn't show something happening. Weird also is that at the default level the log file didn't even show something suggesting an error. With the default log level, copying the file & causing the error just adds two entries to the log file: [2014/05/10 22:50:15, 1] smbd/service.c:close_cnum(1230) lorient (192.168.123.80) closed connection to service public [2014/05/10 22:50:16, 1] smbd/service.c:make_connection_snum(1033) lorient (192.168.123.80) connect to service public initially as user admin (uid=501, gid=500) (pid 2060) I'd rather expect it to mention an error - as being the suspected source of the bug. ================================================================================================================= No other log files could be found on this device. '/var/log' contains one directory 'samba'. However, MAYBE, something interesting is the latest entry to dmesg. It is added each time I'm trying to copy the file. So it must be related to the problem. ------------[ cut here ]------------ WARNING: at arch/arm/plat-oxnas/fast_open_filter.c:52 non_fast_access_denied+0xa8/0xc0() Modules linked in: ntfs gmac mii [<c0244d08>] (unwind_backtrace+0x0/0xe4) from [<c0263098>] (warn_slowpath_common+0x4c/0x80) [<c0263098>] (warn_slowpath_common+0x4c/0x80) from [<c024c618>] (non_fast_access_denied+0xa8/0xc0) [<c024c618>] (non_fast_access_denied+0xa8/0xc0) from [<c02c5ce4>] (vfs_read+0x1c/0x1cc) [<c02c5ce4>] (vfs_read+0x1c/0x1cc) from [<c02c5f08>] (sys_pread64+0x74/0x90) [<c02c5f08>] (sys_pread64+0x74/0x90) from [<c0240040>] (ret_fast_syscall+0x0/0x2c) ---[ end trace 57ae9d6a742cbc11 ]--- vfs_read() File c4b5b940, offset 932838 denying due to FAST mode ================================================================================================================= I'm including both the log file and smb.conf. Maybe those contain usable clues ? That's all I have for you now. :-)
Created attachment 9935 [details] smb.conf + log level 10 smbd
Max SMB1 receive size is less than 65536.(In reply to comment #6) > Why does the client code send two requests concurrently of those sizes? > When copying, gvfs sends requests of size 65534 bytes, not concurrently. Max SMB1 receive size is less than 65536. So libsmbclient splits up the amount requested into as many read requests as it can and sends them all simultaneously. That's why gvfs should ask for as large an amount as it can, let libsmbclient take care of the lower level multiplexing.
Is there anything else I can help with ?
I don't see any Samba errors in that log - and what your dmesg log looks like is a kernel bug. Possibly the newer libsmbclient code is driving the kernel much harder on that box and has exposed a kernel bug. That's what it looks like for now - I don't see any Samba errors at all. Jeremy.
From your smb.conf, there are options in the share definitions : incoherent=yes direct writes=2 These are not options that we (the Samba developers) know anything about :-(. Looks like this box has a custom kernel, and a custom version of Samba that has been heavily modified. Did you get source code for the Samba that ships with it ? Can you get source code for the Samba that ships with it ? I'd be very interested in seeing what those options do... CC:ing Richard on this one so he can add it to his list of weird Samba modifications by vendors who haven't contributed them back upstream :-). Jeremy.
(In reply to comment #4) > While being the owner of such a device, I understand the point of view. > Unfortunately, those are closed end-user consumer products. Destined to common > mortals. We're not all hackers who can force any Linux-running device into > submission. :-) > > If the product vendor has no more updates, then that's the end of it for many > people. I've just checked: no updates. And I'm not expecting any more to come. The problem is that we don't know what they have done and are not in any position to support their product. > Wouldn't it be a bit of nonsensical to cut off people from their nas device > without offering them a workaround ? Some kind of legacy mode switch ? An > optional package to install ? I don't know ... There *must* be something else > possible besides asking people to hack into their devices. :-) > > This is the democratic, common-sense, open-source world we're talking about. > Not some kind of bureaucratic, private, closed, commercial operation ! > > But if it's a choice of replacing my Nas box or my Linux... :-) Does it not support NFS? Also, you should be able to ask them for the source code, since you are a customer. I looked around their web site but could not see anywhere to download the source code.
Well if it is really a Zyxel device, ftp://opensource.zyxel.com may help.
(In reply to comment #15) > Well if it is really a Zyxel device, ftp://opensource.zyxel.com may help. Which Zyxel model is it? There's quite a few large tarfiles hidden behind that link.
I was wondering that too - but where to look ?
(In reply to comment #17) > I was wondering that too - but where to look ? Do you actually own the box, so that you could take a look at it and see whether it includes any type description? Maybe you also have access to some sort of configuration GUI, those might also have some type indication. You might also want to take a look at the invoice, those tend to have something in that direction as well, at least in Germany.
Randomly browsing around shows that this one has Samba 3.0.32 (which seems to be the affected version): ftp://opensource.zyxel.com/NSA-2401/111.update.Nov.19/Readme.txt
(In reply to comment #18) > (In reply to comment #17) > > I was wondering that too - but where to look ? > > Do you actually own the box, so that you could take a look at it and see > whether it includes any type description? Maybe you also have access to some > sort of configuration GUI, those might also have some type indication. You > might also want to take a look at the invoice, those tend to have something in > that direction as well, at least in Germany. Yes - it's standing on my desk here. But alas, nothing besides it's a Medion device. I've just got a cashier ticket... This is Belgium, Mate ! :-) I'll poke the device tonight. Might dmesg show something ?
Hi Jeremy, Unfortunately I'm not sure how to update Samba on my NAS. It's an Iomega / Lenovo "Home Media Network Hard Drive, Cloud Edition" home NAS and it's fairly closed off to home users and the manufacturer is no longer offering updates. I can open an SSH terminal to the server, but I don't know how to go about updating Samba on it. I'm currently working on getting you that level 10 log, but I'm having a little getting the log generated on the server. I'll post it as soon as I've managed to iron out the trouble. In any event, looking over these comments, I just want to reiterate that this isn't a problem that's limited to just one Zyxel box. I certainly agree that you shouldn't make a change that would restrict everyone using the latest version of Samba, but since this does affect a few different NASs it seems worth offering a switch for those of us using the affected servers. Thanks, -Brian (In reply to comment #3) > Ok, looking at that trace at packet 139 it's a RST reply from the server, > meaning a crash I think. > > The file it's trying to read is of size 304402 and the client is sending 2 > back-to-back readX requests of: > > Packet 132. read 64512 bytes at offset 0 > Packet 133. read 1022 bytes at offset 64512 > > to which no response is received other than an ACK at the TCP layer from the > remote kernel. Then after a long timeout the client sends a close request at > packet 137 (it's given up) and then the server crashes. > > This does look just like a server bug. We do drive servers much harder now in > the client code in order to get decent wire speed. > > Can you update the server on that box ? It's Samba 3.0.x, so you should be able > to. > > The only other option would be to deliberately hobble the libsmbclient code > against 3.0.x servers, but I'm loathe to do that just for a specific bug. > > Can you get a smbd debug level 10 log out of the NAS server ?
Just poking around in the device's file system for clues... ======================================== cat /proc/nas_model => "STG212" This leads to a number of hits in google. Where it confirms the device runs on Arch arm. Which leads to a model id "AB03" hidden in /etc/init.d/rcS. As indicated in Zyxel's wiki: http://zyxel.nas-central.org/wiki/Usb_key_func.sh ======================================== Probably related: https://github.com/ingmar-k/2.6.31.14_OXNAS_kernel/blob/master/build/fw.stg212.txt It matches the version of the device's kernel exactly: Linux nas-server 2.6.31.14_SMP_820 #4 SMP Mon Oct 22 21:13:08 CST 2012 armv6l GNU/Linux ======================================== Is this something you guys can do something with ?
Why don't we start attacking this from a different angle? I did look at those ftp directories, but I did not find any clear indication for source code. One of those 500MB monsters just contained binaries for example. Everybody with such a device has been given an offer to get the source code for the system. If you have not, your dealer or the manufacturer of the device has committed a violation of our license, the GPL. Can you dig up that documentation and point us at the relevant sources? Thanks, Volker
(In reply to comment #23) > Why don't we start attacking this from a different angle? I did look at those > ftp directories, but I did not find any clear indication for source code. One > of those 500MB monsters just contained binaries for example. > > Everybody with such a device has been given an offer to get the source code for > the system. If you have not, your dealer or the manufacturer of the device has > committed a violation of our license, the GPL. Can you dig up that > documentation and point us at the relevant sources? > > Thanks, > > Volker Isn't the github link not the one for the source code ?
(In reply to comment #24) > (In reply to comment #23) > > Why don't we start attacking this from a different angle? I did look at those > > ftp directories, but I did not find any clear indication for source code. One > > of those 500MB monsters just contained binaries for example. > > > > Everybody with such a device has been given an offer to get the source code for > > the system. If you have not, your dealer or the manufacturer of the device has > > committed a violation of our license, the GPL. Can you dig up that > > documentation and point us at the relevant sources? > > > > Thanks, > > > > Volker > > Isn't the github link not the one for the source code ? That link is just a text file, it is neither Samba nor Kernel source code. I browsed a bit through the git repo there, but I did not find Samba source code there. You should have received the written offer for source code when you bought the device. Please look up that offer how to retrieve the correct source code. You as the buyer of the device are the only one entitled to get the source code, so please download it and put it somewhere, so that we can get easier access to it. Thanks, Volker
Alas, I didn't. As I said: Belgium country. But maybe I can ask the Zyxel guys ?
(In reply to comment #26) > Alas, I didn't. As I said: Belgium country. > > But maybe I can ask the Zyxel guys ? Well, I believe the GPL is international, and also is valid in Belgium. I am not aware about any exemption that a Belgium dealer is not bound by the GPL. I would have to have agreed to this, because I have some copyright in Samba. This really, really smells like a GPL violation by now. Where *exactly* did you buy the device? Thanks, Volker
For what it's worth, I purchased my Iomega Home Media Network Hard Drive, Cloud Edition new from Amazon in the U.S. and the only source code I'm aware of that was included was for a little desktop application that allowed you to configure the NAS. Here's a link to the available downloads online: https://lenovo-na-en.custhelp.com/app/answers/list/p/1031,1085/answer_type/172 (In reply to comment #27) > (In reply to comment #26) > > Alas, I didn't. As I said: Belgium country. > > > > But maybe I can ask the Zyxel guys ? > > Well, I believe the GPL is international, and also is valid in Belgium. I am > not aware about any exemption that a Belgium dealer is not bound by the GPL. I > would have to have agreed to this, because I have some copyright in Samba. > > This really, really smells like a GPL violation by now. Where *exactly* did you > buy the device? > > Thanks, > > Volker
(In reply to comment #28) > For what it's worth, I purchased my Iomega Home Media Network Hard Drive, Cloud > Edition new from Amazon in the U.S. and the only source code I'm aware of that > was included was for a little desktop application that allowed you to configure > the NAS. Here's a link to the available downloads online: > > https://lenovo-na-en.custhelp.com/app/answers/list/p/1031,1085/answer_type/172 The source code does not have to be included, but you need to be shown a way to retrieve it. That link equivalent would be perfectly fine for that Zyxel device we are talking about. Volker
(In reply to comment #27) > (In reply to comment #26) > > Alas, I didn't. As I said: Belgium country. > > > > But maybe I can ask the Zyxel guys ? > > Well, I believe the GPL is international, and also is valid in Belgium. I am > not aware about any exemption that a Belgium dealer is not bound by the GPL. I > would have to have agreed to this, because I have some copyright in Samba. > > This really, really smells like a GPL violation by now. Where *exactly* did you > buy the device? > > Thanks, > > Volker Aldi - Medion is their electronics & co brand.
Understood. I just wanted to report that I'm experiencing the same problems with a very similar setup on an NAS that uses the same undocumented Samba share definitions in smb.conf ("incoherent=yes", etc...) on my Iomega HMNHD/CE and there doesn't seem to be a link to download the relevant source code. -Brian (In reply to comment #29) > (In reply to comment #28) > > For what it's worth, I purchased my Iomega Home Media Network Hard Drive, Cloud > > Edition new from Amazon in the U.S. and the only source code I'm aware of that > > was included was for a little desktop application that allowed you to configure > > the NAS. Here's a link to the available downloads online: > > > > https://lenovo-na-en.custhelp.com/app/answers/list/p/1031,1085/answer_type/172 > > The source code does not have to be included, but you need to be shown a way to > retrieve it. That link equivalent would be perfectly fine for that Zyxel device > we are talking about. > > Volker
Voila ! I've asked where to download the source code. We'll see if that poses a problem to them. http://forum.nas-central.org/viewtopic.php?f=283&t=15009
AHA !? Would this be what you're looking for ? There is a link to a big .exe file which should be GPL sources. http://zyxel.nas-central.org/wiki/Category:NSA-212
Created attachment 9960 [details] Level 10 samba server log This is a level 10 log from an Iomega Home Media Network Hard Drive / Cloud Edition NAS exhibiting a failure when attempting to copy files to a client running Ubuntu 14.04
As I mentioned in an earlier comment, I've been seeing this problem with the Iomega Home Media Network Hard Drive / Cloud Edition, and it also has the mysterious share settings ("Incoherent = yes") in its smb.conf file. I spoke with someone at Lenovo and they referred my to this webpage where you can download the open source components. https://lenovo-na-en.custhelp.com/app/answers/detail/a_id/26776 The direct download address for the file is: http://download.iomega.com/lifeline-firmware/EMCLifeline-GPL-Bellagio-3.2.6.zip Hopefully this can be of help...?
I think the Medion exe file is the source code - there are numerous .c files among which are referring Samba. The readme also mentions it: ============================================================ To build samba-3.0.32: 1.please refer to trunk/sysapps/samba-3.0.32/source/readme-crossbuild.txt ============================================================ Unfortunately this readme-crossbuild.txt file is not in the tar file.
Hi Jan, The Medion file does, indeed, seem to be the source code for your Medion / Zyxel NAS. (If you open and extract the contents of the included trunk.tar.gz file, you'll see that trunk/sysapps/samba-3.0.32/source/readme-crossbuild.txt and all the other files are indeed there.) The file I mentioned should be the source code for the Iomega / Lenovo NAS that I'm using, which is causing the same problem for me. -Brian (In reply to comment #36) > I think the Medion exe file is the source code - there are numerous .c files > among which are referring Samba. The readme also mentions it: > > ============================================================ > > To build samba-3.0.32: > > 1.please refer to trunk/sysapps/samba-3.0.32/source/readme-crossbuild.txt > > ============================================================ > > Unfortunately this readme-crossbuild.txt file is not in the tar file.
Jan, Sorry -- I see what you mean. The directory is there, but readme-crossbuild.txt isn't. My mistake. -B (In reply to comment #36)
Checking the source code itself is a bit ... out of my league I'm afraid. :-) Is there anything else for your help ?
Hi all, I see that there hasn't been any activity on this report in the last week and that it's still labeled "NEEDINFO". Is there anything else you need to diagnose the problem? Thanks, -Brian
This is (unfortunately) going to be low priority for the active developers. It's a bug in a vendor-custom build, which doesn't happen with standard 3.0.x, so essentially you're asking us to mop up a mess that someone else made :-).
Well, it DID work with Samba 3. So maybe ... Is there by any chance a way to revert/disable/... something in Samba 4 so it does work again ? By configuration or source change & rebuild ? I certainly would appreciate a hint or hunch about the possibilities of a workaround. I would even know where to start looking. The only way I have now is to throw away the OS and replace it with something running Samba 3. Which would a be shame really. But I'm not going to throw away the NAS device. EVEN not if somebody at Zyxel made a mess from it.
(In reply to comment #42) > Well, it DID work with Samba 3. So maybe ... > > Is there by any chance a way to revert/disable/... something in Samba 4 so it > does work again ? Not easily by code changes in the client. We'd have to rip out an entire layer that was added to allow multiplexing of reads/writes deliberately to increase transfer speed. Although this is gated off the server-returned max-mux value. Can you try setting the paramter: max mux = 1 in the [global] section of your server smb.conf and restart the NAS server. That might help (although it might break Windows clients, if you care).
I can certainly appreciate that. Clearly, though, it's not just one bad vendor-custom build since it affects, at a minimum, both Zyxel and Iomega NAS boxes. In any event, I'll get in touch with the folks responsible for my Iomega NAS and will let them know that there's a problem with it. Thanks, -Brian (In reply to comment #41) > This is (unfortunately) going to be low priority for the active developers. > It's a bug in a vendor-custom build, which doesn't happen with standard 3.0.x, > so essentially you're asking us to mop up a mess that someone else made :-).
Developers are sincerely requested to give this bug a higher priority because it is not a performance problem, it is usability breaking bug as without proper Samba working, the OS (Linux Distros) loose functionality , that being said, I have found a (temporary) workaround to (at-least) IOmega Home Media Network hard Drive Cloud Edition by... enabling "Apple File Sharing" protocol in Protocol Settings of the Web Administration Interface of the IOmega NAS . Then using Files/Nautilus (whatever..) to connect to hard drive using "afp" protocol on Ubuntu 14.04 i.e. instead of smb://<ip_of_nas>/ use afp://<ip_of_nas>/ ( ... when clicking on "Connect to Server" option in the side panel ) ( its better than "ftp" as I have started to face some problems on that too, but that bug is for another bug tracker ;) , some video streaming issues and stuff..) ( functionality similar to smb is restored ) ( p.s. it may also help developers in tracking bug (or creating a more proper workaround ) , as "afp" is fully functional whereas "smb" is not.. )
(In reply to comment #45) > Developers are sincerely requested to give this bug a higher priority because > it is not a performance problem, it is usability breaking bug as without proper > Samba working, the OS (Linux Distros) loose functionality , > that being said, > > I have found a (temporary) workaround to (at-least) IOmega Home Media Network > hard Drive Cloud Edition by... > > enabling "Apple File Sharing" protocol in Protocol Settings of the Web > Administration Interface of the IOmega NAS . > > Then using Files/Nautilus (whatever..) to connect to hard drive using "afp" > protocol on Ubuntu 14.04 > i.e. instead of smb://<ip_of_nas>/ > use afp://<ip_of_nas>/ > > ( ... when clicking on "Connect to Server" option in the side panel ) > > ( its better than "ftp" as I have started to face some problems on that too, > but that bug is for another bug tracker ;) , some video streaming issues and > stuff..) > > ( functionality similar to smb is restored ) > ( p.s. it may also help developers in tracking bug (or creating a more proper > workaround ) , as "afp" is fully functional whereas "smb" is not.. ) one more thing, I don't like using the PROPRIETARY "afp" protocol (its PROPRIETARY for Cryin' Out Loud ) ( And apologies in advance for using Highly "un bug-tracker" -like language
We fully understand your frustration. But the problem is: This is a bug that does not happen with the Samba version the Samba developers ship. It is with a Samba and Kernel version that was modified without telling the Samba developers. And apparently it is even very difficult to find the exact changes that were made. See the multi-step hunt for sources. We publish our software with the permission to modify it. We fully support people who have requirements that our Software does not fulfill with all our knowledge. You can see that every day on our mailing lists. And we would certainly even support Zyxel and IOMEGA and everybody else who finds a bug in their changes if they came to us. I would really ask everybody with this problem to raise a support ticket with their respective manufacturer to make them fix the problem in their changes. Then if they come to us with proper sources, possibly a development environment and everything needed, we will certainly help them. Even if we spotted the offending source code line that causes you the trouble, you would not gain much, because you would have to get a fixed Firmware on your system. For that you need that mentioned development environment anyway. Patching the client to make it severely slower for everybody just for this broken server is not an option, sorry. If with async requests you can crash the server, the server is broken. This will happen with many versions of Windows as well, given the right client load.
I see that Jeremy says that the client is issuing two reads: Packet 132. read 64512 bytes at offset 0 Packet 133. read 1022 bytes at offset 64512 and after this the smbd on the NAS seems to crash. Windows never does this, it does reads of 61440 and 4096. Perhaps a small change for the case where the MAX XMIT or whatever is less than 65536 would fix the problem.
Richard, feel free to submit a patch to make us to only ever send the same pattern as Windows does and hope that Zyxel is happier then.
(In reply to comment #43) > (In reply to comment #42) > > Well, it DID work with Samba 3. So maybe ... > > > > Is there by any chance a way to revert/disable/... something in Samba 4 so it > > does work again ? > > Not easily by code changes in the client. We'd have to rip out an entire layer > that was added to allow multiplexing of reads/writes deliberately to increase > transfer speed. Although this is gated off the server-returned max-mux value. > Can you try setting the paramter: > > max mux = 1 > > in the [global] section of your server smb.conf and restart the NAS server. > That might help (although it might break Windows clients, if you care). YES ! That works ! A VERY big Thank You Samba Team ! :-) It generates a fresh /etc/samba/smb.conf file at each startup. Each time the device is rebooted the changes are discarded. A thing to be remembered. Instead of rebooting: Do the change and run "/etc/init.d/samba.sh restart". The modified smb.conf is used and SMB transfers work again. So - hopefully - this narrows it down a bit ? :-)
(In reply to comment #47) > We fully understand your frustration. But the problem is: This is a bug that > does not happen with the Samba version the Samba developers ship. It is with a > Samba and Kernel version that was modified without telling the Samba > developers. And apparently it is even very difficult to find the exact changes > that were made. See the multi-step hunt for sources. > > We publish our software with the permission to modify it. We fully support > people who have requirements that our Software does not fulfill with all our > knowledge. You can see that every day on our mailing lists. And we would > certainly even support Zyxel and IOMEGA and everybody else who finds a bug in > their changes if they came to us. > > I would really ask everybody with this problem to raise a support ticket with > their respective manufacturer to make them fix the problem in their changes. > Then if they come to us with proper sources, possibly a development environment > and everything needed, we will certainly help them. > > Even if we spotted the offending source code line that causes you the trouble, > you would not gain much, because you would have to get a fixed Firmware on your > system. For that you need that mentioned development environment anyway. > > Patching the client to make it severely slower for everybody just for this > broken server is not an option, sorry. If with async requests you can crash the > server, the server is broken. This will happen with many versions of Windows as > well, given the right client load. I agree. Unfortunately. But I wouldn't mind some kind of throttle-down switch for us poor owners of such devices. It'll be slower, but at least work as intended by the Samba team. In the meantime "max mux = 1" helped for me. So I can keep the OS in its current state for error-testing the problem. Let's see where I can ask Medion for support. Not that I expect them to jump on it...
(In reply to comment #48) > I see that Jeremy says that the client is issuing two reads: > > Packet 132. read 64512 bytes at offset 0 > Packet 133. read 1022 bytes at offset 64512 > > and after this the smbd on the NAS seems to crash. > > Windows never does this, it does reads of 61440 and 4096. > > Perhaps a small change for the case where the MAX XMIT or whatever is less than > 65536 would fix the problem. Richard, you can try this but I doubt it'll fix the problem. The issue is that smbclient/libsmbclient/Linux CIFSFS now issue multiple outstanding reads up to max-mux to a server, something the Windows CIFS/SMB1 client never does (unless it's an internally threaded Win32 program, issuing one read request per thread - and I'm guessing this server would crash in the same way if a Windows client did such a thing). It's not the offsets or sizes that matter, it's dealing with the simultaneous requests. That's why setting max mux=1 as recommended above fixes it.
(In reply to comment #51) > I agree. Unfortunately. But I wouldn't mind some kind of throttle-down switch > for us poor owners of such devices. It'll be slower, but at least work as > intended by the Samba team. Setting max mux = 1 on the server *is* the throttle-down switch for such devices :-). You don't want to mess with the client code, as we want it to be fast against non-broken servers. > In the meantime "max mux = 1" helped for me. So I can keep the OS in its > current state for error-testing the problem. > > Let's see where I can ask Medion for support. Not that I expect them to jump on > it... Unless I see strenuous objections I consider the max mux=1 solution a fix/workaround for this. Jeremy.
> Setting max mux = 1 on the server *is* the throttle-down switch for such > devices :-). You don't want to mess with the client code, as we want it to be > fast against non-broken servers. But then some of us need to hack the devices. Which isn't exactly everybody's cup of tea. Hence my question about a client-side switch. It will put a break on all SMB traffic for the client. People with such modest device probably have no other nas or smb usage anyway. > > > In the meantime "max mux = 1" helped for me. So I can keep the OS in its > > current state for error-testing the problem. > > > > Let's see where I can ask Medion for support. Not that I expect them to jump on > > it... > I've asked at medion.com. But this is probably much too technical for the common helpdesk employee. Expect it to be "classified vertically". :-) > Unless I see strenuous objections I consider the max mux=1 solution a > fix/workaround for this. > > Jeremy. For now yes - however is it *that* uncommon for FOSS projects to work around established flaws of peripheral hardware ? I bet the Linux kernel guys would gladly strangle some vendors concerning incompetence, but in the end they have to work around nonetheless. Because those vendors usually are not inclined to fix their mess. Granted, it's not clean coding to add exceptions to cope with device X and Z. But to expect all those companies to respect standards (or licenses for that matter) ... I would GLADLY fix it and share the solution !
Have you opened the same request with Microsoft? As Jeremy pointed out, a multithreaded Win32 program will also kill that server. Please contact Microsoft to provide the patch for all Windows versions as well.
Ah, by the way, modern Linux kernel clients will have the same problem. Please also contact the Linux community to fix their clients. Very likely you also need to contact Apple and every other reasonably modern SMB client that you might ever use, including your home audio system like SONOS or similar things.
It wasn't meant as a critic ! It's just that most people won't be able to hack into their device. Therefor it would be very helpful for the "common people" to have something on the client side. That's all.
And my remark about "gladly" doing it - means that I am unable to do it. Not as a critic that you or anybody else was not willing to do it. Why would I address a reproach to any of you guys ?
(In reply to comment #58) > And my remark about "gladly" doing it - means that I am unable to do it. Not as > a critic that you or anybody else was not willing to do it. > > Why would I address a reproach to any of you guys ? Don't mind Volker, that's just his way of telling you he's paying attention to the bug :-). Seriously though, the bug is on the server side, and that's where the fix belongs - not in the client libraries.
(In reply to comment #59) > (In reply to comment #58) > > And my remark about "gladly" doing it - means that I am unable to do it. Not as > > a critic that you or anybody else was not willing to do it. > > > > Why would I address a reproach to any of you guys ? > > Don't mind Volker, that's just his way of telling you he's paying attention to > the bug :-). Well, as long as it's clear I didn't mean to be offensive. :-) > > Seriously though, the bug is on the server side, and that's where the fix > belongs - not in the client libraries. I agree. Because it was made without the involvement of the Samba team apparently. However, the probability of Medion/IOMega/Zyxel/... fixing it is practically nil. They sent me to the helpdesk at medion.com. I've made a request - just in case. I bet they scratched their heads about what this moron was expecting from them. :-) Neither am I well versed enough in this subject to fix it myself. It would have been nice to have some kind of ... break/legacy mode/something in the client. Because then the owners of the devices would just have to activate it. But that's apparently far from being easy. My apologies for being slow to grasp this. It's all a shame really because I bought the device EXACTLY because I didn't want to hack around this time. Just put it there and let it work. Next NAS is an RPi with a big hard disk attached ! Anyway, thanks for all the help guys !
I surprised no one has made this comment earlier, but almost certainly the easiest "fix" is for someone to provide a ppa for Ubuntu 14.04 that provides Samba 3 and rebuilds the necessary stuff. I'm sure among the affected owners, someone would be willing (and capable) of doing that (although that would be best discussed on the Ubuntu bug tracker/forum/mailing lists). No messing around with the hardware or making workarounds in Samba 4.
(In reply to comment #61) > I surprised no one has made this comment earlier, but almost certainly the > easiest "fix" is for someone to provide a ppa for Ubuntu 14.04 that provides > Samba 3 and rebuilds the necessary stuff. I'm sure among the affected owners, > someone would be willing (and capable) of doing that (although that would be > best discussed on the Ubuntu bug tracker/forum/mailing lists). No messing > around with the hardware or making workarounds in Samba 4. No, that's not a fix, that's a *regression*. Guys, Samba3 is gone - there is no more Samba3 - it's out of maintenance. There are no more security releases for it (we will of course provide security patches for people doing their own releases based on it). I don't see what the problem is with the server-side max-mux smb.conf fix ? It's fixing the problem in the right place - it's the server that's crashing due to OEMs messing with the code release, so it's the server that needs the fix. You're only lucky you haven't run across Windows client apps that trigger the same problem because believe me it will suffer from the same issue. And what would you do then ? Microsoft isn't going to patch Win7 - Win8.1 due to a buggy modified OEM Samba server installation. Just be happy we figured out how you can make this work. As the OEMs aren't providing updates anyway for these servers just make the one line fix in the smb.conf and be happy.
(In reply to comment #62) > > Just be happy we figured out how you can make this work. As the OEMs aren't > providing updates anyway for these servers just make the one line fix in the > smb.conf and be happy. Of course, it would help if OEMs were smart enough to provide a UI feature allowing users to add lines to the smb.conf and have them added each time the NAS restarts. Sigh. Most of them are too dumb to do that.
(In reply to comment #63) > (In reply to comment #62) > > > > Just be happy we figured out how you can make this work. As the OEMs aren't > > providing updates anyway for these servers just make the one line fix in the > > smb.conf and be happy. > > Of course, it would help if OEMs were smart enough to provide a UI feature > allowing users to add lines to the smb.conf and have them added each time the > NAS restarts. > > Sigh. Most of them are too dumb to do that. Actually, the can't just auto-generate it completely as they have to have users add their own shares somewhere, so you can add the: [global] max mux = 1 line in the text file they use for share definitions.
Next try: Bring back the device to the dealer and get your money back, in case they have advertised SMB support. This is NOT a functional SMB server. That's the only language those guys understand.
(In reply to comment #62) > (In reply to comment #61) > > I surprised no one has made this comment earlier, but almost certainly the > > easiest "fix" is for someone to provide a ppa for Ubuntu 14.04 that provides > > Samba 3 and rebuilds the necessary stuff. I'm sure among the affected owners, > > someone would be willing (and capable) of doing that (although that would be > > best discussed on the Ubuntu bug tracker/forum/mailing lists). No messing > > around with the hardware or making workarounds in Samba 4. > > No, that's not a fix, that's a *regression*. > I agree with that; I quoted "fix" for a reason. But unfortunately some of these devices are locked down, or owned by someone else and you can't just go poking around on its configuration. And maybe it's out of warranty so you can't return it.
(In reply to comment #66) > I agree with that; I quoted "fix" for a reason. But unfortunately some of these > devices are locked down, or owned by someone else and you can't just go poking > around on its configuration. And maybe it's out of warranty so you can't return > it. If you are working with such old devices, please go back to Samba 3.0 libs. There we did not have multi-issue SMB client libraries. That will nicely solve your issue as well.
I bought it last year - still under warranty. :-) I bet I can't be reimbursed because it doesn't properly implement Samba. I'm afraid the FOSS spirit is not really present in those companies. Do yo mean client or server to revert to Samba 3 ? Is that at all possible on Ubuntu 14.04 ? (sorry if this is a dumb question)
If it's still in warranty then hurry up to make your vendor fix the device. I will be happy to assist you if a judge has to decide whether the server has the advertised capabilities or not. I mean to downgrade the client to 3.0. I don't think this will be possible with Ubuntu 14.04, you will have to use a much older version I guess. Your NAS just does not support modern SMB clients, so either you get your NAS fixed, you get it back to the dealer and get your money back or you use ancient clients that your NAS still supports.
(In reply to comment #68) > I bought it last year - still under warranty. :-) > > I bet I can't be reimbursed because it doesn't properly implement Samba. I'm > afraid the FOSS spirit is not really present in those companies. > > Do yo mean client or server to revert to Samba 3 ? Is that at all possible on > Ubuntu 14.04 ? (sorry if this is a dumb question) Ah, by the way: In clear non-FOSS speak: This device is just broken. If your car engine shuts down whenever you change to the 2nd gear, what do you do? Only use the 1st gear or get the engine fixed? I bet you will not be happy with just using the 1st gear, limiting you to something like 30 or 40 km/h.
You are preaching to the choir. :-) However it would amount to a debate about principles only. Because it would be much cheaper to replace the NAS right away and forget about it than to pay a lawyer. :-) Such is reality I'm afraid...
I don't know if this will work on the Zyxel box or other affected NASs, but here's how I was able to apply the fix to my Iomega NAS. I'm assuming that you've already figured out how to establish ssh access to the NAS. There's information on how to do this here: http://iomega.nas-central.org/wiki/Root_access_enabling_(Home_Media_CE) Every time you reboot the server will rebuild the smb.conf file, so you need to edit the one of the files from which it's built. Here's how: Open an ssh shell to the server and log in. Then, mount the volume where the configuration files are located by entering these commands: mkdir /appsvol mknod /dev/loop3 b 7 3 mount -o loop /boot/images/apps /appsvol Then create a backup copy of /appsvol/usr/local/cfg/Shares.xml and add this line to the original file's "<Global Tag>" section using a text editor: <Global Tag="max mux" Value="1"/> Reboot the server and that should cure what ails you. And for the record, this bug does indeed totally crash the NAS when Windows 8 tries to access one of the shares. Adding the new "max mux" setting seems to have resolved all the trouble.
Thanks for the hint- I'll have a try once weather isn't crushing me. arrr... :-)
I know its late, but for those using IOmega NAS ( Home Media Network Hard Drive Cloud Edition ( untested on other models of IOmega or any other manufacturer) ). There is another way to solve the problem ( without throttling ( i.e. Without using "max mux = 1" in smb.conf ) ) [ Link_1 ] : [ http://iomega.nas-central.org/wiki/Application:FFP ] [ Link_2 ] : [ http://nas-tweaks.net/82/installing-and-uninstalling-packages-and-activation-and-deactivation-of-daemons-in-fonz-fun_plug/ ] In Short: Above link helps you to install FFP ( Fonz Fun Plug ) on your NAS which in turn will allow you to install different Linux Software on your NAS including updated smb server. Entire process is way to long to include in comment (hence the above link acts as a pointer in direction to solve your problem) , In Short (again... ): Steps: 1. Install FFP on your NAS ( Link_1 ) 2. Disable SMB protocol support via web-browser based GUI of your NAS 3. Install updated Samba server on your NAS ( Link_2 ) 4. Set the (updated) Samba Server to Auto Start ( Link_2 : Permanent activation of Daemons ). 5. Enjoy ... :) [ Side Note : It works on Windows and Ubuntu 14.04 (and others possibly) ] [ (Another) Side Note : After this fix the auto-detect part in Network Folder ( Ubuntu and Windows ) doesn't seem to work for me, any pointers to how it detects NAS on network might help , but for the time being manually entering IP address of NAS works ] [ (Another)^2 Side Note : FFP plugin is usually available for ( NAS from other manufacturers ) as well, google it ( and the above solution may work for you(Non-IOmega users) as well ) ]
Thanks very much, all! I have an "IOmega Home Media Network Hard Drive, Cloud Edition" that stopped working when we switched from Ubuntu 12.04 to Fedora 20. This thread has been very useful. I will try to complain to Lenovo (the proud owners of IOmega) but I don't have much hope. My warranty is long over. If I had the energy, I'd consider trying to build new firmware from the released GPL code. But I fear that there is too much userland code that is not part of that release.
No such luck with the Medion device - those paranoids dump the whole root file system at boot time. I couldn't find yet how it is reconstructed. It HAS to come from somewhere ! :-)
(In reply to comment #73) > Adding the new "max mux" setting seems to have resolved all the > trouble. I was not able to open files on the NAS after installing Ubuntu 14.04 Following Brian's instructions (exactly as written) resolved my problem. Ubuntu 14.04, Iomega Home Media Network Hard Drive Cloud Edition 1TB, Firmware 3.2.3.15290 This has been a valuable learning experience: it is the first time that I have followed a bug report as it moved though the various development groups until the root cause was identified. Thanks to all!
In Response to Comment 77 Jan Goyvaerts 2014-08-02 12:29:40 UTC: For the Medion device (I've got a Medion Life P89636 / MD86805 / MSN 5004 3463 / EAN 20003418) I've tried an approach similiar to http://www.mikrocontroller.net/articles/P89626/dropbear, i.e. make a script to modify smb.conf and manage to get it started at startup. I also modified zyshd which seems to generate files but that is not necessary, the solution with an additional start-script seems much easier.
(In reply to Jeremy Allison from comment #13) I took a look at the source provided by Medion. They definatly modifyed a few thing, looks like it's an attempt at making things happen a bit faster. They added the following 5 parameters: "preallocate", P_BOOL, default is False "direct writes", P_INTEGER, default is 0 "process timeout" (also refered to as FastTimeout), P_INTEGER, default is 1 "incoherent", P_BOOL, default is False "peek command type", P_BOOL, default is False The modified code where preallocate is used have the following link: http://marc.info/?l=samba&m=123128881423560&w=2 (talks about slow file allocation on vfat partitions) There is a few references to zero_copy_capable in the code. Both "incoherent" and "direct writes" are referenced in the modified receive_smb_raw function, and the comments mention "Is a fast read/write" All in all the changes (minus arm build configurations) comes to about 1250 new lines, and 200 deleted.
(In reply to SkyKnight from comment #75) Sorry to bother you in this solved issue. After quit some work as a newbee in linux i have managed to get past point 2 of your instructions: In Short (again... ): Steps: 1. Install FFP on your NAS ( Link_1 ) 2. Disable SMB protocol support via web-browser based GUI of your NAS 3. Install updated Samba server on your NAS ( Link_2 ) 4. Set the (updated) Samba Server to Auto Start ( Link_2 : Permanent activation of Daemons ). 5. Enjoy ... :) However, now i am at 3, updated the and i can not seem to find the package or way to install samba4 or update the exsisting samba3 server on my Medion Nas... Can you please let me know how to do this?