Bug 8637 - vfs_recycle move to different partition/mount (cross-device move) "silently" fails (creates only directory, but no recycled file)
Summary: vfs_recycle move to different partition/mount (cross-device move) "silently" ...
Status: NEW
Alias: None
Product: Samba 3.6
Classification: Unclassified
Component: VFS Modules (show other bugs)
Version: 3.6.1
Hardware: All All
: P5 enhancement
Target Milestone: ---
Assignee: Samba Bugzilla Account
QA Contact: Samba QA Contact
URL:
Keywords:
: 10666 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-11-30 03:10 UTC by bugs-samba
Modified: 2023-01-24 21:38 UTC (History)
9 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description bugs-samba 2011-11-30 03:10:22 UTC
Hi!

[global]
    recycle:keeptree = yes
    recycle:versions = yes
    recycle:touch = yes
    recycle:touch_mtime = yes
    recycle:exclude = ?~$*,~$*,*.tmp,*.temp,*.TMP
    recycle:exclude_dir=  /tmp,/temp,/cache,/Tmp,Tmp
    recycle:repository = _recycle
    recycle:directory_mode = 0777
    recycle:maxsize = 536870912

[Data]
    path = /mnt/data/data
    vfs objects =  recycle
    recycle:repository = Tmp/_recycle/_recycle.%u

/mnt/data/daten contains it's own data + a subdir "projects" which is a different mount from a different device.
Deleting from any subdir and depth of the main partition works, deleting from "projects" results in the directory structure being created, but no deleted file appearing. There is no error in any log file after this at log level = 1.

I guess it may be using rename(2) or something similar for performance reasons and that may not work inter-device. I don't know why there is no logged error though (at level 1).

At level 3, I think the problem is apparent:

[2011/11/30 04:05:53.811158,  3] modules/vfs_recycle.c:646(recycle_unlink)
  recycle: Move error 18 (Invalid cross-device link), purging file XXX (Tmp/_recycle/_recycle.YYY/XXX)

I guess the ",  3" means it's a level 3 msg. I don't know why such an error message is only available at level 3, because level 3 already includes lots of misc. output making it kind of hard to see this error amidst all the messages.

Thanks,
Regards
Comment 1 Jeremy Allison 2011-12-05 20:30:22 UTC
Not a blocker I'm afraid - this would be an enhancement to how the vfs_recycle works.

What is needed here is a background process to do the copy whilst the reply to the unlink is deferred.

Jeremy.
Comment 2 bugs-samba 2011-12-07 11:38:05 UTC
How is it not a blocker when the basic functionality is not working (file is not recycled) and no user visible error is produced? A user would expect this to work.
At least critical would apply (loss of data).

Also, as a quick fix/workaround, could you just use simple move or copy/delete logic if the code path fails?

Thanks
Comment 3 Jeremy Allison 2011-12-14 17:21:40 UTC
This comes under the "admin should know better" case. Not the users problem, the admin should have arranged the storage more intelligently.
Comment 4 Roland Langner 2012-05-11 13:25:15 UTC
Hi,

I think this could be a problem for many samba-sites.
So with one of my samba servers. That run continuously for 220 days.
Now i updated to version 3.6.5 and recycle was broken for me.
I didn't realize it. A user informed me that deleted files doesn't appear in the bin.

It worked before. What was the reason to change the old behaviour?
Shouldn't there be no waring in the release notes?

Regards
Roland Langner
Comment 5 Stefan Metzmacher 2012-11-27 11:58:33 UTC
Does the "crossrename" vfs module after the recycle module fixes this?

vfs objects = recycle crossrename
Comment 6 Reolin 2013-04-17 18:50:09 UTC
Hi. 
"crossrename" vfs module helps, but now there is no any records about renaming or deletion in log (using full_audit). 
Now I can't see if user deleted the file.
Comment 7 Dmitry Melekhov 2013-04-26 03:47:11 UTC
Hello!

Just found that I hit this bug ( and this is really bug because behaviour was changed without any notice ) when redhat released 3.6 for rhel.
And man page still have nothing about this. 
Is it possible to update documentation at least?
So "admin should know better"  will be true? ;-)

btw, I think it is better to revert back expected behaviour.

Thank you!
Comment 8 Stefano Torricella 2013-05-11 14:00:54 UTC
+1 !

just upgraded from debian squeeze to wheezy and hit this annoying bug.
why annoying ? because in the previous version this worked !!

i see that this is also an old bug (2 years ago).
luckily adding the "crossrename" has helped and i dont' need
the audit but i don't understand why change the previous behaviour
when you can add another parameter or create a different modules (recycle_fast ?)
with the "link" instead of the "file copy".
I understand that with the hard link (vs the file copy) you gain in performance but as already told by other, this don't work across different file systems and the fact that the path is created but the file is missing is a sneaky bug !
Comment 9 HHJB 2013-05-13 16:47:37 UTC
To give this a little more input, I've got the same problem here on a wheezy driven server now but without this directories spreaded over different filesystems. In my case there is only ONE filesystem but on a btrfs filesystem which spans two disks at the moment. The funny thing, the recyler works within user homes on the same filesystem. Only shared recycler for group directories has the described issue.

Awaiting to be fixed.
Comment 10 heupink 2013-06-12 19:02:09 UTC
I just found out the hard way that my recycle bin no longer works... :-(
Comment 11 Magnetworks 2013-07-15 17:39:20 UTC
 Hello, 

   Adding the crossrename helped but as already stated this inhibit the audit logs. 
    
   We would appreciate to having this feature enable in the next patch cycle. 

  Thanks
Comment 12 Cédric 2014-01-27 22:37:42 UTC
I used to store the trash bin on a different device for several reasons, and that's really annoying.

With vfs objects = recycle crossrename it can do the trick for small files, but on bigger files, exactly the same problem.

I hop it'll be fixed on the next update.
Comment 13 Björn Jacke 2014-01-27 22:41:09 UTC
On 2014-01-27 at 22:37 +0000 samba-bugs@samba.org sent off:
> With vfs objects = recycle crossrename it can do the trick for small files, but
> on bigger files, exactly the same problem.

see man page of vfs_crossrename how to modify the size of files that the
crossrename module will handle
Comment 14 Marcio Merlone 2014-05-30 18:19:34 UTC
I agree this is critical and not an enhancement, since there is data loss. And the "admin *could* know this better" if it were documented somewhere. It seems intelligent to me to move what may be garbage (the user is deleting, so may be garbage) out of my expensive SAS drives to cheap SATA ones. And this bug status should also be changed to "OLD" instead of "NEW".

Now that I've protested and feel relieved, to the constructive side :)
Use crossrename may be a workaround, but not a solution, this behavior should be adjustable and better documented on vfs_recycle man page/wiki.

Best regards.
Comment 15 CurlyMoo 2014-06-24 07:00:48 UTC
*** Bug 10666 has been marked as a duplicate of this bug. ***
Comment 16 j.rypacek 2020-02-25 08:14:39 UTC
It's almost 10 years without progress, but I still would like to add some related facts, because I've just setup new NAS with samba 4.10.7 (ubuntu) and hit this bug. ZFS is more popular nowadays and will be probably used more and more, so samba should handle this (at least some way).

Possible solutions
- Not using multiple ZFS datasets, but they are really useful in terms of backups...
- Not using "global" samba share over multiple ZFS datasets. But this is also useful for managing files of other people or just when having many ZFS datasets.
- Use vfs_crossrename with high sizelimit. (example below)
- Fix vfs_recycle to do "crossrename functionality" itself
- Fix vfs_recycle to do "move" on the background, but not sure how safe it is.

But even without fix, vfs_recycle should write error into log and add note into man page, because this is data loss.

I'm also little confused, because vfs_crossrename is advertised to do move between filesystems, but samba (at least 4.10.7) is able to move files even without vfs_crossrename.


My config to handle this

# recycle and crossrename set to 100 GiB
# not using single recycle for all shares to prevent slow moving between ZFS datasets for regular shares

vfs objects = recycle crossrename
crossrename:sizelimit = 102400
recycle:repository = .recycle
recycle:maxsize = 107374182400
Comment 17 CurlyMoo 2022-10-23 21:39:47 UTC
Ever since my last message samba has worked fine in the setup i ran back then. So with the addition of the vfs module crossrename. However, since i switched to Proxmox and run Samba inside an LXC container, it doesn't anymore. 

Basically my setup is the same as in post https://bugzilla.samba.org/show_bug.cgi?id=10666#c0

The ZFS filesystems are exposed to the LXC container via bind mounts as configured in the Proxmox configuration:

mp0: /data/users1/Work,mp=/data/users1/Work

All of this works fine except for the recycle bin. When deleting a file, the directory tree is created but the file itself isn't copied to the recycle bin inside the tree. The recycle functionality works fine when files are on the same filesystem as the recycle bin itself, but not across filesystems.

This is the relevant part (i believe) of the Samba configuration:

    vfs objects = recycle crossrename
    recycle:repository = recycle
    recycle:keeptree = yes
    recycle:touch = yes
    recycle:versions = yes
    recycle:maxsize = 107374182400
    recycle:exclude =
    recycle:excludedir =
    recycle:noversions =
    crossrename:sizelimit = 102400

As you can see i already tried the suggestions given by j.rypacek@gmail.com

I found that this is the conflicting line inside vfs_crossrename.c but haven't been able to dig deeper because of my lack of knowledge of the samba architecture:

	ret = SMB_VFS_NEXT_UNLINKAT(handle,
				    dstfsp,
				    dest,
				    0);

https://github.com/samba-team/samba/blob/master/source3/modules/vfs_crossrename.c#L97
Comment 18 CurlyMoo 2022-10-23 21:42:01 UTC
In addition to my last post, this is the OS and samba version i'm running (inside the LXC container):

# lsb_release -a
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy

# smbd --version
Version 4.15.9-Ubuntu
Comment 19 Virtually Nick 2023-01-24 21:38:23 UTC
Just adding my input to this - I also ran into this issue and was completely baffled until stumbling across this particular bug entry.

I have several Linux servers (CentOS 7/8, Rocky 8/9) running ZFS (2.1.x), and had thought it would be great if I could keep my recycle folder on its own filesystem. After banging my head against the wall for quite a while, and doing several Google searches, I came across this and at least know that I'm not the only one hitting the issue.

I've tried using the crossrename VFS module in combination with the recycle module, and, like the most recent folks posting here, that doesn't seem to resolve the issue - I get the same behavior where the folder structure is created but no files ever show up.

Moving back to keeping the recycle folder within the same filesystem seems to work.