Bug 7815 - sparse bit handling needs to be more windows-like
Summary: sparse bit handling needs to be more windows-like
Status: RESOLVED FIXED
Alias: None
Product: Samba 3.6
Classification: Unclassified
Component: File services (show other bugs)
Version: 3.6.0pre1
Hardware: Other All
: P3 enhancement
Target Milestone: ---
Assignee: Jeremy Allison
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-11-24 07:07 UTC by Björn Jacke
Modified: 2011-03-01 05:36 UTC (History)
1 user (show)

See Also:


Attachments
0001-s3-add-FILE_ATTRIBUTE_SPARSE-to-get_stat_dos_flags.patch (837 bytes, patch)
2010-12-16 17:28 UTC, Björn Jacke
no flags Details
0002-s3-remove-set_sparse_flag.patch (2.15 KB, patch)
2010-12-16 17:28 UTC, Björn Jacke
no flags Details
0003-s3-vfs_gpfs-map-SPARSE-attribute-to-from-WINATTRs-to.patch (2.23 KB, patch)
2010-12-16 17:29 UTC, Björn Jacke
no flags Details
0004-s3-smbd-add-file_set_sparse-function.patch (2.73 KB, patch)
2010-12-16 17:29 UTC, Björn Jacke
no flags Details
0005-s3-smbd-don-t-filter-attribs-in-g-set_ea_dos_attribu.patch (1.44 KB, patch)
2010-12-16 17:29 UTC, Björn Jacke
no flags Details
0006-s3-smbd-implement-FSCTL_SET_SPARSE-more-correctly.patch (2.67 KB, patch)
2010-12-16 17:29 UTC, Björn Jacke
no flags Details
0007-s3-don-t-do-strict-allocate-with-sparse-files-minor-.patch (907 bytes, patch)
2010-12-16 17:30 UTC, Björn Jacke
no flags Details
0008-s3-don-t-do-strict-allocate-with-sparse-files-minor-.patch (862 bytes, patch)
2010-12-16 17:30 UTC, Björn Jacke
no flags Details
Replacement for 0005 (967 bytes, patch)
2010-12-17 14:26 UTC, Jeremy Allison
no flags Details
0007-file_set_sparse-needs-to-be-a-handle-based-call.patch (3.17 KB, patch)
2010-12-17 14:27 UTC, Jeremy Allison
no flags Details
0008-Move-checks-inside-file_set_sparse-to-allow-it-to-be.patch (2.73 KB, patch)
2010-12-17 14:27 UTC, Jeremy Allison
no flags Details
incomplete torture tests (12.48 KB, patch)
2010-12-18 01:59 UTC, Stefan Metzmacher
no flags Details
Additional patch to fix the get/set sparse issue. (1012 bytes, patch)
2010-12-20 12:28 UTC, Jeremy Allison
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Björn Jacke 2010-11-24 07:07:42 UTC
windows saves sparseness information in a sparse bit. we should do the same and not determine sparsenedd from allocation info. Patches will follow ...
Comment 1 Björn Jacke 2010-12-16 17:28:23 UTC
Created attachment 6139 [details]
0001-s3-add-FILE_ATTRIBUTE_SPARSE-to-get_stat_dos_flags.patch
Comment 2 Björn Jacke 2010-12-16 17:28:44 UTC
Created attachment 6140 [details]
0002-s3-remove-set_sparse_flag.patch
Comment 3 Björn Jacke 2010-12-16 17:29:03 UTC
Created attachment 6141 [details]
0003-s3-vfs_gpfs-map-SPARSE-attribute-to-from-WINATTRs-to.patch
Comment 4 Björn Jacke 2010-12-16 17:29:19 UTC
Created attachment 6142 [details]
0004-s3-smbd-add-file_set_sparse-function.patch
Comment 5 Björn Jacke 2010-12-16 17:29:35 UTC
Created attachment 6143 [details]
0005-s3-smbd-don-t-filter-attribs-in-g-set_ea_dos_attribu.patch
Comment 6 Björn Jacke 2010-12-16 17:29:52 UTC
Created attachment 6144 [details]
0006-s3-smbd-implement-FSCTL_SET_SPARSE-more-correctly.patch
Comment 7 Björn Jacke 2010-12-16 17:30:13 UTC
Created attachment 6145 [details]
0007-s3-don-t-do-strict-allocate-with-sparse-files-minor-.patch
Comment 8 Björn Jacke 2010-12-16 17:30:31 UTC
Created attachment 6146 [details]
0008-s3-don-t-do-strict-allocate-with-sparse-files-minor-.patch
Comment 9 Björn Jacke 2010-12-16 17:36:19 UTC
Jeremy, Patch 1-6 should be fine for upstream, to make the thing more complete, #7 and #8 need some more work. Do you have an idea how we can get the sparseness info here without doing another call out to the filesystem?
Comment 10 Jeremy Allison 2010-12-16 18:48:24 UTC
I'm going to push 1-6, with a couple of additional patches on top to make file_set_sparse() a handle based call (which it clearly is).
I'll look more closely at #7 and #8 later.
Jeremy.
Comment 11 Jeremy Allison 2010-12-17 11:42:17 UTC
These patches cause the samba3.posix_s3.raw.open test to fail. I'm looking at it.
Jeremy.
Comment 12 Jeremy Allison 2010-12-17 14:26:39 UTC
Created attachment 6147 [details]
Replacement for 0005
Comment 13 Jeremy Allison 2010-12-17 14:27:13 UTC
Created attachment 6148 [details]
0007-file_set_sparse-needs-to-be-a-handle-based-call.patch
Comment 14 Jeremy Allison 2010-12-17 14:27:30 UTC
Created attachment 6149 [details]
0008-Move-checks-inside-file_set_sparse-to-allow-it-to-be.patch
Comment 15 Stefan Metzmacher 2010-12-18 01:51:34 UTC
Comment on attachment 6147 [details]
Replacement for 0005

Hi Jeremy,

I think that patch is wrong, as the SPARSE flag
needs to be ignored in the setfileinfo and setpathinfo calls.
Comment 16 Stefan Metzmacher 2010-12-18 01:59:10 UTC
Created attachment 6150 [details]
incomplete torture tests

This is not complete, but demonstrates that FSCTL_SET_SPARSE
is the only call that can change the FILE_ATTRIBUTE_SPARSE,
setfileinfo and setpathinfo ignore this flag. But
getfileinfo and getpathinfo show it.

That's why the FILE_ATTRIBUTE_SPARSE flag just indicates that any
stream of the file is sparse and FSCTL_SET_SPARSE works on the
handle to the specific stream.
Comment 17 Jeremy Allison 2010-12-20 12:28:23 UTC
Created attachment 6155 [details]
Additional patch to fix the get/set sparse issue.

You're right metze - this is what I've added to fix that issue.
Jeremy.
Comment 18 Björn Jacke 2011-03-01 05:36:04 UTC
looks like this is in master and in 3.6. Looks like we can close this one now. Jeremy: if I'm wrong, please reopen and comment it :-)