Bug 3325 - smbclient fails to extract a 4Gb (possibly any 2gb+) file from a tarball.
Summary: smbclient fails to extract a 4Gb (possibly any 2gb+) file from a tarball.
Status: RESOLVED FIXED
Alias: None
Product: Samba 3.0
Classification: Unclassified
Component: smbclient (show other bugs)
Version: 3.0.9
Hardware: x86 Linux
: P3 normal
Target Milestone: none
Assignee: Samba Bugzilla Account
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-12-14 05:46 UTC by Brad Ellis (mail address dead)
Modified: 2014-09-18 10:11 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Brad Ellis (mail address dead) 2005-12-14 05:46:46 UTC
When trying to restore a file that is 4GB in size from a tarball,
the restore fails with 2gb (exactly) of the file being written to the
host.

command: smbclient //<target ip>/<share> "<password>" -U <user> -Tx <tarball>

error reported:
--
Error writing remote file
Abandoning restore
--

This was tried with the following versions:
Version 3.0.3-5
Version 3.0.20b

Using smbclient and a "put"ting the 4gb file from the tarball works fine.

--

Having had a look at client/clitar.c:

I'm curious if pos,dsize (and possibly bpos) variables be 
off_t's and not ints ?

As ints there is the possibility that they are restricting 
the maximum file size that can be restored from a tarball to 2gb.

I've tried converting them to off_t's, but it seems a little more
complicated than that as these changes result in some 9728 bytes 
extra (total file size of being written to the file 4177683968 bytes),
and a error returned regarding the tarfile.

Extracting the file from the tarball using tar extracts the file successfully.

----
Changes trialed in client/clitar.c - get_file() function:
----
static int get_file(file_info2 finfo)
{
/*      int fnum = -1, pos = 0, dsize = 0, bpos = 0; existing variable
declarations */
/* new declarations */
        int fnum = -1;
        off_t pos = 0, dsize = 0, bpos = 0;

        SMB_BIG_UINT rsize = 0;

        DEBUG(5, ("get_file: file: %s, size %.0f\n", finfo.name,
(double)finfo.size));

        if (ensurepath(finfo.name) &&
                        (fnum=cli_open(cli, finfo.name, O_RDWR|O_CREAT|O_TRUNC,
DENY_NONE)) == -1) {
                DEBUG(0, ("abandoning restore\n"));
                return(False);
        }

        /* read the blocks from the tar file and write to the remote file */

        rsize = finfo.size;  /* This is how much to write */

        while (rsize > 0) {

                /* We can only write up to the end of the buffer */
                dsize = MIN(tbufsiz - (buffer_p - tarbuf) - bpos, 65520); /*
Calculate the size to write */
                dsize = MIN(dsize, rsize);  /* Should be only what is left */
                DEBUG(5, ("writing %i bytes, bpos = %i ...\n", dsize, bpos));

                if (cli_write(cli, fnum, 0, buffer_p + bpos, pos, dsize) !=
dsize) {
                        DEBUG(0, ("Error writing remote file\n"));


-----
cli_write protocol definition from include/proto.h
-----
size_t cli_write(struct cli_state *cli,
                 int fnum, uint16 write_mode,
                 const char *buf, off_t offset, size_t size);

-----
Initially these seemed like they fitted bug #563, but having thought about it more, it seems like they don't. The entries that I've added there could be removed.
Comment 1 Justin Best (mail address dead) 2006-04-28 11:41:48 UTC
I am also experiencing the same issue under Debian Sarge, running smbclient Version 3.0.14a-Debian. A file slightly larger than 2GB is failing to restore (writes exactly 2GB, then quits), whereas files < 2GB are successful. Here is the log file from my program, BackupPC (backuppc.sf.net), which uses smbclient to do its restores:

-------------------
Running: /usr/bin/smbclient \\\\ao-pc102\\c\$ -U DOMAIN\\Administrator -E -N -d 1 -c tarmode\ full -Tx -
Running: /usr/share/backuppc/bin/BackupPC_tarCreate -h ao-pc102 -n 1 -s Outlook -t /Madalyns\ Personal\ Folders.pst
Xfer PIDs are now 1218,1219
Domain=[DOMAIN] OS=[Windows 5.1] Server=[Windows 2000 LAN Manager]
tarmode is now full, system, hidden, noreset, verbose
Error writing remote file
This backup will fail because: Abandoning restore
Abandoning restore
tarCreate: Unable to write to output file (Connection reset by peer)
Restore failed: Abandoning restore
-------------------

This feels really similar to <a href="https://bugzilla.samba.org/show_bug.cgi?id=3592">bug 3592</a> The problem there was that one of the variables was type INT. Changing its type to SMB_BIG_UINT solved the problem. I suspect the issue is the same here.
Comment 2 David Disseldorp 2014-09-18 10:11:56 UTC
This should be fixed with Aurélien's new libarchive smbclient tarmode implementation in master. Which will be released with Samba 4.2.

Please test and reopen if still broken.