Bug 8210 - Performance measurement on SMBTA
Summary: Performance measurement on SMBTA
Alias: None
Product: smbta
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Hardware: All All
: P5 normal
Target Milestone: ---
Assignee: Holger Hetterich
QA Contact:
Depends on:
Blocks: 8204
  Show dependency treegraph
Reported: 2011-06-07 09:17 UTC by Holger Hetterich
Modified: 2012-05-11 10:31 UTC (History)
5 users (show)

See Also:

run_tests.sh (947 bytes, application/x-shellscript)
2011-09-07 10:32 UTC, Robert Piasek (dead mail address)
no flags Details
download.xml (349 bytes, text/xml)
2011-09-07 10:36 UTC, Robert Piasek (dead mail address)
no flags Details
install.sh (208 bytes, application/x-shellscript)
2011-09-07 10:36 UTC, Robert Piasek (dead mail address)
no flags Details
test-definition.xml (1010 bytes, text/xml)
2011-09-07 10:36 UTC, Robert Piasek (dead mail address)
no flags Details
results-definition.xml (366 bytes, text/xml)
2011-09-07 10:36 UTC, Robert Piasek (dead mail address)
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Holger Hetterich 2011-06-07 09:17:04 UTC
One of the questions that have been raised at SambaXP is the one about Performance of SMBTA. During the presentation, I had to admit that for the new cache algorithm in smbtad and the DBI interface change, we have no performance measurement done. So I think we could create a test environment, and run a test with SMBTA activated and the other time without, on the pure Samba server.

We would do the test several times, and log the times w/ and w/o SMBTA. We would then produce a chapter in the documentation about the performance penalty that is created by SMBTA, shortly describe the test environment, having the numbers in a table, and finally create a finishing statement like "Overall, there is a performance penalty of X % in this test environment, compared to Samba w/o SMBTA."
Comment 1 Holger Hetterich 2011-06-07 09:18:15 UTC
Robert, I had the idea you could do that :)

Feel free to reassign it to me if you don't have time or nerves to do such a test.
Comment 2 Robert Piasek (dead mail address) 2011-06-07 13:15:28 UTC
I will gladly do some analysis on smbtad impact.

Test hardware:

CPU: Intel(R) Core(TM) i7 CPU 860  @ 2.80GHz
MemTotal:       16431148 kB

Model Family:     SandForce Driven SSDs
Device Model:     OCZ-VERTEX2 3.5
Serial Number:    OCZ-L9PA9UZ323139DNJ
Firmware Version: 1.23
User Capacity:    120,034,123,776 bytes

Model Family:     Seagate Barracuda 7200.11 family
Device Model:     ST3500320AS
Serial Number:    9QM4RF93
Firmware Version: SD15
User Capacity:    500,107,862,016 bytes

samba version: 3.6.0_rc1
smbtad version: 1.2.5
postgresql version: 9.0.4
kernel version: 2.6.39 x86_64

test files sizes: 128KB, 1MB, 100MB, 10GB

Each test will be performed 100 times and I will use average to calculate final results.

I will do test scenario based on:
1) performance impact on different filesystems (ext4 vs btrfs)
2) performance impact differences between hdd and ssd
3) performance impact if DB server is located on same server vs located on remote server
4) performance impact using small files vs using big files

Please feel free to add to this list
Comment 3 Holger Hetterich 2011-07-05 20:55:47 UTC
any results to post in the documentation as well as somewhere else? I would love to have this for 1.2.6 :)
Comment 4 Holger Hetterich 2011-07-06 17:31:18 UTC
Looks like we have results that are able to being posted in the SMBTA documentation, so I make this blocking the release tracker bug

Robert, did you make sure that we (as in GPLv3) are able to use the Phoronix Logo and graphics created by the Phoronix testsuite?
Comment 6 Robert Piasek (dead mail address) 2011-07-07 09:15:32 UTC

Phoronix test suite is distributed under GPLv3

I asked phoronix creator about it, but didn't get reply yet. I will update this ticket once I've got more information.
Comment 7 Holger Hetterich 2011-07-07 20:30:51 UTC
Robert, I'd also like to point out _how_ the tests have been generated. Can we publish the scripts you have created for the tests and a brief description with the 1.2.6 package?
Comment 8 Holger Hetterich 2011-07-26 20:34:27 UTC
Robert, please see comment#7, to include the graphics with the documentation we would need details on how the tests where performed and what the test is doing. Best would be if you could provide this info in plain text here, so that I can commit this into the documentation. The graphic is perfect, but it doesn't explain "what has been run" to do the test.
Comment 9 Holger Hetterich 2011-08-04 21:54:10 UTC
Comment 10 Holger Hetterich 2011-09-06 20:04:32 UTC
Hi Robert, I'll use the graphics provided in comment#5 for upcoming presentations for SMBTA. I hope you are fine with it. In an ideal world, I would also have information on how the tests where done, and I have some time pressure (4 days). Please also consider comment#7.
Comment 11 Robert Piasek (dead mail address) 2011-09-07 10:30:31 UTC
For each of file sizes, simple copy from/to were carried out 10 times. Average transfer speed were used. It was all automated using Phoronix Test Suite.

To reproduce these tests, you will require the following:
1) Download and install PTS (http://www.phoronix-test-suite.com/)
2) create local profile for specified file size (10MB, 100MB, 1GB etc)
you will require following files (examples attached):
* downloads.xml
* install.sh
* results-definition.xml
* test-definition.xml

3) install and run tests. For convenience, I attached simple script which does it automatically:
* run_tests.sh

In my test scenario, samba+smbtad were installed on one computer and tests were carried out from remote machine.
Comment 12 Robert Piasek (dead mail address) 2011-09-07 10:32:07 UTC
Created attachment 6858 [details]
Comment 13 Robert Piasek (dead mail address) 2011-09-07 10:36:06 UTC
Created attachment 6859 [details]
Comment 14 Robert Piasek (dead mail address) 2011-09-07 10:36:21 UTC
Created attachment 6860 [details]
Comment 15 Robert Piasek (dead mail address) 2011-09-07 10:36:33 UTC
Created attachment 6861 [details]
Comment 16 Robert Piasek (dead mail address) 2011-09-07 10:36:45 UTC
Created attachment 6862 [details]
Comment 17 Holger Hetterich 2012-05-11 10:25:13 UTC

pushed the benchmark utility files into a directory in doc
Comment 18 Holger Hetterich 2012-05-11 10:31:15 UTC
added a chapter in the documentation