The performance using aes-gcm for SMB3 encryption is very poor compared to aes-ccm. (Using our software implenentation is both cases). Callgrind showed that we use 28,165,720,719 cpu cycles to deliver a 100MB file to a client using aes-ccm. With aes-gcm this is raises up to 723,094,413,831 cpu cycles. As a reference copying without signing or encryption uses just 134,264,902 cpu cycles. And signing with aes-cmac uses 13,321,036,015 cycles. While SMB2 signing with sha256 uses 14,124,659,113 cycles. We should optimize the aes-{cmac,ccm,gcm} software implementions. The most problematic parts are the xor and right- and leftshift implementations. As it seems that software aes-gcm will still be much slower than software aes-ccm, we should disable aes-gcm support for 4.3.0.
The real solution will be usage of hardware encryption (aesni) on modern CPUs. See bug #11286.
Created attachment 11346 [details] Patches for v4-3-test (part1)
Pushed to autobuild-v4-3-test
Pushed part1 to v4-3-test
Created attachment 11374 [details] Patches for v4-3-test (part2)
Comment on attachment 11374 [details] Patches for v4-3-test (part2) LGTM.
Re-assigning to Karolin for inclusion in 4.3.0.
Part2 pushed to v4-3-test.