Bug 14142 - "interfaces" extra data for RSS/RDMA/speed not parsed correctly
Summary: "interfaces" extra data for RSS/RDMA/speed not parsed correctly
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: File services (show other bugs)
Version: 4.10.8
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Samba QA Contact
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-09-26 08:28 UTC by Steven Noonan
Modified: 2020-07-24 15:50 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Steven Noonan 2019-09-26 08:28:28 UTC
From the smb.conf man page:

---
In order to support SMB3 multi-channel configurations, smbd understands some extra data that can be appended after the actual interface with this extended syntax:

interface[;key1=value1[,key2=value2[...]]]

Known keys are speed, capability, and if_index. Speed is specified in bits per second. Known capabilities are RSS and RDMA. The if_index should be used with care: the values must not coincide with indexes used by the kernel. Note that these options are mainly intended for testing and development rather than for production use. At least on Linux systems, these values should be auto-detected, but the settings can serve as last a resort when autodetection is not working or is not available.
---

This feature is currently broken. The interfaces line is parsed as type P_CMDLIST. This type treats comma, semicolon, and whitespace as token delimiters. A line like this:

interfaces = eth0;capability=RSS,speed=10000000000

effectively gets treated like this:

interfaces = [ 'eth0', 'capability=RSS', 'speed=10000000000' ]

So the extra properties get treated as interface names but since it can't match them to anything, it silently drops them on the floor. (One of the higher log levels does mention that they couldn't be matched to anything.)

The code to parse the extra properties is all in there, but it is non-functional due to how the tokens are separated before being handled by interpret_interface().

The simplest fix would probably be to use different delimiters than semicolon and comma. I chose pipe for my use case:

https://git.uplinklabs.net/steven/projects/archlinux/misc-packages.git/tree/samba/0001-interface-treat-as-separator-for-parameters.patch?id=98dfa667c37d7132dcfb5471c7da3a96c2dea337

e.g. "interfaces = lan0|capability=RSS|speed=10000000000"

Comma would be ambiguous in this context anyway, as I know I've seen smb.conf files with e.g. "interfaces = eth0,eth1" and such. Maybe the pipe isn't the best replacement for delimiting the properties, but it worked for me.


Another issue with handling these extra properties, delimiter issues aside, is that they would be ignored when appended to named interfaces, e.g. "interfaces = eth0;capability...". The properties only correctly apply where IP addresses are used instead. So I patched that too:

https://git.uplinklabs.net/steven/projects/archlinux/misc-packages.git/tree/samba/0002-interface-allow-overriding-speed-cap-if_index-for-na.patch?id=98dfa667c37d7132dcfb5471c7da3a96c2dea337


I'm also considering writing a patch to auto-detect the "RSS" capability, because it currently never set unless explicitly specified in the "interfaces" line (which isn't possible right now due to the delimiter issue). Detecting it would probably be as simple as looking in /sys/class/net/$interfacename/queues and seeing if there is more than one directory starting with "rx-". We could also populate the field currently marked as "reserved" in fsctl_net_iface_info with the number of RSS queues we found. According to Wireshark, the "reserved" field is used to encode the RSS queue count, though filling it is apparently not necessary to make Windows clients successfully negotiate multi-channel sessions.
Comment 1 Stefan Metzmacher 2019-09-26 08:45:02 UTC
For me the following line seemed to work (everything in one line):

interfaces = "172.31.9.162;if_index=1,capability=RSS,speed=100000" "172.31.9.62;if_index=2,capability=RSS,capability=RDMA"

From your patches the 2nd looks promising I'll test that.
Comment 2 Steven Noonan 2019-09-26 09:03:00 UTC
Wow. I'd been scouring the code and didn't realize it was even possible to quote tokens like that.

I'm working on the RSS auto-detection patch to make adding these parameters kind of redundant anyway (link speed is already detected, RSS isn't).
Comment 3 Stefan Metzmacher 2019-09-26 09:25:36 UTC
(In reply to Steven Noonan from comment #2)

I was also wondering and needed a while to find that out.
Now I have a template in my test vms and only have to copy
it from there.

Thanks for taking a look at the auto detection.

But we really need to make this simpler to use...
Comment 4 Steven Noonan 2019-09-26 10:06:55 UTC
OK, here's how we can detect RSS support (and RSS queue count) from a physical interface:

https://git.uplinklabs.net/steven/projects/archlinux/misc-packages.git/plain/samba/0002-interfaces-teach-how-to-identify-interfaces-with-RSS.patch

This does not work with non-physical interfaces (e.g. bridges), though. I think it's probably reasonable for users to manually turn on RSS in smb.conf for those kinds of interfaces.


And we can include the RSS queue count in the reserved field (which Wireshark says is a 32-bit integer representing the RSS queue count):

https://git.uplinklabs.net/steven/projects/archlinux/misc-packages.git/plain/samba/0003-interfaces-add-RSS-queue-count-to-FSCTL_QUERY_NETWOR.patch

I tested this out on my network and my config of just "interfaces = lan0" correctly identifies RSS support and the queue count without any trouble.
Comment 5 Steven Noonan 2019-09-26 10:47:03 UTC
Actually, while my previous patch works fine, a better way is probably just to use the ethtool API like we do for discovering link speed:

https://git.uplinklabs.net/steven/projects/archlinux/misc-packages.git/plain/samba/0002-interfaces-teach-how-to-identify-interfaces-with-RSS.patch?id=64ce9506265d6acd0cc79a88e176879d316853a5

Both approaches only work on physical interfaces, so they're functionally the same. But the SIOCETHTOOL approach is easier to read (also cheaper).
Comment 6 Stefan Metzmacher 2020-07-24 15:50:51 UTC
This is fixed with afd3bd01eb463081cb85724260a0840cd98b38c2 in 4.13.0rc1