The prototype of smbc_open() (and the open member of SMBCCTX) should be int smbc_open(const char *furl, int flags); and the mode parameter should be obtained through va_start/va_arg/va_end, so that people can write fd = smbc_open("foo", O_RDONLY); just like with open()
You're absolutely right that that's what the prototype should have been. At this point, though, I don't want to change it. Although your suggested change should be backward compatible, I prefer not to make interface changes when this has so many existing users already.
That wouldn't change the API at all, so existing code would just compile fine. That said, in the meanwhile, I realized that on some architectures, the ABI changes between functions with static number of parameters and functions with variable number of parameter. _That_ would break existing _binaries_.