Bug 11473 - Contribution: WSD and LLMNR support
Contribution: WSD and LLMNR support
Status: ASSIGNED
Product: Samba 4.1 and newer
Classification: Unclassified
Component: Other
unspecified
All All
: P5 normal
: ---
Assigned To: Jeremy Allison
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-29 22:21 UTC by Jose M. Prieto
Modified: 2016-05-24 09:12 UTC (History)
3 users (show)

See Also:


Attachments
patches for wsd and llmnr support (56.59 KB, application/x-bzip2)
2015-08-29 22:21 UTC, Jose M. Prieto
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jose M. Prieto 2015-08-29 22:21:18 UTC
Created attachment 11380 [details]
patches for wsd and llmnr support

Dear Samba Team,

The attached collection of patches based on branch "v4-2-test" will add support for Web Services for Devices (WSD) and also a LLMNR service to Samba4. As you may know, those services are nice-to-have features that will allow any Samba4 server to publish/unpublish itself into modern Windows networks and hence a potential candidate for replacing the old NetBios.

I was able to compile and test the contribution into my Linux box making my Samba4 server to publish (on start) and unpublish (on stop) itself instantly on the Windows network neighborhood. 

Code is based on initial work of my colleague Tobias Waldvogel which I adapted and enhanced a bit for Samba. Despite the set of patches attached to the ticket, the whole work can be clone from my GIT repository at:

https://priejos@bitbucket.org/priejos/samba.git

You can pull from tag "wsdd-v0_2".

I hope you can take a look at the work. We both are open to questions if any.

Cheers.
Jose M. Prieto
Comment 1 Jeremy Allison 2015-08-31 17:20:47 UTC
Wow - this looks *great* ! Thanks ! As this is such a significant amount of work can you send in the email detailed here:

https://www.samba.org/samba/devel/copyright-policy.html

so we can start working on looking to integrate this ?

Cheers,

Jeremy.
Comment 2 Jose M. Prieto 2015-08-31 21:38:03 UTC
Hi,

Thanks for the compliment. 

I just sent DCO policy to contributing@samba.org with my sign-off email address (priejosm@hotmail.com). However since this has been a collaboration work by Tobias and myself, should Tobias also send such an email or is it enough with mine? All attached patches are signed off by myself only since Tobias wrote an initial version for Linux and I adapted and integrated to Samba code.

Cheers.
Jose M. Prieto
Comment 3 Jeremy Allison 2015-08-31 21:48:01 UTC
Just for safety's sake if Tobias has (C) on any of the work it's probably better for him to send one in too, so we have clear authority to merge in this code.

Thanks !

Jeremy.
Comment 4 Tobias Waldvogel 2015-09-01 07:19:16 UTC
(In reply to Jeremy Allison from comment #3)
Hi Jeremy,

I am happy to support this project and I just sent the DCO from my email account <my name>@trustanywhere.com

Thanks a lot to Jose M. for all the enhancements and all the work to integrate it with Samba and make it platform independent.

Cheers,
Tobias
Comment 5 Jeremy Allison 2016-01-14 00:56:17 UTC
Just want to say I haven't forgotten about this. Might end up making it a standalone service rather than part of the samba binary, but this is still on my TODO list.

Thanks,

Jeremy.
Comment 6 Jose M. Prieto 2016-01-19 22:05:48 UTC
Hi Jeremy,

Thanks for the info.

Other than this, please feel free to go to my git repository at bitbucket.org where you can find latest commits including the migration to samba 4.3 test branch:

https://priejos@bitbucket.org/priejos/samba.git

Please let us know if any question.

Cheers.
Jose M. Prieto
Comment 7 Jose M. Prieto 2016-05-20 16:23:56 UTC
Hello Team,

I am thinking on refactoring a bit wsdd service by doing something similar to winbindd service. In other words, if samba is set up as a domain member for instance you can run wsdd as standalone service or be a child of "samba" process in case of a ad dc controller.

I was wondering whether you can share any skeleton and/or docs I can use to start this refactoring work instead of putting extra effort of reverse engineering services like winbindd.

Thanks
Jose M. Prieto
Comment 8 Stefan Metzmacher 2016-05-23 08:08:00 UTC
(In reply to Jose M. Prieto from comment #7)

Hi Jose,

first thank you very much for your contribution!
It would be very nice to have these protocols implemented in Samba.

I have a few questions/comments:

- You may avoid the custom debug class for now
  and use new common debug levels we have in master now:

/*
 * Debug levels matching RFC 3164
 */
#define DBGLVL_ERR       0      /* error conditions */
#define DBGLVL_WARNING   1      /* warning conditions */
#define DBGLVL_NOTICE    3      /* normal, but significant, condition */
#define DBGLVL_INFO      5      /* informational message */
#define DBGLVL_DEBUG    10      /* debug-level message */

#define DBG_ERR(...)            DBG_PREFIX(DBGLVL_ERR,          (__VA_ARGS__))
#define DBG_WARNING(...)        DBG_PREFIX(DBGLVL_WARNING,      (__VA_ARGS__))
#define DBG_NOTICE(...)         DBG_PREFIX(DBGLVL_NOTICE,       (__VA_ARGS__))
#define DBG_INFO(...)           DBG_PREFIX(DBGLVL_INFO,         (__VA_ARGS__))
#define DBG_DEBUG(...)          DBG_PREFIX(DBGLVL_DEBUG,        (__VA_ARGS__))

- Is the LLMNR service useful on it's own?
  If so it would be nice to have that separate
  As the PDUs are similar to the once we use for NBNS and DNS
  I think we should try to reuse some of librpc/idl/nbt.idl
  and librpc/idl/dns.idl. We may need an llmnr.idl.

- It would be nice to have at least some basic tests for llmnr,
  similar to the tests in python/samba/tests/dns.py.

With the above the LLMNR part could be merged into master easily.

- The yxml code seems to autogenerated, we need the source of
  it in order to regenerate it.

- You should use talloc_zero() and more allocation failure
  checks in xmlns.c

- The xml related code will need a deep review and unittests
  exploring corner cases in order to avoid security problems.

- Would it be possible that you avoid the socket_send() and socket_recv()
  functions and use the tstream abstraction instead? (Your're already using
  the tdgram abstraction in the LLMNR code)

- Is WSD only based on HTTP (via TCP)?
  If so would it make sense to reuse some of source4/web_server/
  and implement WSD in python?
  Just as an idea, using xml in C always scares me..

- We also need some basic WSD tests, this is needed in order to avoid
  trivial regressions. I guess using python for the tests would be the easiest
  way.

Regarding the winbindd-like redesign: we just have examples in the existing deamons: nmbd, smbd, winbindd and samba.
I'd propose to get this code in with the current design as subprocesses of
'samba', but as two of them one llmnr and one wsd. Splitting them into little
standalone daemons can be done as last step.
Comment 9 Jose M. Prieto 2016-05-23 19:30:05 UTC
Hi,

Thanks for your inputs. 

Starting by the end, yes I agree it makes much sense to have two separated Samba services, one for LLMNR and another one for WSD that runs more or less autonomously of each other but always under samba control. Eventually I guess it will be nothing else than take same skeleton/pattern based on nmbd, winbindd or smbd for both. I realized indeed that as long as you use your Samba server not as AD DC -ie it won't run out of "samba"- my service won't work :(( ... So I will look at this and explore the possibilities.

Regarding the dbg class, the simple reason I did create a new class was for easing my life when debugging tracing the services. I will go back to standard dbg classes. This is not a problem for me.

Regarding rest of your inputs/questions:

>- Is the LLMNR service useful on it's own?
>  If so it would be nice to have that separate
>  As the PDUs are similar to the once we use for NBNS and DNS
>  I think we should try to reuse some of librpc/idl/nbt.idl
>  and librpc/idl/dns.idl. We may need an llmnr.idl.

The idea behind LLMNR is that Windows can discover Samba servers even though DNS is not available or in other words it will be like activate "network discovery" on Windows. Actually LLMNR and DNS PDUs are *exactly* the same with the specifics mentioned on RFC 4795. In fact LLMNR is a link-local scoped name resolution system based on multicast. The LLMNR service implemented in this contribution is only authoritative to answer queries involving "local" IP addresses of the Samba server. 

Apart from this, I am not really familiar with IDL being quite frankly so it is very hard to judge the benefits of including IDL. However if you believe it will make any better, I am fine.

>- It would be nice to have at least some basic tests for llmnr,
>  similar to the tests in python/samba/tests/dns.py.
>
>- We also need some basic WSD tests, this is needed in order to avoid
>  trivial regressions. I guess using python for the tests would be the easiest
>  way.

Let me answer these ones all together.
Ok but I am not familiar with the Samba unit testing framework either. So I will rely on your advice on this. I would also need to go a bit inside this python-based unit testing framework to be able to write them.

>- The yxml code seems to autogenerated, we need the source of
>  it in order to regenerate it.
>
>- The xml related code will need a deep review and unittests
>  exploring corner cases in order to avoid security problems.

Let me answer these ones all together as well.
Yeap, you're right. YXML seems to be auto-generated but I have no clue where it comes from (seems to be some kind of flex/bison-like stuff since it seems to implement a state machine). Anyway, while implementing the WSD service I realized it would be great to have a small XML library to have a very basic but powerful XML parser of incoming WSD XML messages and this is why I picked YXML: it is open source, free, small, simple and does the work. However, I am not tight or bound to it. We can think at this stage whether to stay or not with it. Nonetheless, think of we will need any kind of XML parsing mechanism to able to process XML messages (WSD is eventually at XML-based web service).

Just for your reference this is where I found YXML:
    https://dev.yorhel.nl/yxml

>- You should use talloc_zero() and more allocation failure
>  checks in xmlns.c

xmlns.c was a quick repo of helper functions I wrote to support XML namespaces and all stuff it implies (e.g. scoping, etc). I think I was using talloc everywhere as a memory allocator library but I am very open and willing to improvements if you see any weakness/potential issue in the code -there will for sure :)-

>- Would it be possible that you avoid the socket_send() and socket_recv()
>  functions and use the tstream abstraction instead? (Your're already using
>  the tdgram abstraction in the LLMNR code)

Quite honestly I can't recall right now why I used socket_send and socket_recv instead of tstream :( Anyway, as before I am very open and willing to improve the code as much as possible for which I hope I will get your support (your are more familiar with tstream).

>- Is WSD only based on HTTP (via TCP)?
>  If so would it make sense to reuse some of source4/web_server/
>  and implement WSD in python?
>  Just as an idea, using xml in C always scares me..

No, WSD service is not only serving HTTP requests over TCP but also generating/handling multicast traffic over UDP for anouncing itself as an available service. More details about the WSD message exchange pattern is available at:

    https://msdn.microsoft.com/en-us/library/windows/desktop/bb513677(v=vs.85).aspx

So I couldn't simply -and fully- rely on source4/web_server. I did need something else. I could not find any support for multicast within tdgram library so this is why I needed to write all multicast helper functions by myself and not simply relied on the high level API functions. In this sense, let me ask you then whether there is any plan to support multicast within tdgram?

Cheers.
Jose M. Prieto
Comment 10 Andreas Schneider 2016-05-24 06:25:49 UTC
For writing unit tests you can use https://cmocka.org/ it can print subunit output which is needed for the Samba test framework to parse results. We plan to use it for DNS tests too.