A request for sorting on the attribute displayName is denied by the server. Outlook use this kind of request when connected to a samba server (when it announce that it supports the following 1.2.840.113556.1.4.800 capability, which is needed to make adcu work in windows server 2008).
Created attachment 4189 [details] Tcpdump capture of an LDAP exchange
The attachement located here :https://bugzilla.samba.org/attachment.cgi?id=4188 Show a similar request made when the capability is removed from the server, the request is successful because it's on attribute CN (instead of displayName).
After more investigation it turns out that it works. The problem was created by the fact that groups or users do not have everytime the displayName attribute and it's not a filter parameter of the request but it's used as a sort key (strange logic ...).