Bug 15310 - New samba-dcerpc architecture does not scale gracefully
Summary: New samba-dcerpc architecture does not scale gracefully
Status: RESOLVED FIXED
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: DCE-RPCs and pipes (show other bugs)
Version: unspecified
Hardware: All All
: P5 normal (vote)
Target Milestone: ---
Assignee: Jule Anger
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-02-14 15:57 UTC by Volker Lendecke
Modified: 2023-03-09 09:22 UTC (History)
1 user (show)

See Also:


Attachments
Patch (3.79 KB, patch)
2023-03-01 14:55 UTC, Volker Lendecke
no flags Details
git-am fix for 4.18.next (23.12 KB, patch)
2023-03-07 00:31 UTC, Jeremy Allison
vl: review+
Details
git-am fix for 4.17.next (23.13 KB, patch)
2023-03-07 00:39 UTC, Jeremy Allison
vl: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Volker Lendecke 2023-02-14 15:57:14 UTC
We have reports from a large print server that rpcd_spoolss logs messages like

[2023/02/14 07:40:42,  0] ../../source3/rpc_server/spoolss/srv_spoolss_util.c:58(winreg_printer_binding_handle)
  winreg_printer_binding_handle: Could not connect to winreg pipe: NT_STATUS_NETWORK_BUSY

together with samba-dcerpcd in state "D". Highly likely this is due heavy fork() activity from spoolss accessing exclusive winreg services.

We need several steps: First, make the listen queue for named pipes configurable and by default much higher than 5. Second, implement a retry loop after the async connect in np_sock_connect_send fails with EAGAIN or alternatively implement a scheme where we somehow connect to the pipe in a blocking fashion. Possibly via a socketpair with one half sent to samba-dcerpcd via the messaging_send() infrastructure, which has a blocking fallback.
Comment 1 Volker Lendecke 2023-03-01 14:55:15 UTC
Created attachment 17785 [details]
Patch

This is a proposal for a quick workaround. The real fix must involve some async yet deterministic handling of the connect. Possibly by sending one end of a socketpair via messages_dgm.c, which already has a blocking fallback.
Comment 2 Jeremy Allison 2023-03-02 17:40:48 UTC
Comment on attachment 17785 [details]
Patch

Looks good to me as a "quick-fix" for 4.18.0 and below. Happy to help review a more sophisticated fix later.
Comment 4 Samba QA Contact 2023-03-06 22:35:06 UTC
This bug was referenced in samba master:

86e95b57d6848482dc73c624c2e8d2bdb79c1d21
f23eb1b3b728d7a13add8c6614d9992aad2c3653
0ab7b84ccbd383bb2f696ce258438d4af57fe6f0
a1780ed8d1b46e4760319b27a4978e7ce7a1df80
Comment 5 Jeremy Allison 2023-03-07 00:31:04 UTC
Created attachment 17794 [details]
git-am fix for 4.18.next

Cherry-picked from master.
Comment 6 Jeremy Allison 2023-03-07 00:39:32 UTC
Created attachment 17795 [details]
git-am fix for 4.17.next

Cherry-picked and back-ported from master.
Comment 7 Jule Anger 2023-03-07 10:13:36 UTC
Pushed to autobuild-v4-{18,17}-test.
Comment 8 Samba QA Contact 2023-03-07 11:17:05 UTC
This bug was referenced in samba v4-17-test:

529e76a51dfde8218362ce478e852f53553332d7
32a6eb37fbdc3aa31beb27c81908d20ae9e1e3fd
743d7600fba554306356a8254f0ca041308fa332
bfbb854d7462e486775d087efda63906869218ed
Comment 9 Samba QA Contact 2023-03-08 11:17:04 UTC
This bug was referenced in samba v4-18-test:

04e0412a659b2542bc9c6d4a22bc443d8445b0ef
06483b02e9db952a17b0590a3ec8d0fe2d132b3e
501ae551da9c49047a2f67e9fec8732aa32309b1
2030aa02ce0be576ca2180adf7cd30efce329825
Comment 10 Jule Anger 2023-03-08 12:04:35 UTC
Closing out bug report.

Thanks!
Comment 11 Samba QA Contact 2023-03-08 12:29:21 UTC
This bug was referenced in samba v4-18-stable (Release samba-4.18.0):

04e0412a659b2542bc9c6d4a22bc443d8445b0ef
06483b02e9db952a17b0590a3ec8d0fe2d132b3e
501ae551da9c49047a2f67e9fec8732aa32309b1
2030aa02ce0be576ca2180adf7cd30efce329825
Comment 12 Samba QA Contact 2023-03-09 09:22:44 UTC
This bug was referenced in samba v4-17-stable (Release samba-4.17.6):

529e76a51dfde8218362ce478e852f53553332d7
32a6eb37fbdc3aa31beb27c81908d20ae9e1e3fd
743d7600fba554306356a8254f0ca041308fa332
bfbb854d7462e486775d087efda63906869218ed