Bug 15691 - subunit returns successful exit code even if tests fail or error
Summary: subunit returns successful exit code even if tests fail or error
Status: NEW
Alias: None
Product: Samba 4.1 and newer
Classification: Unclassified
Component: Test infrastructure (show other bugs)
Version: 4.21.0rc1
Hardware: All All
: P5 regression (vote)
Target Milestone: ---
Assignee: Samba QA Contact
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-08-02 03:06 UTC by Jennifer Sutton
Modified: 2024-08-02 03:06 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jennifer Sutton 2024-08-02 03:06:57 UTC
TestProtocolClient.writeOutcome() removes items from self.errors and self.failures via TestProtocolClient._filterErrors(). This makes wasSuccessful() inappropriately return True even if there were errors or failures. subunit.run.runTests() uses wasSuccessful() to determine the exit code.

To fix this, we should not remove items from self.errors or self.failures, but instead use another method to keep track of how many items we have already processed in each of self.errors and self.failures.

This is a regression introduced by commit 421dc7fc4d83629d3a5f9e558d378f44c7b9dad3.