Bug 13748 - pytalloc's tp_compare/tp_richcompare functions should return only -1, 0 or 1
Summary: pytalloc's tp_compare/tp_richcompare functions should return only -1, 0 or 1
Status: NEW
Alias: None
Product: TALLOC
Classification: Unclassified
Component: libtalloc (show other bugs)
Version: unspecified
Hardware: All All
: P5 normal
Target Milestone: ---
Assignee: Simo Sorce
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-01-16 11:33 UTC by Adam Sampson
Modified: 2019-01-16 11:33 UTC (History)
0 users

See Also:


Attachments
Patch: use a wrapper function to limit the returned value (1.70 KB, patch)
2019-01-16 11:33 UTC, Adam Sampson
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Adam Sampson 2019-01-16 11:33:25 UTC
Created attachment 14774 [details]
Patch: use a wrapper function to limit the returned value

In talloc 21.1.15 and earlier versions, pytalloc_default_cmp and pytalloc_base_default_cmp implement Python comparison functions for talloc objects. These functions return the difference between two pointers.

This is wrong for a couple of reasons:

- These functions are meant to return only -1, 0 or 1 (see https://docs.python.org/2/c-api/typeobj.html). Recent versions of the Python runtime system enforce this and produce a warning -- "tp_compare didn't return -1, 0 or 1" -- when they're used.

- The return value is an int, which is smaller than ptrdiff_t on some platforms.

I've attached a patch which fixes this using a wrapper function.