PEP 661: Update

[JelleZijlstra]

@taleinat gracefully agreed to let me take over the last part of finishing the PEP.

These changes generally align the PEP with what the SC asked for in https://discuss.python.org/t/pep-661-sentinel-values/9126/234 .

---

📚 Documentation preview 📚: https://pep-previews--4923.org.readthedocs.build/

[Viicos]

Responding to the open questions in JelleZijlstra/cpython#4 (comment)

Should the sentinel class be subclassable? My inclination is no.

Agree with this. Some motivation for being subclassable was provided here:

If the Sentinel class could be subclassed, people could simply create their own subclass with the boolean behaviour they wanted. But because it can’t be, people who don’t like the default have to write their own sentinel class.

But I'd rather have this customizable (see below) from the constructor instead of using subclassing for this to be achieved.

Should there be a C API like PySentinel_New? Probably.

You gave a good example in https://discuss.python.org/t/9126/306, to be able to re-use sentinels in CPython itself. Do you happen to know if it requires little or some work?

What about __bool__? Lots of discussion on Discourse, I don't care too much but don't want this to derail the feature. This draft implementation says sentinels are always truthy.

I would be in favor of not defaulting to True for thuthiness. The meaning of the truth value of a sentinel depends on the context: for MISSING/UNSET sentinels, it would make sense to evaluate the truthiness as False, but I'm pretty sure we could find examples where a sentinel would evaluate to True. And no matters if it evaluates to True/False, doing bool(val) when val is typed as <some_type> | SOME_SENTINEL (or even <some_type> | SOME_SENTINEL | None) can easily lead to logical errors ¹.

On the other hand, some users mentioned customizable bool would add complexity ². I don't think this is really true? It is just a matter of adding a __bool__() method on sentinel. If we want to reuse the sentinel class in the stdlib ³, we may have to preserve the existing truthiness behavior, at least for public sentinels. For instance, typing.NoDefault currently evaluates to True, and I believe it would be preferable this behavior.

So to me it would make sense to provide a bool argument, typed as bool | None, defaulting to None (which means it raises on __bool__() -- customizing the exception is adding unnecessary complexity imo). It means we rely on developers owning their sentinel definition to customize this behavior wisely, but I feel this is fine (the potential logical errors mentioned above aren't that much of an anti-pattern to me).

Should sentinels be weakrefable?

As per the issue python/cpython#106403 you referenced to me, I've tried using cloudpickle on Pydantic models with sentinel instances used as value, and it worked fine, although I'm not sure what was the initial issue with cloudpipe/cloudpickle#507. Maybe it could be added later if cloudpickle maintainers report any issues?

Footnotes

1. good argument in https://discuss.python.org/t/9126/292. ↩

2. "I firmly believe that if (as the SC suggested) configurable truthiness of a sentinel is too much complexity" (https://discuss.python.org/t/9126/278). ↩

3. a non-exhaustive list is available here: https://discuss.python.org/t/9126/299. ↩

[Viicos]

What would be the motivation to make it lowercase? Is it to match existing built-ins that are already all lowercase (and perhaps this is even an established convention)?

[JelleZijlstra]

Yes, all other builtin types are lowercase (other than exception classes).

[Viicos]

As per JelleZijlstra/cpython#4 (comment), I'm not sure this matches the C implementation, as it defaults to builtins?

[JelleZijlstra]

Yeah I think both cases here are questionable, maybe the best bet is to default it to None and make reduce raise if the module is missing.

[willingc]

The text looks good. Thanks @JelleZijlstra.

[JelleZijlstra]

Thanks! I added a proposed C API. I am going to hold off on changing anything related to __bool__ as I hear some decision was made about that offline.

[DanielNoord]

Thanks again for picking this up @JelleZijlstra! Hope my review can help with getting this approved :)

I am going to hold off on changing anything related to bool as I hear some decision was made about that offline.

Is there a way to read about this for those not included in that discussion? 😄

I also left one more comment in the PEP thread (link) that I started to wonder about during review.

[DanielNoord]

Does it? Doesn't it make the naming problem more prominent as you explain in the Backwards Compatibility section?

[JelleZijlstra]

I do think this is accurate. Backwards compatibility is a problem when code that works in 3.x no longer works when run unchanged in 3.x+1. That would be the case if we picked a new library name, because then in the first version that has this PEP, import sentinels (or whatever name we choose) will start importing from the standard library instead of the third-party library. But if we're adding a new builtin, existing code that uses the name sentinel is unaffected; it just can't easily use the new functionality.

[JelleZijlstra]

Is there a way to read about this for those not included in that discussion? 😄

I'll share more as soon as I hear more!