Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Non-nullable extension types #40554

Closed
gsakkis opened this issue Mar 21, 2021 · 5 comments
Closed

ENH: Non-nullable extension types #40554

gsakkis opened this issue Mar 21, 2021 · 5 comments
Labels
Closing Candidate May be closeable, needs more eyeballs Enhancement ExtensionArray Extending pandas with custom dtypes or arrays. Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments

@gsakkis
Copy link

gsakkis commented Mar 21, 2021

As far as I can tell, all custom extension types are nullable. It would be useful to provide a way to create non-nullable extensions. One way to support this could be by setting the na_value attribute to NotImplemented. Thoughts?

@gsakkis gsakkis added Enhancement Needs Triage Issue that has not been reviewed by a pandas team member labels Mar 21, 2021
@dsaxton
Copy link
Member

dsaxton commented Mar 21, 2021

Sounds like you want something like a database constraint but for DataFrames? That's an interesting idea, although I suspect this might be easiest to accomplish in end-user code by simply asserting that a given column contains no null values, rather than creating whole new data types.

@dsaxton dsaxton added Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Mar 21, 2021
@gsakkis
Copy link
Author

gsakkis commented Mar 22, 2021

IMO nullability is part of the type definition, it's not some extra constraint to be tested by the end-user code. Checking the constraint in the extension code itself (e.g. in __init__ and/or _from_sequence) would be an improvement but even that is not sufficient for my use case. Ideally nullability should be an inspectable property of the type, discoverable by a new pd.api.types.is_nullable_dtype(arr_or_dtype) function.

@jbrockmendel
Copy link
Member

xref #40574

@lithomas1 lithomas1 added the ExtensionArray Extending pandas with custom dtypes or arrays. label May 25, 2021
@jbrockmendel
Copy link
Member

A 3rd party EA could just raise if any NA-like is passed to the relevant constructor or __setitem__. I guess we could document this? Otherwise not clear that this is actionable.

@jbrockmendel jbrockmendel added the Closing Candidate May be closeable, needs more eyeballs label Jul 27, 2023
@mroeschke
Copy link
Member

Looks like there's not much support for this feature from the core team so closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Closing Candidate May be closeable, needs more eyeballs Enhancement ExtensionArray Extending pandas with custom dtypes or arrays. Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Projects
None yet
Development

No branches or pull requests

5 participants