-
-
Notifications
You must be signed in to change notification settings - Fork 18.7k
Description
We need to better describe the exact semantics of _ndarray_values
: what is it expected to return and how it is used.
Currenlty it is defined on the ExtensionArray, but mentioned it is not part of the "official" interface:
pandas/pandas/core/arrays/base.py
Lines 687 to 697 in 712fa94
@property | |
def _ndarray_values(self): | |
# type: () -> np.ndarray | |
"""Internal pandas method for lossy conversion to a NumPy ndarray. | |
This method is not part of the pandas interface. | |
The expectation is that this is cheap to compute, and is primarily | |
used for interacting with our indexers. | |
""" | |
return np.array(self) |
One Series/Index, the property will either give you what EA._ndarray_values
gives, or the underlying ndarray:
Lines 768 to 780 in 712fa94
@property | |
def _ndarray_values(self): | |
# type: () -> np.ndarray | |
"""The data as an ndarray, possibly losing information. | |
The expectation is that this is cheap to compute, and is primarily | |
used for interacting with our indexers. | |
- categorical -> codes | |
""" | |
if is_extension_array_dtype(self): | |
return self.values._ndarray_values | |
return self.values |
What it currently is for the EAs:
- Categorical: integer codes
- IntegerArray: the integer
_data
, so but losing any information about missing values - PeriodArray: the integer ordinals
- IntervalIndex: object array of Interval objects
For what it is currently used (this needs to be better looked at, copying now from #19954 (comment), quoting Tom here):
- Index.itemsize (deprecated)
- Index.strides (deprecated)
- Index._engine
- Index set ops
- Index.insert
- DatetimeIndex.unique
- MultiIndex.equals
- pytables._convert_index (shared across integer and period)
There are a few other uses (mostly datetime / timedelta / period) that could maybe uses asi8 instead. I'm not familiar enough with indexing to know whether that can operate on something other than ndarrays. In theory, EAs can implement the buffer protocol, which would get the data to cython. But I don't know what ops would be required when we're down there.