Skip to content

Implement skipna in xr.quantile for speedup #3843

@aaronspring

Description

@aaronspring

xr.quantile uses np.nanquantile which is slower than np.quantile but only needed when ignoring nans is needed. Adding skipna as kwarg would lead to a speedup for many use-cases.

MCVE Code Sample

np.quantile is much faster than np.nanquantile

control = xr.DataArray(np.random.random((50,256,192)),dims=['time','x','y'])
%time _ = control.quantile(dim='time',q=q)
CPU times: user 4.14 s, sys: 61.4 ms, total: 4.2 s
Wall time: 4.3 s


%time _ = np.quantile(control,q,axis=0)
CPU times: user 47.1 ms, sys: 4.27 ms, total: 51.4 ms
Wall time: 52.6 ms

%time _ = np.nanquantile(control,q,axis=0)
CPU times: user 3.18 s, sys: 21.4 ms, total: 3.2 s
Wall time: 3.22 s

Expected Output

faster xr.quantile:

%time _ = control.quantile(dim='time',q=q)
CPU times: user 4.95 s, sys: 34.3 ms, total: 4.98 s
Wall time: 5.88 s

%time _ = control.quantile(dim='time',q=q, skipna=False)
CPU times: user 85.3 ms, sys: 16.7 ms, total: 102 ms
Wall time: 127 ms

Problem Description

np.nanquantile not always needed

Versions

Output of `xr.show_versions()` xr=0.15.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions