DEPR: deprecate .ix

[jreback]

enough said.

[frol]

What is the suggested replacement for the deprecated .ix? Is it .loc?

For me .ix works 5-10% faster than .loc:

>>> df.shape
(10000, 211)

>>> df.index
CategoricalIndex(['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
                  ...
                  'C', 'C', 'C', 'C', 'C', 'C', 'C', 'C', 'C', 'C'],
                 categories=['A', 'B', 'C'], ordered=False, dtype='category', length=10000)

>>> df.loc[['C']].shape
(8000, 211)

>>> %timeit df.loc['C']
100 loops, best of 3: 5.61 ms per loop

>>> %timeit df.ix['C']
100 loops, best of 3: 5.37 ms per loop

BTW, passing a list into the indexer adds another 25-50% overhead:

>>> %timeit df.loc['C']
100 loops, best of 3: 5.61 ms per loop

>>> %timeit df.loc[['C']]
100 loops, best of 3: 9.97 ms per loop

>>> %timeit df.ix['C']
100 loops, best of 3: 5.37 ms per loop

>>> %timeit df.ix[['C']]
100 loops, best of 3: 7.57 ms per loop

[jreback]

yes .loc and .iloc are the expected replacements. Timings are expected to eventually be faster, though a single sub-millisecond access difference is pretty meaningless in any real usecase.

[frol]

@jreback Having terabytes of data and processing it with a help of Dask DataFrame which uses Pandas DataFrames as chunks turns "milliseconds" into minutes...

[jreback]

@frol doesn't matter how much data you have. you are almost certainly ineffeciently using indexing operations.

[wesm]

@frol the indexing code paths are going to be rewritten in C/C++ as part of the pandas 2.0 effort, so the microperformance should improve by a factor of 10 or more. Some refactoring or Cythonization may be able to give some quick perf wins in .loc or .iloc

[Liam3851]

Question on .ix deprecation-- suppose you want to set the first row of a DataFrame in a particular column with a value (assume that the index is not an Int64Index). Then you can currently use:

df.ix[0, 'colname'] = 5

In the future can you safely do:

df.iloc[0].loc['colname'] = 5

(this seems to beg for SettingWithCopyWarning)? Or is the only proper option going to be
df.loc[df.index[0], 'colname'] = 5
?

[wesm]

Our experience has been that mixing positional and label indexing has been a significant source of problems for users. Here you might want to do df['colname'][0]

[jreback]

unambigously safe setting (may be better syntactically nicer in 2.0)

df.iloc[0, df.columns.get_loc('colname')] = 5

or

df.loc[df.index[0], 'colname'] = 5

[Liam3851]

@jreback Thanks, makes sense.

[johne13]

@jreback I think you have a typo with square brackets used instead of parens?

df.iloc[0, df.columns.get_loc['colname']] = 5

should be

df.iloc[0, df.columns.get_loc('colname')] = 5