Documentation showing placeholders instead of text

[MicaelJarniac]

Location of the documentation

pandas.DataFrame.mad

Documentation problem

The documentation is showing placeholders instead of the actual text.

{desc}, {axis_descr}, {name1}, {name2}, {see_also}, {examples}

pandas/pandas/core/generic.py

Lines 10890 to 10901 in 3e89b4c

	@doc(
	NDFrame.mad,
	desc="Return the mean absolute deviation of the values "
	"over the requested axis.",
	name1=name1,
	name2=name2,
	axis_descr=axis_descr,
	see_also="",
	examples="",
	)
	def mad(self, axis=None, skipna=None, level=None):
	return NDFrame.mad(self, axis, skipna, level)

[simonjayhawkins]

Thanks @MicaelJarniac for the report. docs rendered correctly in 1.1.5

[MicaelJarniac]

pandas/pandas/core/generic.py

Lines 10491 to 10510 in 2aadaa8

	def mad(self, axis=None, skipna=None, level=None):
	"""
	{desc}
	Parameters
	----------
	axis : {axis_descr}
	Axis for the function to be applied on.
	skipna : bool, default None
	Exclude NA/null values when computing the result.
	level : int or level name, default None
	If the axis is a MultiIndex (hierarchical), count along a
	particular level, collapsing into a {name1}.
	Returns
	-------
	{name1} or {name2} (if level specified)\
	{see_also}\
	{examples}
	"""

I believe this could be related. I'm just poking inside to see if I can find something, but I have absolutely no idea what's going on.

[MicaelJarniac]

While poking inside, I think I found some other places where this is happening:

pandas.DataFrame.tz_convert

pandas/pandas/core/generic.py

Lines 9529 to 9555 in 2aadaa8

	@final
	def tz_convert(
	self: FrameOrSeries, tz, axis=0, level=None, copy: bool_t = True
	) -> FrameOrSeries:
	"""
	Convert tz-aware axis to target time zone.
	Parameters
	----------
	tz : str or tzinfo object
	axis : the axis to convert
	level : int, str, default None
	If axis is a MultiIndex, convert a specific level. Otherwise
	must be None.
	copy : bool, default True
	Also make a copy of the underlying data.
	Returns
	-------
	{klass}
	Object with time zone converted axis.
	Raises
	------
	TypeError
	If the axis is tz-naive.
	"""

[MicaelJarniac]

The documentation for pandas.DataFrame.shift also seems to use the same kind of placeholders, but it renders correctly.

pandas/pandas/core/generic.py

Lines 9133 to 9236 in 2aadaa8

	@doc(klass=_shared_doc_kwargs["klass"])
	def shift(
	self: FrameOrSeries, periods=1, freq=None, axis=0, fill_value=None
	) -> FrameOrSeries:
	"""
	Shift index by desired number of periods with an optional time `freq`.
	When `freq` is not passed, shift the index without realigning the data.
	If `freq` is passed (in this case, the index must be date or datetime,
	or it will raise a `NotImplementedError`), the index will be
	increased using the periods and the `freq`. `freq` can be inferred
	when specified as "infer" as long as either freq or inferred_freq
	attribute is set in the index.
	Parameters
	----------
	periods : int
	Number of periods to shift. Can be positive or negative.
	freq : DateOffset, tseries.offsets, timedelta, or str, optional
	Offset to use from the tseries module or time rule (e.g. 'EOM').
	If `freq` is specified then the index values are shifted but the
	data is not realigned. That is, use `freq` if you would like to
	extend the index when shifting and preserve the original data.
	If `freq` is specified as "infer" then it will be inferred from
	the freq or inferred_freq attributes of the index. If neither of
	those attributes exist, a ValueError is thrown.
	axis : {{0 or 'index', 1 or 'columns', None}}, default None
	Shift direction.
	fill_value : object, optional
	The scalar value to use for newly introduced missing values.
	the default depends on the dtype of `self`.
	For numeric data, ``np.nan`` is used.
	For datetime, timedelta, or period data, etc. :attr:`NaT` is used.
	For extension dtypes, ``self.dtype.na_value`` is used.
	.. versionchanged:: 1.1.0
	Returns
	-------
	{klass}
	Copy of input object, shifted.
	See Also
	--------
	Index.shift : Shift values of Index.
	DatetimeIndex.shift : Shift values of DatetimeIndex.
	PeriodIndex.shift : Shift values of PeriodIndex.
	tshift : Shift the time index, using the index's frequency if
	available.
	Examples
	--------
	>>> df = pd.DataFrame({{"Col1": [10, 20, 15, 30, 45],
	... "Col2": [13, 23, 18, 33, 48],
	... "Col3": [17, 27, 22, 37, 52]}},
	... index=pd.date_range("2020-01-01", "2020-01-05"))
	>>> df
	Col1 Col2 Col3
	2020-01-01 10 13 17
	2020-01-02 20 23 27
	2020-01-03 15 18 22
	2020-01-04 30 33 37
	2020-01-05 45 48 52
	>>> df.shift(periods=3)
	Col1 Col2 Col3
	2020-01-01 NaN NaN NaN
	2020-01-02 NaN NaN NaN
	2020-01-03 NaN NaN NaN
	2020-01-04 10.0 13.0 17.0
	2020-01-05 20.0 23.0 27.0
	>>> df.shift(periods=1, axis="columns")
	Col1 Col2 Col3
	2020-01-01 NaN 10 13
	2020-01-02 NaN 20 23
	2020-01-03 NaN 15 18
	2020-01-04 NaN 30 33
	2020-01-05 NaN 45 48
	>>> df.shift(periods=3, fill_value=0)
	Col1 Col2 Col3
	2020-01-01 0 0 0
	2020-01-02 0 0 0
	2020-01-03 0 0 0
	2020-01-04 10 13 17
	2020-01-05 20 23 27
	>>> df.shift(periods=3, freq="D")
	Col1 Col2 Col3
	2020-01-04 10 13 17
	2020-01-05 20 23 27
	2020-01-06 15 18 22
	2020-01-07 30 33 37
	2020-01-08 45 48 52
	>>> df.shift(periods=3, freq="infer")
	Col1 Col2 Col3
	2020-01-04 10 13 17
	2020-01-05 20 23 27
	2020-01-06 15 18 22
	2020-01-07 30 33 37
	2020-01-08 45 48 52
	"""

It has a decorator, and I believe that might be what's missing on the other ones.

pandas/pandas/core/generic.py

Line 9133 in 2aadaa8

@doc(klass=_shared_doc_kwargs["klass"])

[MicaelJarniac]

take

[MicaelJarniac]

I've managed to find the source of this issue and am now testing my fix.
Basically, when the mad method is created without the @doc(...) decorator, it doesn't get a _docstring_components attribute added to it.

Because of it not having a _docstring_components, when the next mad function is created (the one under the _add_numeric_operations method), it treats the docstring of the previous mad as a finished, plaintext, one, thus ignoring all the templating.

By adding an empty @doc() decorator on top of the first mad method, it gets the _docstring_components attribute added to it, so while generating the docstring for the next mad, it correctly parses the templating from the previous one.

There's a catch, though, and that's where my workaround comes in: simply adding an empty @doc() decorator will instantiate its **params with an empty dict, but since its corresponding docstring contains formatting placeholders, when the docstring gets parsed, it attempts to fill the placeholders using **params, and that results in an error, because it can't find the corresponding keys on the (empty) dict.

I've added a check on this parsing step, that first ensures that **params is not empty before attempting to use it for filling placeholders. If it's empty, the docstring is treated as plaintext.

By doing that, the empty @doc() decorator can now be used on objects with docstrings containing formatting placeholders, and that ensures they also get the _docstring_components attribute added.

[MicaelJarniac]

Well, it works! Took ages to build the documentation...