Some options in `pyproject.toml` are ignored

[jorenham]

repost from scipy/scipy-stubs#261 by @miloth:

I found the following issue while investigating the task-runners. Running mdformat . --check will print files in .venv and .tox, even though there are settings to exclude them:

[tool.mdformat]
end_of_line = "lf"
exclude = [".tox/**", ".venv/**", "**/*.m[!d]", "**/*.[!m]d"]

[csala]

Thanks for reporting this @jorenham

I'm not sure about why this would fail in pyproject.toml but work in the original .mdformat.toml file 🤔

Let me explore a bit and try to find out why and where it happens. Or, if you already have a fix, PR will be welcome, OC!

[jorenham]

I wasn't able to figure out what was causing this, and instead submitted a couple of unrelated neurotic PR's 🤷🏻

[jorenham]

@miloth in the mdformat docs I noticed something suspicious:

The --exclude option is only available on Python 3.13+.

So @miloth, did you confirm that the exclude option actually works with the native config file on python < 3.13? And when you concluded that exclude in a pyproject.toml didn't properly work, were you using python>=3.13?

[miloth]

If you have the exclude option in .mdformat.toml on an env which is <3.13, you get the following error:

Error: 'exclude' patterns are only available on Python 3.13+.

Please remove the 'exclude' list from your .mdformat.toml or upgrade Python
version.

If the same settings are in pyproject.toml with this plugin, then it behaves as if that flag was ignored.

[jorenham]

If you have the exclude option in .mdformat.toml on an env which is <3.13, you get the following error:

Error: 'exclude' patterns are only available on Python 3.13+.

Please remove the 'exclude' list from your .mdformat.toml or upgrade Python
version.

If the same settings are in pyproject.toml with this plugin, then it behaves as if that flag was ignored.

Ahh I see.
So in a sense mdformat-pyproject is actually more forgiving 🤔.

IMHO raising an error is a bit too harsh in this situation; I think a warning would be more appropriate.

But I agree that that mdformat-pyproject shouldn't silently let this pass by on python<3.13. I suppose that this is something that could be worked around within mdformat-pyproject, but I actually think that this is a bug in mdformat itself. Because all that this project does (and should do), is parsing the pyproject.toml and passing the relevant config to mdformat.

[miloth]

Looking at some trials, it appears that all of the settings in pyproject.toml are ignored in scipy-stubs, like the plugin is not injecting them. Both cli_opts and toml_opts are empty in the actual run of mdformat . with the above mentioned config in pyproject.toml.

[jorenham]

Looking at some trials, it appears that all of the settings in pyproject.toml are ignored in scipy-stubs, like the plugin is not injecting them. Both cli_opts and toml_opts are empty in the actual run of mdformat . with the above mentioned config in pyproject.toml.

Maybe this should use mdit.set or mdit.configure instead?

mdformat-pyproject/mdformat_pyproject/plugin.py

Lines 90 to 100 in 22220f8

	def update_mdit(mdit: markdown_it.MarkdownIt) -> NoReturn:
	"""Read the pyproject.toml file and re-create the mdformat options."""
	mdformat_options = mdit.options["mdformat"]
	file_path = mdformat_options.get("filename", "-")
	pyproject_path = _find_pyproject_toml_path(file_path)
	if pyproject_path:
	pyproject_opts = _parse_pyproject(pyproject_path)
	if pyproject_opts is not None:
	cli_opts = _reload_cli_opts()
	new_options: Mapping = {**pyproject_opts, **cli_opts}
	mdformat_options.update(new_options)

[csala]

I have been unavailable for a few days and I'm just coming back to this.

This is quite puzzling, because the problems is more complex than it seems.

Looking at some trials, it appears that all of the settings in pyproject.toml are ignored in scipy-stubs, like the plugin is not injecting them. Both cli_opts and toml_opts are empty in the actual run of mdformat . with the above mentioned config in pyproject.toml.

So, based on my testing, it is not as simple as pyproject.toml options being ignored, only some of them are!
In particular, out of the default mdformat options, only end_of_line and exclude are ignored. However, wrap and number are not.

I'm now digging into why some are ignored.

[csala]

Here are the conclusions: the problem is not with mdit, which is properly configured by the current process, but with mdformat itself, because some configuration is exclusively theirs and handled outside of mdit.

Some examples of this are end_of_line and exclude, which are the ones used in the referenced scipy-stubs config.

The only alternative I see here is to avoid the update_mdit paradigm and instead exploit the fact that plugins are loaded before the run function is executed, to monkey patch the read_toml_opts function to read from pyproject.toml instead of .mdformat.toml before it is used.