PIP cache should cache the installed packages as well

[crabhi]

Description:
Currently, setup-python caches only the ~/.cache/pip directory to avoid redownloads. However, it doesn't cache the installed packages. As some package have lengthy installation steps, this leads to delays in builds.

You can see the current behaviour for example in https://github.com/crabhi/setup-python-cache-test/actions/runs/1789016634 (or in attached build.txt) - the pip install output shows "Collecting" and "Installing" instead of "Requirement already satisfied" for all packages.

Justification:
For example installing the ansible package takes well over a minute even if it's already downloaded.

Are you willing to submit a PR?
Yes, I can try.

[nikita-bykov]

Hello @crabhi, thanks for your request!
We will look at it.

[barbieri]

would be also nice to follow https://github.com/actions/cache#outputs and provide an output cache-hit so we can if: steps.[id]. cache-hit != 'true' to avoid calling pip altogether.

[barbieri]

which is done at https://github.com/actions/cache/blob/2d8d0d1c9b41812b6fd3d4ae064360e7d8762c7b/src/utils/actionUtils.ts#L25-L27 and https://github.com/actions/cache/blob/main/src/restore.ts#L55

[jbergstroem]

This pattern affects more languages (actions/setup-node works the same - only caches downloads, not installs) - would love to see a general consensus towards caching installs, not tarballs (perhaps behind a flag/attribute for future compat cache: pip-install).

[barbieri]

for pip I think it makes even more sense as there are no postinstall actions... with setup-node I'm also caching the node_modules instead of the packages, but it "broke" some flows where there was a postinstall script to configure other things (like pre-build some typescript scripts). The solution is simple, just run that script manually (or in my case, cache the built scripts)... but not "one size fits all".

For pip AFAIR there are no postinstall scripts, then this would not be an issue.

[jbergstroem]

For pip AFAIR there are no postinstall scripts, then this would not be an issue.

I'm experimenting with this at the moment and caching site-packages (read: pip output) isn't straightforward either; for instance binary wrappers (black, ..) won't work (python -m black works fine tho). Might be one of thos YMMV cases that makes it hard to standardize for everyone.

[dhvcc]

would be also nice to follow https://github.com/actions/cache#outputs and provide an output cache-hit so we can if: steps.[id]. cache-hit != 'true' to avoid calling pip altogether.

Hey, this feature was merged today and should be a part of the near-future release

[barbieri]

but the cache-hit is just for the packages, not the installation, right? IOW: do I still need to call pip install?

[dhvcc]

but the cache-hit is just for the packages, not the installation, right? IOW: do I still need to call pip install?

Oh you're talking pip. Well yeah, then you'll have to wait for this action to support caching venv's out of the box. It's a case for pipenv and poetry though. The best this for now is to manually cache

[belm0]

I have a case where building packages for pypy (grpcio, grpcio-tools) takes about 6 minutes-- it's way too slow to introduce a matrix.

If anyone has a manual example using actions/cache, please share it.

[rashidnhm]

I was creating a python venv and then caching that directory, however I hit an issue where that was broken once restored (behaviour was inconsistent).

I currently have a job that takes ~6 min to complete, 4 min of which is installation of pip packages. An effective caching of installed packages would be a great boost.

[dhvcc]

I was creating a python venv and then caching that directory, however I hit an issue where that was broken once restored (behaviour was inconsistent).

I currently have a job that takes ~6 min to complete, 4 min of which is installation of pip packages. An effective caching of installed packages would be a great boost.

Could you share the workflow so the people can take a look at it? I think it's possible to hack around while this feature is not here