Take TORCH_HOME env variable into account while setting the cache dir #1741
Conversation
| _TORCH_HOME = os.getenv("TORCH_HOME") | ||
| if _TORCH_HOME is None: | ||
| _TORCH_HOME = "~/.cache/torch" #default | ||
| _CACHE_DIR = os.path.expanduser(os.path.join(_TORCH_HOME, "text")) |
There was a problem hiding this comment.
I think it can be as simple as
_CACHE_DIR = os.path.normpath(
os.path.join(torch.hub.get_dir(), "..", "text"))torch.hub.get_dir will do the check for you.
There was a problem hiding this comment.
hmm, I wonder if someone use the set_dir from hub, then wouldn't that path would deviate from $TORCH_HOME? In other words do we want to give preference to torch.hub root directly over $TORCH_HOME for domains?
Another question for us now is if TORCH_HOME is not set do we want to default to torch.hub root :)
There was a problem hiding this comment.
At the beginning I thought the proposal was to piggy-back on PyTorch's default directory. Which has somewhat elaborated, but reasonable way of resolving the default. My view here is that I would avoid re-inventing the wheel so my suggestion was to reuse the implementation from PyTorch.
However, I get the sense that your view is that only TORCH_HOME should be considered and not necessarily how Torch Hub is resolving default path. I see that is another valid view and I don't have basis to say which is better.
But fundamentally, this kind of thinking is what happens when trying to incorporate yourself into something larger. I think simply using ~/.cache/torchtext would avoid this kind of unnecessary design thinking, and allow users have complete and precise control over where it should be, without worrying how these configurations mix with PyTorch's configuration.
There was a problem hiding this comment.
Having said that, let me stamp on the PR to unblock you.
There was a problem hiding this comment.
_get_torch_home() might be what you need for this.
It's private, but since we also maintain torchhub, it's not a problem. The way it's currently done in this PR is fine too, but wouldn't respect the XDG_CACHE_HOME env variable.
Addresses comment #1740 (comment)