x/tools/gopls: gopls implementation result does not include all types in vendor directory

[x-yuri]

What version of Go, VS Code & VS Code Go extension are you using?

Version Information

• Run go version to get version of Go from the VS Code integrated terminal.
  • go version go1.19.1 linux/amd64
• Run gopls -v version to get version of Gopls from the VS Code integrated terminal.

  Build info
  ----------
  golang.org/x/tools/gopls v0.9.5
      golang.org/x/tools/gopls@(devel)
      github.com/BurntSushi/toml@v1.2.0 h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
      github.com/google/go-cmp@v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
      github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
      golang.org/x/exp@v0.0.0-20220722155223-a9213eeb770e h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
      golang.org/x/exp/typeparams@v0.0.0-20220722155223-a9213eeb770e h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
      golang.org/x/mod@v0.6.0-dev.0.20220419223038-86c51ed26bb4 h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
      golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4 h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
      golang.org/x/sys@v0.0.0-20220722155257-8c9f86f7a55f h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
      golang.org/x/text@v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
      golang.org/x/tools@v0.1.13-0.20220908144252-ce397412b6a4 h1:glzimF7qHZuKVEiMbE7UqBu44MyTjt5u6j3Jz+rfMRM=
      golang.org/x/vuln@v0.0.0-20220901221904-62b0186a1058 h1:YnB27EXBD8XxB0JcaOeluuvhF2kS4DrH0k+lpopG2xc=
      honnef.co/go/tools@v0.3.2 h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34=
      mvdan.cc/gofumpt@v0.3.1 h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
      mvdan.cc/xurls/v2@v2.4.0 h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
  go: go1.19.1
  

• Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders.

  1.71.2
  74b1f979648cc44d385a2286793c226e611f59e7
  x64
  

• Check your installed extensions to get the version of the VS Code Go extension
  • v0.35.2
• Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command.

  Checking configured tools....
  GOBIN: undefined
  toolsGopath: 
  gopath: /home/yuri/go
  GOROOT: /usr/lib/go
  PATH: /home/yuri/perl5/bin:/home/yuri/bin:/home/yuri/Dropbox/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
  
      go: /usr/bin/go: go version go1.19.1 linux/amd64
  
      gotests:    not installed
      gomodifytags:   not installed
      impl:   not installed
      goplay: not installed
      dlv:    /usr/bin/dlv    (version: (devel) built with go: go1.19)
      staticcheck:    /usr/bin/staticcheck    (version: (devel) built with go: go1.18.5)
      gopls:  /usr/bin/gopls  (version: (devel) built with go: go1.19.1)
  
  go env
  Workspace Folder (docker): /home/yuri/go/src/github.com/docker/docker
      GO111MODULE=""
      GOARCH="amd64"
      GOBIN=""
      GOCACHE="/home/yuri/.cache/go-build"
      GOENV="/home/yuri/.config/go/env"
      GOEXE=""
      GOEXPERIMENT=""
      GOFLAGS=""
      GOHOSTARCH="amd64"
      GOHOSTOS="linux"
      GOINSECURE=""
      GOMODCACHE="/home/yuri/go/pkg/mod"
      GONOPROXY=""
      GONOSUMDB=""
      GOOS="linux"
      GOPATH="/home/yuri/go"
      GOPRIVATE=""
      GOPROXY="https://proxy.golang.org,direct"
      GOROOT="/usr/lib/go"
      GOSUMDB="sum.golang.org"
      GOTMPDIR=""
      GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
      GOVCS=""
      GOVERSION="go1.19.1"
      GCCGO="gccgo"
      GOAMD64="v1"
      AR="ar"
      CC="gcc"
      CXX="g++"
      CGO_ENABLED="1"
      GOMOD="/dev/null"
      GOWORK=""
      CGO_CFLAGS="-g -O2"
      CGO_CPPFLAGS=""
      CGO_CXXFLAGS="-g -O2"
      CGO_FFLAGS="-g -O2"
      CGO_LDFLAGS="-g -O2"
      PKG_CONFIG="pkg-config"
      GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1841263837=/tmp/go-build -gno-record-gcc-switches"
  

Share the Go related settings you have added/edited

I didn't change any settings.

Describe the bug

I trying to examine the moby project, and the extension seems to ignore the vendor dir. Possibly because there's no go.mod at the root of the repository. E.g. it can't find the implementation of p.repo.Blobs, that is located here. As long as I need to navigate within the main module it seems to work fine.

Steps to reproduce the behavior:

1. Clone https://github.com/moby/moby to ~/go/src/github.com/docker/docker.
2. Open the directory in code, then distribution/pull_v2.go (line 903).
3. Right click on Blobs, choose "Go to Implementations".

You'll see only some mock implementations from distribution/push_v2_test.go.

Screenshots or recordings

[hyangah]

Can you try with the gopls at master?

git clone https://go.googlesource.com/tools
cd tools/gopls
go install

And restart the gopls (Go: Restart Language Server) or reload the vscode window.

I could reproduce the issue with the released version of gopls. But I see the gopls built from master now finds code in the vendor location correctly.

[x-yuri]

I've just tried, but see no difference. It can't find that particular implementation in the vendor dir. I tried both restarting the server and code.

$ ~/go/bin/gopls version
golang.org/x/tools/gopls master
    golang.org/x/tools/gopls@(devel)

Is there a way to make a curl request to gopls or something to confirm this? Can you possibly tell what's the extension's part in it? Does it simply tell code how to connect to the server? Or there's more to it? Any links to the code?

[suzmue]

Could you please collect a trace and share it here?

You can do this by setting "go.languageServerFlags": ["-rpc.trace"] in your settings.json and then sharing the logs you see in the output channel:

[hyangah]

To check the detailed version of gopls -

$ go version -m  /usr/bin/gopls 

The gopls your extension is using is printed in the Go: Locate Configured Go Tools output.

[x-yuri]

The trace: https://gist.github.com/x-yuri/985f8289b15e07fe1038c1eafe679b0b

$ go version -m  /usr/bin/gopls 
/usr/bin/gopls: go1.19.1
	path	golang.org/x/tools/gopls
	mod	golang.org/x/tools/gopls	(devel)	
	dep	github.com/BurntSushi/toml	v1.2.0	h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
	dep	github.com/google/go-cmp	v0.5.8	h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
	dep	github.com/sergi/go-diff	v1.1.0	h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
	dep	golang.org/x/exp	v0.0.0-20220722155223-a9213eeb770e	h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
	dep	golang.org/x/exp/typeparams	v0.0.0-20220722155223-a9213eeb770e	h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
	dep	golang.org/x/mod	v0.6.0-dev.0.20220419223038-86c51ed26bb4	h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
	dep	golang.org/x/sync	v0.0.0-20220722155255-886fb9371eb4	h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
	dep	golang.org/x/sys	v0.0.0-20220722155257-8c9f86f7a55f	h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
	dep	golang.org/x/text	v0.3.7	h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
	dep	golang.org/x/tools	v0.1.13-0.20220908144252-ce397412b6a4	h1:glzimF7qHZuKVEiMbE7UqBu44MyTjt5u6j3Jz+rfMRM=
	dep	golang.org/x/vuln	v0.0.0-20220901221904-62b0186a1058	h1:YnB27EXBD8XxB0JcaOeluuvhF2kS4DrH0k+lpopG2xc=
	dep	honnef.co/go/tools	v0.3.2	h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34=
	dep	mvdan.cc/gofumpt	v0.3.1	h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
	dep	mvdan.cc/xurls/v2	v2.4.0	h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
	build	-compiler=gc
	build	-trimpath=true
	build	CGO_ENABLED=1
	build	GOARCH=amd64
	build	GOOS=linux
	build	GOAMD64=v1

$ go version -m  ~/go/bin/gopls
/home/yuri/go/bin/gopls: go1.19.1
	path	golang.org/x/tools/gopls
	mod	golang.org/x/tools/gopls	(devel)	
	dep	github.com/BurntSushi/toml	v1.2.0	h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
	dep	github.com/google/go-cmp	v0.5.8	h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
	dep	github.com/sergi/go-diff	v1.1.0	h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
	dep	golang.org/x/exp	v0.0.0-20220722155223-a9213eeb770e	h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
	dep	golang.org/x/exp/typeparams	v0.0.0-20220722155223-a9213eeb770e	h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
	dep	golang.org/x/mod	v0.6.0-dev.0.20220419223038-86c51ed26bb4	h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
	dep	golang.org/x/sync	v0.0.0-20220722155255-886fb9371eb4	h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
	dep	golang.org/x/sys	v0.0.0-20220808155132-1c4a2a72c664	h1:v1W7bwXHsnLLloWYTVEdvGvA7BHMeBYsPcF0GLDxIRs=
	dep	golang.org/x/text	v0.3.7	h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
	dep	golang.org/x/tools	v0.1.13-0.20220928184430-f80e98464e27
	=>	../	(devel)	
	
	dep	golang.org/x/vuln	v0.0.0-20221004232641-2aa0553d353b	h1:8Tu9pgIV7kt8ulNtzidzpLl9E9l1i+U4QLdKG0ZzHyE=
	dep	honnef.co/go/tools	v0.3.3	h1:oDx7VAwstgpYpb3wv0oxiZlxY+foCpRAwY7Vk6XpAgA=
	dep	mvdan.cc/gofumpt	v0.3.1	h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
	dep	mvdan.cc/xurls/v2	v2.4.0	h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
	build	-compiler=gc
	build	CGO_ENABLED=1
	build	CGO_CFLAGS=
	build	CGO_CPPFLAGS=
	build	CGO_CXXFLAGS=
	build	CGO_LDFLAGS=
	build	GOARCH=amd64
	build	GOOS=linux
	build	GOAMD64=v1
	build	vcs=git
	build	vcs.revision=bd8c28ff5c76b957a23f93a67337acc0c5a4f67e
	build	vcs.time=2022-10-06T14:25:42Z
	build	vcs.modified=false

Can you try with the gopls at master?

Now that I think about it, maybe I should uninstall gopls and try again.

[x-yuri]

But Go: Locate Configured Go Tools says:

gopls: /home/yuri/go/bin/gopls (version: (devel) built with go: go1.19.1)

All in all looks like a gopls issue. Should I report it there?

[hyangah]

Thanks for capturing the log.

I also verified this is reproducible - sorry that I was confused by "Find all references" reporting the interface definition correctly. From your original report, I see that you are reporting that the "Find all implementations" (or Implementations) result is missing a type in the vendor directory.

In the example you shared, p.repo.Blobs implements a method of the interface distribution.Repository, so you wanted the implementations query to return all implementations found not only in the main workspace, including distribution.repository. (not distribution.blobs.Get). Here what I mean by main workspace is the code excluding vendor or third-party dependencies.

I think this happens because the implementation type (distribution.repository) in vendor is unexported. For performance, gopls employs various heuristics; one of them is to avoid full type-checking when a package is outside the workspace (i.e. packages in dependency). (relevant code: workspaceParseMode). Packages under vendor seem to be considered packages outside workspace, then gopls will operate in exported symbol only mode. This can affect the implementations query search space and results in skipping unexported types.

I verified gopls returns the implementation type if I rename distribution.repository to an exported name. (See the one with red underline)

I will transfer this to gopls issue tracker, but I am afraid this is subtle and may not be fixable easily.

[x-yuri]

Oh, I just noticed that one of the links in my report is wrong (the second one):

E.g. it can't find the implementation of p.repo.Blobs, that is located here.

And that's probably what you meant here:

In the example you shared, p.repo.Blobs implements a method of the interface distribution.Repository, so you wanted the implementations query to return all implementations found not only in the main workspace, including distribution.repository. (not distribution.blobs.Get).

So to make sure we're on the same page...

I know that p.repo.Blobs() here (distribution/pull_v2.go, v2Puller.pullSchema2Config()) invokes repository.Blobs() here (vendor/github.com/docker/distribution/registry/client/repository.go). According to vscode p.repo (from v2Puller.pullSchema2Config()) implements the Repository interface from vendor/github.com/docker/distribution/registry.go.

In the same vein blobs.Get() here (distribution/pull_v2.go, v2Puller.pullSchema2Config()) invokes blobs.Get() here (vendor/github.com/docker/distribution/registry/client/repository.go). According to vscode blobs (from v2Puller.pullSchema2Config()) implements the BlobProvider interface from vendor/github.com/docker/distribution/blobs.go.