x/build: enable builder-specific GOPROXY settings on reverse builders

[bcmills]

Looking at the builder log in https://build.golang.org/log/26ea2fd1426a51a224b5d321915717e906ded84a, I see the following line. Note that it lists GOPROXY three different times and GOROOT_BOOTSTRAP twice:

:: Running /tmp/farm/go/bin/go with args ["/tmp/farm/go/bin/go" "test" "-short" "golang.org/x/crypto/..."] and env ["LANG=zh_CN.UTF-8" "PATH=/tmp/farm/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" "HOME=/home/gopher" "LOGNAME=gopher" "USER=gopher" "SHELL=/bin/bash" "INVOCATION_ID=fe969678216d4cceaebbdc436df708c2" "JOURNAL_STREAM=8:4142335" "GO_BUILDER_ENV=host-linux-amd64-wsl" "WORKDIR=/tmp/farm" "HTTPS_PROXY=http://proxy:8123" "HTTP_PROXY=http://proxy:8123" "GOPROXY=https://goproxy.io,direct" "WSL_DISTRO_NAME=Ubuntu" "WSL_INTEROP=/run/WSL/20_interop" "WSLENV=BASH_ENV/u" "GO_STAGE0_NET_DELAY=700ms" "GO_STAGE0_DL_DELAY=600ms" "GOROOT_BOOTSTRAP=/tmp/farm/go1.4" "GO_BUILDER_NAME=linux-amd64-wsl" "GO_BUILDER_FLAKY_NET=1" "GOROOT_BOOTSTRAP=/usr/lib/go" "GOROOT=/tmp/farm/go" "GOPATH=/tmp/farm/gopath" "GOPROXY=http://10.240.0.3:30157" "GOPROXY=https://proxy.golang.org" "TMPDIR=/tmp/farm/tmp" "GOCACHE=/tmp/farm/gocache"] in dir /tmp/farm/gopath/src/golang.org/x/crypto

The second GOROOT_BOOTSTRAP seems to come from here, and might be working as intended: https://github.com/golang/build/blob/a9f75ee9992b28221ceec0c56af51265ea857ce2/cmd/buildlet/buildlet.go#L263

The second GOPROXY setting comes from here: https://github.com/golang/build/blob/5f1e306b98a112c236997d393adc5c11405377ca/cmd/coordinator/coordinator.go#L2702

The third GOPROXY setting comes from here: https://github.com/golang/build/blob/b6220de1adc000cad31eadf59558e1d3a8e32205/dashboard/builders.go#L921-L925

---

Given that one of the tests in that log failed with an EOF error communicating with proxy.golang.org, I suspect that it really should be using the local GOPROXY default (https://goproxy.io,direct) instead of the cmd/go default. I'm not sure what needs to happen to fix that configuration.

CC @mengzhuo @cagedmantis @golang/release

[mengzhuo]

Please assign this issue to me. @bcmills
p.s. Why can't I assign this issue to myself?

[mengzhuo]

To other builders maintainer, linux-amd64-wsl using systemd to set the enviroment.

[Unit]
Description=Go Builder For WSL
After=network-online.target

[Service]
ExecStartPre=/bin/bash -c 'chmod -R u+w /tmp/farm; rm -rf /tmp/farm; /bin/mkdir -p /tmp/farm; true'
ExecStart=/home/gopher/go/bin/stage0
Environment=GO_BUILDER_ENV=host-linux-amd64-wsl
Environment=WORKDIR=/tmp/farm
Environment=HTTPS_PROXY=http://proxy:8123
Environment=HTTP_PROXY=http://proxy:8123
Environment=GOPROXY=https://goproxy.io,direct

User=gopher
WorkingDirectory=/home/gopher

Restart=always
RestartSec=5
StartLimitInterval=0

[Install]
WantedBy=multi-user.target

[dmitshur]

p.s. Why can't I assign this issue to myself?

@mengzhuo Take a look at https://golang.org/wiki/GitHubAccess#assignees.

[gopherbot]

Change https://golang.org/cl/275332 mentions this issue: buildlet: check $workdir/go1.4 existance before append env

[gopherbot]

Change https://golang.org/cl/275412 mentions this issue: cmd/coordinator,dashboard: add privateGoProxy setting

[oiooj]

If anything else need to be change, I'd like to push it.

[bcmills]

greplogs --dashboard -md -l -e 'https://proxy\.golang\.org.*: Service Unavailable' --since=2021-01-01

2022-02-04T14:01:19-77f5fa5-4afcc9f/linux-amd64-wsl
2022-02-02T23:37:18-6944b10-a9384ee/linux-amd64-wsl
2022-02-01T21:46:52-d1c84af-36b81ac/linux-riscv64-unmatched
2022-02-01T19:09:20-6944b10-e22a14b/linux-amd64-wsl
2022-01-27T16:35:59-03fcf44-6cbcf58/linux-amd64-wsl
2022-01-25T22:04:10-03971e3-38729cf/linux-mips64le-mengzhuo
2022-01-18T16:01:17-e26b9f0-378766a/linux-amd64-wsl
2022-01-05T23:17:27-416c73c-2b39d86/linux-amd64-wsl
2022-01-05T09:50:29-f2171f2-2c58bb2/linux-riscv64-unmatched
2022-01-04T21:07:03-c09c1d3-95b240b/linux-riscv64-unmatched
2022-01-04T16:08:05-cd4a9ba-95b240b/linux-riscv64-unmatched
2021-12-30T12:17:33-53d65af-6178d25/linux-riscv64-unmatched
2021-12-30T12:11:21-084fe85-6178d25/linux-riscv64-unmatched
2021-12-20T22:25:28-30823e0-7c94355/linux-amd64-wsl
2021-12-16T22:15:57-b76863e-d01cd8f/linux-amd64-wsl
2021-12-15T15:54:46-d92f387-de690c2/linux-amd64-wsl
2021-12-04T11:59:35-c33205f-1876b38/linux-amd64-wsl
2021-12-02T05:26:57-ae814b3-1b2930d/linux-amd64-wsl
2021-12-01T15:58:46-ae814b3-029dfbc/linux-amd64-wsl
2021-12-01T15:58:46-6944b10-029dfbc/linux-amd64-wsl
2021-12-01T13:12:25-eb34ac4-b7651e5/linux-amd64-wsl
2021-11-22T03:24:07-03fcf44-a287c4a/linux-amd64-wsl
2021-11-20T08:47:36-ae814b3-91abe4b/linux-amd64-wsl
2021-11-19T21:36:24-b76863e-aec5c2e/linux-amd64-wsl
2021-11-10T02:26:41-03fcf44-02d7eab/linux-riscv64-unmatched
2021-11-02T06:25:39-4a448f8-088bb4b/linux-mips64le-mengzhuo
2021-09-30T01:32:54-22bfdb6-6656269/linux-riscv64-unmatched
2021-09-06T14:02:24-ddb58f2-ecfff58/linux-amd64-wsl
2021-09-02T12:49:05-027fa76-08e2519/linux-amd64-wsl
2021-08-17T04:37:32-b64cdd1-a304273/linux-amd64-wsl
2021-08-16T18:44:48-5d84be3-d1ba047/linux-amd64-wsl
2021-08-16T18:44:48-0a44fdf-d1ba047/linux-amd64-wsl
2021-04-02T17:58:59-cd4b3ce-254948a/linux-amd64-wsl

[bcmills]

@golang/release: this is a frequent failure mode on multiple reverse builders, and there has been a fix CL (CL 275412) out for review since December 2020. Can we prioritize a review for that change at some point in the near future?