Proxy WebSocket communication through the main thread

[georgestagg]

Also:

• Adds support for webr::eval_js(..., await = TRUE), based on a similar mechanism.
• Downloads SSL cacert bundle from curl.se at build time, and installs it to the Emscripten VFS.
• Set default WebSocket-to-TCP protocol to wss://.
• Document networking with webR.

This PR also removes the service worker communication channel. This channel has been deprecated for a while now, is never chosen in Automatic mode, and when explicitly selected emits a warning message saying it will be deleted soon.

After this change the channel will indeed be gone and can no longer be selected at all.

---

Once everything has been setup, and a WebSocket proxy + SOCKS tunnel is running outside the browser:

install.packages(c("curl", "httr2", "jsonlite"))
library(httr2)

response <- request("https://hb.cran.dev/get") |>
  req_perform() |>
  resp_body_json()

response$headers$`User-Agent`
#
# [1] "httr2/1.2.0 r-curl/6.4.0 libcurl/8.14.1"

[jeroen]

Could you add the ca-bundle path explicitly to the libcurl configuration:

--with-ca-bundle=$(WASM)/etc/ssl/cert.pem

[jeroen]

Has https://webr.r-wasm.org/proxy-websockets/ been updated yet to this latest PR? We still need to set wss:// at runtime for this to work.

Sys.setenv(ALL_PROXY='socks5h://test:[email protected]:443')

install.packages('curl')
library(curl)

# Show that proxy works
curl_fetch_memory('http://hb.opencpu.org/get', handle = new_handle(verbose=TRUE))

# Run more complex examples
example(multi_run)
example(curl)

Perhaps we just need to rebuild libcurl and/or the curl R package with this new -s WEBSOCKET_URL=wss:// flag.

[georgestagg]

Has https://webr.r-wasm.org/proxy-websockets/ been updated yet to this latest PR?

Not yet, I trigger PR deployments manually. The wss:// line is not needed for my local build with the latest changes.

I'll add the ca-bundle path config to curl and deploy a new build once I'm back next week :)

[jeroen]

Here is an example pushing this to download 200 files in parallel over HTTP2

This works on https://webr.r-wasm.org/proxy-websockets/

install.packages('curl')
Sys.setenv(ALL_PROXY='socks5h://test:[email protected]:443')
mirror <- 'https://cran.rstudio.com'
df <- read.dcf(curl::curl(sprintf('%s/src/contrib/PACKAGES', mirror)))
pkgs <- df[1:200,'Package']
urls <- sprintf('%s/web/packages/%s/DESCRIPTION', mirror, pkgs)
outdir <- file.path(tempdir(), 'descriptions')
files <- sprintf('%s/%s.txt', outdir, pkgs)
dir.create(outdir, showWarnings = FALSE)
results <- curl::multi_download(urls, files, progress = FALSE, multiplex = T, verbose=T)
all(results$status == 200)
list.files(outdir)

[jeroen]

It seems the path to the CA bundle is different at build time than in the r-wasm runtime. I get an error in the WebR:

Error setting certificate file: /opt/webr/wasm/etc/ssl/cert.pem 

So I think we need to configure the runtime path location the CA bundle in curl:

--with-ca-bundle=/etc/ssl/cert.pem

Or alternatively the runtime needs a symlink /opt/webr/wasm -> /

[georgestagg]

OK, let’s try with --with-ca-bundle=/etc/ssl/cert.pem.

[jeroen]

Thanks, all looks good now from my end. I think this is ready for public preview.