url: add fileURLToPathBuffer API #58700

jasnell · 2025-06-12T20:49:22Z

The existing fileURLToPath() does not handle the case where the input URL contains percent-encoded characters that are not valid UTF-8 sequences. This can lead to issues, for instance, when the URL is constructed using file names in non-Unicode encodings (like Shift-JIS). This commit introduces a new API, fileURLToPathBuffer(), which returns a Buffer representing the path, allowing for accurate conversion of file URLs to paths without attempting to decode the percent-encoded bytes into characters.

This is part of the fix for #58634 but there's more to do.

nodejs-github-bot · 2025-06-12T20:49:27Z

Review requested:

@nodejs/startup
@nodejs/url

nodejs-github-bot · 2025-06-12T21:46:22Z

CI: https://ci.nodejs.org/job/node-test-pull-request/67426/

codecov · 2025-06-12T22:33:45Z

Codecov Report

Attention: Patch coverage is 48.76033% with 62 lines in your changes missing coverage. Please review.

Project coverage is 90.12%. Comparing base (5e1537c) to head (e362a1e).
Report is 16 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/url.js	47.45%	62 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #58700      +/-   ##
==========================================
- Coverage   90.16%   90.12%   -0.04%     
==========================================
  Files         637      637              
  Lines      188098   188122      +24     
  Branches    36905    36898       -7     
==========================================
- Hits       169601   169548      -53     
- Misses      11231    11319      +88     
+ Partials     7266     7255      -11

Files with missing lines	Coverage Δ
lib/internal/data_url.js	`100.00% <100.00%> (+8.23%)`	⬆️
lib/url.js	`100.00% <100.00%> (ø)`
lib/internal/url.js	`94.38% <47.45%> (-3.48%)`	⬇️

... and 47 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

nodejs-github-bot · 2025-06-12T22:56:02Z

CI: https://ci.nodejs.org/job/node-test-pull-request/67427/ 💛

lib/internal/url.js

test/parallel/test-fileurltopathbuffer.js

mcollina

lgtm

what could we do with this buffer? How could a user manipulate this?

jasnell · 2025-06-14T14:24:51Z

what could we do with this buffer?

It can be passed directly to node:fs APIs. More importantly tho, it's to help fix the issue discussed in #58634 by making it possible to normalize the arguments passed into the filter callback.

Ethan-Arrowood

I wont block on my one change recommendation, but please consider it in regards to consistency with path.win32 and path.posix pattern.

doc/api/url.md

The existing `fileURLToPath()` does not handle the case where the input URL contains percent-encoded characters that are not valid UTF-8 sequences. This can lead to issues, for instance, when the URL is constructed using file names in non-Unicode encodings (like Shift-JIS). This commit introduces a new API, `fileURLToPathBuffer()`, which returns a `Buffer` representing the path, allowing for accurate conversion of file URLs to paths without attempting to decode the percent-encoded bytes into characters.

nodejs-github-bot · 2025-06-14T19:25:57Z

CI: https://ci.nodejs.org/job/node-test-pull-request/67443/

LiviaMedeiros

Given that this is more robust, should internals that rely on fileURLToPath() be eventually migrated to using this function?

nodejs-github-bot · 2025-06-14T19:33:13Z

CI: https://ci.nodejs.org/job/node-test-pull-request/67444/

jasnell · 2025-06-14T19:34:05Z

... should internals that rely on fileURLToPath() be eventually migrated to using this function?

Probably.

nodejs-github-bot · 2025-06-14T20:54:09Z

Commit Queue failed

- Loading data for nodejs/node/pull/58700
✔  Done loading data for nodejs/node/pull/58700
----------------------------------- PR info ------------------------------------
Title      url: add fileURLToPathBuffer API (#58700)
Author     James M Snell <[email protected]> (@jasnell)
Branch     jasnell:jasnell/add-fileurltopathbuffer -> nodejs:main
Labels     fs, url, semver-minor, whatwg-url, author ready, needs-ci
Commits    1
 - url: add fileURLToPathBuffer API
Committers 1
 - James M Snell <[email protected]>
PR-URL: https://github.com/nodejs/node/pull/58700
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ethan Arrowood <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/58700
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ethan Arrowood <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
--------------------------------------------------------------------------------
   ℹ  This PR was created on Thu, 12 Jun 2025 20:49:22 GMT
   ✔  Approvals: 3
   ✔  - Matteo Collina (@mcollina) (TSC): https://github.com/nodejs/node/pull/58700#pullrequestreview-2927770869
   ✔  - Ethan Arrowood (@Ethan-Arrowood): https://github.com/nodejs/node/pull/58700#pullrequestreview-2928406839
   ✔  - LiviaMedeiros (@LiviaMedeiros): https://github.com/nodejs/node/pull/58700#pullrequestreview-2928714008
   ✘  Last GitHub CI failed
   ℹ  Last Full PR CI on 2025-06-14T19:33:13Z: https://ci.nodejs.org/job/node-test-pull-request/67444/
- Querying data for job/node-test-pull-request/67444/
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu

https://github.com/nodejs/node/actions/runs/15656016109

The existing `fileURLToPath()` does not handle the case where the input URL contains percent-encoded characters that are not valid UTF-8 sequences. This can lead to issues, for instance, when the URL is constructed using file names in non-Unicode encodings (like Shift-JIS). This commit introduces a new API, `fileURLToPathBuffer()`, which returns a `Buffer` representing the path, allowing for accurate conversion of file URLs to paths without attempting to decode the percent-encoded bytes into characters. PR-URL: #58700 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]> Reviewed-By: LiviaMedeiros <[email protected]>

jasnell · 2025-06-14T21:06:19Z

Landed in 3f6ad56

The existing `fileURLToPath()` does not handle the case where the input URL contains percent-encoded characters that are not valid UTF-8 sequences. This can lead to issues, for instance, when the URL is constructed using file names in non-Unicode encodings (like Shift-JIS). This commit introduces a new API, `fileURLToPathBuffer()`, which returns a `Buffer` representing the path, allowing for accurate conversion of file URLs to paths without attempting to decode the percent-encoded bytes into characters. PR-URL: #58700 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]> Reviewed-By: LiviaMedeiros <[email protected]>

Notable changes: doc: * add islandryu to collaborators (Shima Ryuhei) #58714 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #58643 test_runner: * (SEMVER-MINOR) support object property mocking (Idan Goshen) #58438 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 PR-URL: #58727

Notable changes: doc: * add islandryu to collaborators (Shima Ryuhei) #58714 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #58643 test_runner: * (SEMVER-MINOR) support object property mocking (Idan Goshen) #58438 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 PR-URL: #58804

Notable changes: doc: * add islandryu to collaborators (Shima Ryuhei) #58714 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #58643 test: * fix test-timeout-flag after revert of auto subtest wait (Pietro Marchini) #58282 test_runner: * Revert "test_runner: remove promises returned by t.test() (Romain Menke) #58282 * Revert "test_runner: remove promises returned by test() (Romain Menke) #58282 * Revert "test_runner: automatically wait for subtests to finish (Romain Menke) #58282 * (SEMVER-MINOR) support object property mocking (Idan Goshen) #58438 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 PR-URL: #58813

Notable changes: doc: * add islandryu to collaborators (Shima Ryuhei) #58714 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #58643 test: * fix test-timeout-flag after revert of auto subtest wait (Pietro Marchini) #58282 test_runner: * (SEMVER-MINOR) support object property mocking (Idan Goshen) #58438 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 PR-URL: #58813

The existing `fileURLToPath()` does not handle the case where the input URL contains percent-encoded characters that are not valid UTF-8 sequences. This can lead to issues, for instance, when the URL is constructed using file names in non-Unicode encodings (like Shift-JIS). This commit introduces a new API, `fileURLToPathBuffer()`, which returns a `Buffer` representing the path, allowing for accurate conversion of file URLs to paths without attempting to decode the percent-encoded bytes into characters. PR-URL: #58700 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]> Reviewed-By: LiviaMedeiros <[email protected]>

Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) #56350 * (SEMVER-MINOR) update amaro to 1.0.0 (Node.js GitHub Bot) #56350 * (SEMVER-MINOR) update amaro to 0.5.3 (Node.js GitHub Bot) #56350 * (SEMVER-MINOR) update amaro to 0.5.2 (Node.js GitHub Bot) #56350 * (SEMVER-MINOR) update amaro to 0.5.1 (Marco Ippolito) #56350 * (SEMVER-MINOR) update amaro to 0.5.0 (nodejs-github-bot) #56350 doc: * (SEMVER-MINOR) add all watch-mode related flags to node.1 (Dario Piotrowicz) #58719 * add islandryu to collaborators (Shima Ryuhei) #58714 * (SEMVER-MINOR) add history entries to `--input-type` section (Antoine du Hamel) #56350 esm: * (SEMVER-MINOR) implement import.meta.main (Joe) #57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) improve typescript error message format (Marco Ippolito) #56350 * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #56350 * (SEMVER-MINOR) refactor commonjs typescript loader (Marco Ippolito) #56350 * (SEMVER-MINOR) unflag --experimental-strip-types (Marco Ippolito) #56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) #58853 sqlite: * (SEMVER-MINOR) add support for readBigInts option in db connection level (Miguel Marcondes Filho) #58697 src,permission: * (SEMVER-MINOR) add support to permission.has(addon) (Rafael Gonzaga) #58951 test: * (SEMVER-MINOR) add test for async disposable worker thread (James M Snell) #58385 url: * (SEMVER-MINOR) add fileURLToPathBuffer API (James M Snell) #58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) #58719 worker: * (SEMVER-MINOR) make Worker async disposable (James M Snell) #58385 PR-URL: #59256

Notable changes: deps: * (SEMVER-MINOR) update amaro to 1.1.0 (Node.js GitHub Bot) #56350 doc: * add islandryu to collaborators (Shima Ryuhei) #58714 esm: * (SEMVER-MINOR) implement `import.meta.main` (Joe) #57804 fs: * (SEMVER-MINOR) allow correct handling of burst in fs-events with AsyncIterator (Philipp Dunkel) #58490 module: * (SEMVER-MINOR) remove experimental warning from type stripping (Marco Ippolito) #56350 * (SEMVER-MINOR) unflag `--experimental-strip-types` (Marco Ippolito) #56350 permission: * (SEMVER-MINOR) propagate permission model flags on spawn (Rafael Gonzaga) #58853 sqlite: * (SEMVER-MINOR) add support for `readBigInts` option in db connection level (Miguel Marcondes Filho) #58697 src,permission: * (SEMVER-MINOR) add support to `permission.has(addon)` (Rafael Gonzaga) #58951 url: * (SEMVER-MINOR) add `fileURLToPathBuffer` API (James M Snell) #58700 watch: * (SEMVER-MINOR) add `--watch-kill-signal` flag (Dario Piotrowicz) #58719 worker: * (SEMVER-MINOR) make `Worker` async disposable (James M Snell) #58385 PR-URL: #59256 Co-authored-by: Antoine du Hamel <[email protected]>

jasnell requested review from mcollina, anonrig and dario-piotrowicz June 12, 2025 20:49

jasnell added fs Issues and PRs related to the fs subsystem / file system. url Issues and PRs related to the legacy built-in url module. semver-minor PRs that contain new features and should be released in the next minor version. labels Jun 12, 2025

nodejs-github-bot added needs-ci PRs that need a full CI run. whatwg-url Issues and PRs related to the WHATWG URL implementation. labels Jun 12, 2025

This comment was marked as outdated.

Sign in to view

jasnell force-pushed the jasnell/add-fileurltopathbuffer branch 2 times, most recently from 249ab0d to 8ccd48d Compare June 12, 2025 20:59

This comment was marked as outdated.

Sign in to view

jasnell mentioned this pull request Jun 12, 2025

fs.cpSync / fs.cp / fs.promises.cp fails when src/dest args are Buffer #58634

Open

jasnell force-pushed the jasnell/add-fileurltopathbuffer branch from 8ccd48d to 803aa96 Compare June 12, 2025 21:44

jasnell force-pushed the jasnell/add-fileurltopathbuffer branch from 803aa96 to cb63900 Compare June 12, 2025 22:54

anonrig reviewed Jun 12, 2025

View reviewed changes

lib/internal/url.js Show resolved Hide resolved

aduh95 reviewed Jun 13, 2025

View reviewed changes

lib/internal/url.js Outdated Show resolved Hide resolved

aduh95 reviewed Jun 13, 2025

View reviewed changes

lib/internal/url.js Outdated Show resolved Hide resolved

aduh95 reviewed Jun 13, 2025

View reviewed changes

test/parallel/test-fileurltopathbuffer.js Show resolved Hide resolved

mcollina approved these changes Jun 14, 2025

View reviewed changes

jasnell force-pushed the jasnell/add-fileurltopathbuffer branch from cb63900 to 755e985 Compare June 14, 2025 14:23

This comment was marked as outdated.

Sign in to view

Ethan-Arrowood approved these changes Jun 14, 2025

View reviewed changes

doc/api/url.md Show resolved Hide resolved

jasnell added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jun 14, 2025

jasnell force-pushed the jasnell/add-fileurltopathbuffer branch from 755e985 to e362a1e Compare June 14, 2025 15:00

This comment was marked as outdated.

Sign in to view

LiviaMedeiros approved these changes Jun 14, 2025

View reviewed changes

jasnell added the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 14, 2025

nodejs-github-bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Jun 14, 2025

jasnell closed this Jun 14, 2025

github-actions bot mentioned this pull request Jun 16, 2025

2025-06-17, Version 24.3.0 (Current) #58727

Closed

github-actions bot mentioned this pull request Jun 23, 2025

2025-06-24, Version 24.3.0 (Current) #58804

Closed

github-actions bot mentioned this pull request Jun 24, 2025

2025-06-24, Version 24.3.0 (Current) #58813

Merged

github-actions bot mentioned this pull request Jul 28, 2025

2025-07-29, Version 22.18.0 'Jod' (LTS) #59256

Merged

Uh oh!

url: add fileURLToPathBuffer API #58700

url: add fileURLToPathBuffer API #58700

Conversation

jasnell commented Jun 12, 2025

Uh oh!

nodejs-github-bot commented Jun 12, 2025

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

nodejs-github-bot commented Jun 12, 2025

Uh oh!

codecov bot commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

nodejs-github-bot commented Jun 12, 2025 • edited by jasnell Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

jasnell commented Jun 14, 2025

Uh oh!

This comment was marked as outdated.

Ethan-Arrowood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as outdated.

nodejs-github-bot commented Jun 14, 2025

Uh oh!

LiviaMedeiros left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Jun 14, 2025

Uh oh!

jasnell commented Jun 14, 2025

Uh oh!

nodejs-github-bot commented Jun 14, 2025

Uh oh!

jasnell commented Jun 14, 2025

Uh oh!

Uh oh!

codecov bot commented Jun 12, 2025 •

edited

Loading

nodejs-github-bot commented Jun 12, 2025 •

edited by jasnell

Loading