Embed is broken by period or underscore in the prefix of reference notebook

[DmitriyLeybel]

Bug description

In trying to embed a hidden notebook, I encountered an issue with the text parsing of the uri which leads to a NotFound Error due to the / being removed.

Using this reference: {{< embed ../../_cross_test.ipynb >}} led to:

PS C:\Users\dmley\OneDrive\Desktop\Sites\dmitriyleybel.github.io> quarto render
[1/5] about.qmd
[2/5] blog.qmd
[3/5] index.qmd
[4/5] posts\test_poast2\index.ipynb
ERROR: Rendering of output notebook produced an unexpected result
ERROR: NotFound: The system cannot find the file specified. (os error 2): readfile 'C:\Users\dmley\OneDrive\Desktop\Sites\dmitriyleybel.github.io_cross_test.ipynb'
Path: C:\Users\dmley\OneDrive\Desktop\Sites\dmitriyleybel.github.io_cross_test.ipynb

Stack trace:
Path: C:\Users\dmley\OneDrive\Desktop\Sites\dmitriyleybel.github.io_cross_test.ipynb
    at readTextFileSync (ext:deno_fs/30_fs.js:733:14)
    at Object.Deno.readTextFileSync (file:///C:/Program%20Files/Quarto/bin/quarto.js:5625:25)
    at Object.target (file:///C:/Program%20Files/Quarto/bin/quarto.js:38268:37)
    at fileExecutionEngineAndTarget (file:///C:/Program%20Files/Quarto/bin/quarto.js:41263:33)
    at renderFileInternal (file:///C:/Program%20Files/Quarto/bin/quarto.js:77120:42)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async renderFile (file:///C:/Program%20Files/Quarto/bin/quarto.js:77095:13)
    at async Object.renderOutputNotebook [as render] (file:///C:/Program%20Files/Quarto/bin/quarto.js:81477:22)
    at async Object.render (file:///C:/Program%20Files/Quarto/bin/quarto.js:81748:34)
    at async Object.renderPreviews (file:///C:/Program%20Files/Quarto/bin/quarto.js:109695:47)

Steps to reproduce

1. Create a source notebook with either a _ or . prefix. e.g. _ref.ipynb, and let it live in a folder a directory or two away from the notebook you wish to render.
2. Create a notebook to render(e.g. index.ipynb, and as one of the cells in the notebook, create a reference {{< embed ../../_ref.ipynb >}} to the source.
3. Run quarto render Observe error

Expected behavior

No response

Actual behavior

No response

Your environment

• IDE: VS Code 1.86.2
• OS: Win11
• Quarto: 1.4.551

Quarto check output

Quarto 1.4.551
[>] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.11: OK
      Dart Sass version 1.69.5: OK
      Deno version 1.37.2: OK
[>] Checking versions of quarto dependencies......OK
[>] Checking Quarto installation......OK
      Version: 1.4.551
      Path: C:\Program Files\Quarto\bin
      CodePage: 1252

[>] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[>] Checking LaTeX....................OK
      Tex:  (not detected)

[>] Checking basic markdown render....OK

[>] Checking Python 3 installation....OK
      Version: 3.11.3
      Path: C:/Users/dmley/AppData/Local/Programs/Python/Python311/python.exe
      Jupyter: 5.7.2
      Kernels: python3

[>] Checking Jupyter engine render....OK

[>] Checking R installation...........(None)

      Unable to locate an installed version of R.
      Install R from https://cloud.r-project.org/

[DmitriyLeybel]

Similar case for folders prefixed with a period or underscore.

e.g. {{< embed ../../.notebooks/cross_test.ipynb >}} leads to

ERROR: Rendering of output notebook produced an unexpected result
ERROR: NotFound: The system cannot find the path specified. (os error 3): readfile 'C:\Users\dmley\OneDrive\Desktop\Sites\dmitriyleybel.github.io.notebooks\cross_test.ipynb'
Path: C:\Users\dmley\OneDrive\Desktop\Sites\dmitriyleybel.github.io.notebooks\cross_test.ipynb

[cscheid]

Does this happen if you don't use a path that goes outside of the directory where the .qmd file resides?

[DmitriyLeybel]

Does this happen if you don't use a path that goes outside of the directory where the .qmd file resides?

Just moved _cross_test.ipynb to the same level as the qmd file I'm using to reference it and still am getting a similar error. The / disappears.

{{< embed _cross_test.ipynb >}}

Rendering notebook previews
[1/1] dmitriyleybel.github.io_cross_test.ipynb
Rendering of output notebook produced an unexpected result
NotFound: The system cannot find the file specified. (os error 2): readfile 'C:\Users\dmley\OneDrive\Desktop\Sites\dmitriyleybel.github.io_cross_test.ipynb'
Path: C:\Users\dmley\OneDrive\Desktop\Sites\dmitriyleybel.github.io_cross_test.ipynb

[cderv]

This happens because we set the path of the embedded notebook using an attributes on a Divs

quarto-cli/src/core/jupyter/jupyter-embed.ts

Lines 403 to 424 in d39cb2f

	// Wrap any injected cells with a div that includes a back link to
	// the notebook that originated the cells
	const notebookMarkdown = (
	nbAbsPath: string,
	cells: JupyterCellOutput[],
	title?: string,
	) => {
	const cellId = cells.length > 0 ? cells[0].id || "" : "";
	const markdown = [
	"",
	`:::{.quarto-embed-nb-cell notebook="${nbAbsPath}" ${
	title ? `notebook-title="${title}"` : ""
	} notebook-cellId="${cellId}"}`,
	];
	const cellOutput = cells.map((cell) => cell.markdown).join("");
	markdown.push("");
	markdown.push(cellOutput);
	markdown.push("");
	markdown.push(":::");
	return markdown.join("\n");
	};

And we retrieve the information from the HTML div created later

quarto-cli/src/format/html/format-html-notebook.ts

Lines 193 to 198 in d39cb2f

	// Process embedded notebook contents,
	// emitting links to the notebooks inline (where the embedded content is located)
	const notebookDivNodes = doc.querySelectorAll("[data-notebook]");
	for (const nbDivNode of notebookDivNodes) {
	const nbDivEl = nbDivNode as Element;
	const notebookPath = nbDivEl.getAttribute("data-notebook");

However, the path set in attributes and gets processed where _ is not escaped and means \_ means _ hence the results

❯ pandoc -t html
:::{.quarto-embed-nb-cell notebook="C:\Users\chris\Documents\DEV_OTHER\00-TESTS\test-quarto\_ref.ipynb" notebook-title="test" notebook-cellId="cell-0"}
:::
^Z
<div class="quarto-embed-nb-cell"
data-notebook="C:\Users\chris\Documents\DEV_OTHER\00-TESTS\test-quarto_ref.ipynb"
data-notebook-title="test" data-notebook-cellId="cell-0">

</div>

So we need to escape correctly, and / or insure the attributes is not processed as markdown

❯ pandoc -t html
::: {.class attr="a\_b"}
:::
^Z
<div class="class" data-attr="a_b">

</div>

# Same as without slash

❯ pandoc -t html
:::{.class attr="a_b"}
:::
^Z
<div class="class" data-attr="a_b">

</div>

# escaping works

❯ pandoc -t html
::: {.class attr="a\\_b"}
:::
^Z
<div class="class" data-attr="a\_b">

</div>

[cderv]

@cscheid do you know the characters that would require being escaped in attributes? It does not seem the attributes are passed as Markdown, so I wonder what is the strings that needs escaping.

[cscheid]

I don't. It's a very good thing for us to learn precisely.

[cderv]

So I have looked into this, and the markdown reader do loose the backslash

❯ pandoc -t native
::: {.class attr="a\_b"}
:::
^Z
[ Div ( "" , [ "class" ] , [ ( "attr" , "a_b" ) ] ) [] ]

So I think this is in haskel directly

• In the function parsing Divs
  https://github.com/jgm/pandoc/blob/c29c18a0038b62718cf3133247177e3cb8ebf871/src/Text/Pandoc/Readers/Markdown.hs#L2125-L2141

  attribs <- attributes <|> ((\x -> ("",[x],[])) <$> many1Char nonspaceChar)

• The attributes function do parse the attribute line
  https://github.com/jgm/pandoc/blob/c29c18a0038b62718cf3133247177e3cb8ebf871/src/Text/Pandoc/Readers/Markdown.hs#L622-L631

• Which call the key value parser
  https://github.com/jgm/pandoc/blob/c29c18a0038b62718cf3133247177e3cb8ebf871/src/Text/Pandoc/Readers/Markdown.hs#L651-L664

• which call the escaping function
  https://github.com/jgm/pandoc/blob/c29c18a0038b62718cf3133247177e3cb8ebf871/src/Text/Pandoc/Readers/Markdown.hs#L1550-L1556

  escapedChar' :: PandocMonad m => MarkdownParser m Char
  escapedChar' = try $ do
    char '\\'
    (guardEnabled Ext_all_symbols_escapable >> satisfy (not . isAlphaNum))
       <|> (guardEnabled Ext_angle_brackets_escapable >>
              oneOf "\\`*_{}[]()>#+-.!~\"<>")
       <|> oneOf "\\`*_{}[]()>#+-.!"

So it seems this will be related to all_symbols_escapable extension, and the list of characters here \\*_{}>#+-.!`

Maybe we should have a escape function in typescript that does it ?