gh-135661: Fix CDATA section parsing in HTMLParser #135665
Conversation
"] ]>" and "]] >" no longer end the CDATA section.
serhiy-storchaka
added
needs backport to 3.13
serhiy-storchaka
added
type-security
Lib/html/parser.py
Outdated
| j = rawdata.find(']]>') | ||
| if j < 0: | ||
| return -1 | ||
| self.unknown_decl(rawdata[i+3: j]) | ||
| return j + 3 |
There was a problem hiding this comment.
According to the HTML5 standard (https://html.spec.whatwg.org/multipage/parsing.html#markup-declaration-open-state), it should be either data or bogus comment (which ends with >, not ]]>), but this depends on the context. It may be that I incorrectly understand the HTML5 standard, because this part is difficult to implement.
There was a problem hiding this comment.
I tried copying the content of the tests in the following file:
<!DOCTYPE html>
<html>
<body>
<![CDATA[just some plain text]]><hr>
<![CDATA[<!-- not a comment -->]]><hr>
<![CDATA[¬-an-entity-ref;]]><hr>
<![CDATA[<not a='start tag'>]]><hr>
<![CDATA[]]><hr>
<![CDATA[[[I have many brackets]]]]><hr>
<![CDATA[I have a > in the middle]]><hr>
<![CDATA[I have a ]] in the middle]]><hr>
<![CDATA[] ]>]]><hr>
<![CDATA[]] >]]><hr>
<![CDATA[
if (a < b && a > b) {
printf("[<marquee>How?</marquee>]");
}
]]><hr>
</body>
</html>and this was the result on Firefox:
<html><head></head><body>
<!--[CDATA[just some plain text]]--><hr>
<!--[CDATA[<!-- not a comment ---->]]><hr>
<!--[CDATA[¬-an-entity-ref;]]--><hr>
<!--[CDATA[<not a='start tag'-->]]><hr>
<!--[CDATA[]]--><hr>
<!--[CDATA[[[I have many brackets]]]]--><hr>
<!--[CDATA[I have a --> in the middle]]><hr>
<!--[CDATA[I have a ]] in the middle]]--><hr>
<!--[CDATA[] ]-->]]><hr>
<!--[CDATA[]] -->]]><hr>
<!--[CDATA[
if (a < b && a --> b) {
printf("[<marquee>How?</marquee>]");
}
]]><hr>
</body></html>
There was a problem hiding this comment.
Yes, and if you try <svg><text y="100"><![CDATA[foo<br>bar]]></text></svg>, you will see that content between <![CDATA[ and ]]> is interpreted as a raw data.
This is context dependent.
HTMLParser is actually just a tokenizer. To determine the context automatically, it needs to support the stack of open elements and to know what elements are in the HTML namespace. This is all in the specification, and we will implement this in future. But this is a different level of complexity. So I solved the issue by letting the user to determine the context. New method support_cdata() sets how HTMLParser will parse CDATA. This is not good, but perhaps better than the current state.
Lib/html/parser.py
Outdated
| j = rawdata.find(']]>') | ||
| if j < 0: | ||
| return -1 | ||
| self.unknown_decl(rawdata[i+3: j]) |
There was a problem hiding this comment.
| self.unknown_decl(rawdata[i+3: j]) | |
| self.unknown_decl(rawdata[i+3:j]) |
Lib/html/parser.py
Outdated
| j = rawdata.find(']]>') | ||
| if j < 0: | ||
| return -1 | ||
| self.unknown_decl(rawdata[i+3: j]) | ||
| return j + 3 |
There was a problem hiding this comment.
I tried copying the content of the tests in the following file:
<!DOCTYPE html>
<html>
<body>
<![CDATA[just some plain text]]><hr>
<![CDATA[<!-- not a comment -->]]><hr>
<![CDATA[¬-an-entity-ref;]]><hr>
<![CDATA[<not a='start tag'>]]><hr>
<![CDATA[]]><hr>
<![CDATA[[[I have many brackets]]]]><hr>
<![CDATA[I have a > in the middle]]><hr>
<![CDATA[I have a ]] in the middle]]><hr>
<![CDATA[] ]>]]><hr>
<![CDATA[]] >]]><hr>
<![CDATA[
if (a < b && a > b) {
printf("[<marquee>How?</marquee>]");
}
]]><hr>
</body>
</html>and this was the result on Firefox:
<html><head></head><body>
<!--[CDATA[just some plain text]]--><hr>
<!--[CDATA[<!-- not a comment ---->]]><hr>
<!--[CDATA[¬-an-entity-ref;]]--><hr>
<!--[CDATA[<not a='start tag'-->]]><hr>
<!--[CDATA[]]--><hr>
<!--[CDATA[[[I have many brackets]]]]--><hr>
<!--[CDATA[I have a --> in the middle]]><hr>
<!--[CDATA[I have a ]] in the middle]]--><hr>
<!--[CDATA[] ]-->]]><hr>
<!--[CDATA[]] -->]]><hr>
<!--[CDATA[
if (a < b && a --> b) {
printf("[<marquee>How?</marquee>]");
}
]]><hr>
</body></html>
* Add HTMLParser.support_cdata().
"] ]>" and "]] >" no longer end the CDATA section.