Skip to content

Add Document.parseHTMLUnsafe / HTMLDocument.parseHTMLUnsafe and full DOM parsing coverage - #1121

Closed
rbri with Copilot wants to merge 2 commits into
masterfrom
copilot/add-document-parsehtmlunsafe-method
Closed

rbri with Copilot wants to merge 2 commits into
masterfrom
copilot/add-document-parsehtmlunsafe-method

Conversation

Copilot AI commented May 22, 2026 •

Copy link
Copy Markdown

This change adds support for the standards-defined static Document.parseHTMLUnsafe(html) API (also exposed via HTMLDocument) to parse full HTML documents without sanitization. It aligns HtmlUnit with modern browser behavior distinct from DOMParser.parseFromString(..., 'text/html') by exposing the constructor-level static entrypoint.

  • API surface: static parseHTMLUnsafe on Document

    • Added @JsxStaticFunction on Document:
      • parseHTMLUnsafe(Context, VarScope, Scriptable, Object[], Function)
    • Input handling:
      • no argument / undefined → ""
      • other values converted via JavaScriptEngine.toString(...)
    • Implementation reuses existing HTML document parsing pipeline via DOMParser.parseHtmlDocument(...).
  • API parity: static parseHTMLUnsafe on HTMLDocument

    • Added static @JsxStaticFunction on HTMLDocument delegating to Document.parseHTMLUnsafe(...).
    • Ensures static API visibility on both constructors.
  • New conformance-style test suite

    • Added ParseHTMLUnsafeTest with targeted coverage for:
      • static method existence on Document and HTMLDocument
      • return type ([object HTMLDocument])
      • empty / undefined / null argument behavior
      • head/body parsing (title, getElementById, querySelector)
      • unsafe semantics preservation (<script> element present, inline handler attribute preserved)
      • detached-document semantics (defaultView === null, parsed doc not equal to page document)
      • full-document input (<!DOCTYPE html>...) and typical element/attribute preservation (img[src])
@JsxStaticFunction
public static HTMLDocument parseHTMLUnsafe(final Context cx, final VarScope scope,
        final Scriptable thisObj, final Object[] args, final Function funObj) {
    final String html = args.length == 0 || JavaScriptEngine.isUndefined(args[0])
            ? ""
            : JavaScriptEngine.toString(args[0]);

    final Window win = getWindow(funObj);
    final WebWindow webWindow = win.getWebWindow();
    final WebResponse webResponse = new StringWebResponse(html, webWindow.getEnclosedPage().getUrl());
    return (HTMLDocument) DOMParser.parseHtmlDocument(win.getDocument(), webResponse, webWindow);
}

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • central.sonatype.com
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --enable-native-access=ALL-UNNAMED -classpath /usr/share/apache-maven-3.9.16/boot/plexus-classworlds-2.11.0.jar -Dclassworlds.conf=/usr/share/apache-maven-3.9.16/bin/m2.conf -Dmaven.home=/usr/share/apache-maven-3.9.16 -Dlibrary.jansi.path=/usr/share/apache-maven-3.9.16/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/REDACTED/work/htmlunit/htmlunit org.codehaus.plexus.classworlds.launcher.Launcher -Dtest=org.htmlunit.javascript.host.dom.DOMParserTest test (dns block)
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --enable-native-access=ALL-UNNAMED -classpath /usr/share/apache-maven-3.9.16/boot/plexus-classworlds-2.11.0.jar -Dclassworlds.conf=/usr/share/apache-maven-3.9.16/bin/m2.conf -Dmaven.home=/usr/share/apache-maven-3.9.16 -Dlibrary.jansi.path=/usr/share/apache-maven-3.9.16/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/REDACTED/work/htmlunit/htmlunit org.codehaus.plexus.classworlds.launcher.Launcher -Dtest=org.htmlunit.javascript.host.dom.ParseHTMLUnsafeTest test (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

Goal

Implement Document.parseHTMLUnsafe(html) as a static method on the Document class (and also on HTMLDocument) in HtmlUnit, and add a comprehensive test suite in DOMParserTest.java (or a new ParseHTMLUnsafeTest.java).


Background / Spec

Document.parseHTMLUnsafe(string) is defined in the HTML Living Standard §8.4 and DOM Parsing spec. It is a static method on Document that:

  1. Takes a string of HTML markup
  2. Parses it as a full HTML document (not a fragment)
  3. Returns a new Document object (an HTMLDocument)
  4. Does not sanitize the HTML (hence "Unsafe") — script content, event handlers, etc. are preserved in the parsed tree (though they are not executed)
  5. The returned document has no browsing context (not attached to any window)
  6. Unlike DOMParser.parseFromString(html, 'text/html'), this is a static method called directly on the Document constructor: Document.parseHTMLUnsafe("<html>...</html>")

Existing code to understand first

Before implementing, study these files carefully:

  • src/main/java/org/htmlunit/javascript/host/dom/DOMParser.java — this already implements parseFromString for text/html, using StringWebResponse + HtmlPage + HTMLParser. The new parseHTMLUnsafe should reuse the same parseHtmlDocument static helper.
  • src/main/java/org/htmlunit/javascript/host/dom/Document.java — this is where the new @JsxStaticFunction must be added.
  • src/main/java/org/htmlunit/javascript/host/html/HTMLDocument.java — check if parseHTMLUnsafe also needs to be placed here (browsers expose it on both Document and HTMLDocument).
  • src/main/java/org/htmlunit/javascript/host/URL.java — reference for @JsxStaticFunction pattern.
  • src/main/java/org/htmlunit/javascript/host/media/MediaSource.java — another simple @JsxStaticFunction example.
  • src/test/java/org/htmlunit/javascript/host/dom/DOMParserTest.java — existing test patterns to follow.

Implementation

1. Add parseHTMLUnsafe to Document.java

In src/main/java/org/htmlunit/javascript/host/dom/Document.java, add a static method:

/**
 * Parses the given string of HTML without sanitizing it and returns a new HTMLDocument.
 *
 * @param cx        the current context
 * @param scope     the scope
 * @param thisObj   the scriptable this object
 * @param args      the arguments
 * @param funObj    the function object
 * @return a newly created {@link HTMLDocument}
 *
 * @see <a href="/api/browser/proxy?url=https%3A%2F%2Fhtml.spec.whatwg.org%2Fmultipage%2Fdynamic-markup-insertion.html%23dom-parsehtmlunsafe">
 *     HTML spec - parseHTMLUnsafe</a>
 */
@JsxStaticFunction
public static HTMLDocument parseHTMLUnsafe(final Context cx, final VarScope scope,
        final Scriptable thisObj, final Object[] args, final Function funObj) {
    final String html = args.length == 0 || JavaScriptEngine.isUndefined(args[0])
            ? ""
            : JavaScriptEngine.toString(args[0]);

    final Window win = getWindow(funObj);
    // reuse DOMParser.parseHtmlDocument which already does the right thing
    try {
        final WebWindow webWindow = win.getWebWindow();
        final WebResponse webResponse = new StringWebResponse(html, webWindow.getEnclosedPage().getUrl());
        return (HTMLDocument) DOMParser.parseHtmlDocument(win.getDocument(), webResponse, webWindow);
    }
    catch (final IOException e) {
        throw JavaScriptEngine.reportRuntimeError("parseHTMLUnsafe failed: " + e.getMessage());
    }
}

Note: Check how DOMParser.parseHtmlDocument is called elsewhere — its first parameter is HtmlUnitScriptable scriptable. Pass win.getDocument() (which is an HtmlUnitScriptable) so the prototype/scope resolution works correctly.

Note: Also check whether parseHTMLUnsafe needs to live on HTMLDocument as well (browsers expose Document.parseHTMLUnsafe and HTMLDocument.parseHTMLUnsafe). If HTMLDocument extends Document in HtmlUnit's JS class hierarchy, inheriting the static may or may not work — check and add a separate @JsxStaticFunction override in HTMLDocument.java if needed, delegating to the same logic.

2. Required imports

Add whatever imports are needed in Document.java:

  • org.htmlunit.StringWebResponse
  • org.htmlunit.WebResponse
  • org.htmlunit.javascript.host.dom.DOMParser (if not already imported)
  • org.htmlunit.corejs.javascript.Context
  • org.htmlunit.corejs.javascript.Function
  • org.htmlunit.corejs.javascript.Scriptable
  • org.htmlunit.corejs.javascript.VarScope

Test suite

Create src/test/java/org/htmlunit/javascript/host/dom/ParseHTMLUnsafeTest.java extending WebDriverTestCase with the following tests. Follow the exact style of DOMParserTest.java (inline HTML string, LOG_TITLE_FUNCTION, loadPageVerifyTitle2).

Tests to ...

This pull request was created from Copilot chat.

Copilot AI changed the title [WIP] Implement Document.parseHTMLUnsafe static method Add Document.parseHTMLUnsafe / HTMLDocument.parseHTMLUnsafe and full DOM parsing coverage May 22, 2026
Copilot finished work on behalf of rbri May 22, 2026 12:21
Copilot AI requested a review from rbri May 22, 2026 12:21
@sonarqubecloud

Copy link
Copy Markdown

@rbri rbri closed this May 22, 2026
@rbri
rbri deleted the copilot/add-document-parsehtmlunsafe-method branch May 22, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants