Add HTML-in-Canvas APIs #11588

foolip · 2025-08-21T12:51:56Z

Handwavy things that need fleshing out are marked with 👋

At least two implementers are interested (and none opposed):
- Chromium
- …
Tests are written and can be reviewed and commented upon at:
- …
Implementation bugs are filed:
- Chromium: https://issues.chromium.org/u/1/hotlists/6783002
- Gecko: …
- WebKit: …
- Deno (only for timers, structured clone, base64 utils, channel messaging, module resolution, web workers, and web storage): …
- Node.js (only for timers, structured clone, base64 utils, channel messaging, and module resolution): …
Corresponding HTML AAM & ARIA in HTML issues & PRs:
MDN issue is filed: …
The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

/canvas.html ( diff )
/index.html ( diff )
/indices.html ( diff )
/infrastructure.html ( diff )
/references.html ( diff )
/rendering.html ( diff )

Handwavy things that need fleshing out are marked with 👋

Kaiido

Glad to see this being worked on, thanks.

Not quite sure how much discussion should be held at this stage. So to note, this doesn't seem to fully match the latest state of https://github.com/WICG/html-in-canvas. e.g. the rename to drawHTMLElement. The layoutsubtree attribute is also missing along with the implications to the existing fallback contents.
Still, thanks for making this move.

source

foolip · 2025-08-29T09:47:43Z

@Kaiido thank you for the review! I've fleshed things out more, renaming to drawHTMLElement() (because drawElement() will probably not be workable in WebGL) and adding the layoutsubtree attribute.

There's still some handwaving going on of course, in particular what causes the subtree to be laid out but not painted.

foolip · 2025-09-02T20:20:35Z

source

+  <span>represents</span> <span>embedded content</span> and has a <code
+  data-x="attr-canvas-layoutsubtree">layoutsubtree</code> attribute specified is additionally
+  expected to be treated as 👋replaced element with subtree layout👋, where children are laid out
+  but not rendered.</p>


Should also say that each child is laid out as if it's the only child.

Should also put contain: layout in the UA style sheet, in that case maybe don't need isolation: isolate.

foolip · 2025-09-17T16:21:14Z

I've fleshed this out some more now, in particular the hit testing.

Kaiido

One common complain with the use of dictionaries in the Canvas2D API is that this makes GC kick in very often during animations which has a non-negligible performance cost.
This API shape makes a big use of such dictionaries with one for the wrapper CanvasElementHitTestRegion and then a nested one for the CanvasHitTestRect, and I guess there will be scenarios where multiple of these will need to be updated at every frame. Since the values are copied over from the passed objects to new internal objects, it's unclear if even a careful author, who would try to reuse the same objects, could avoid GC at all here.

On the other hand, I really like how this API shape enables future additions like using a Path2D, or even a bitmap mask, instead of a CanvasHitTestRect. (btw can we bikeshed on rect for that purpose?)

It's not my area of expertise, but would an actual exposed interface allow for non copy from JS, so that authors can just update the regions instead of setting new ones?

source

Kaiido · 2025-09-23T01:32:28Z

source

+
+  <ol>
+   <li>
+    <p>For each <span>hit test region</span> <var>region</var> in <var>canvas</var>'s <span>hit test regions</span>:</p>


Definitely an edge case, and I suppose it's a bit of a gray area (see whatwg/infra#396) but how is this supposed to work if setHitTestRegions is called during this iteration? E.g.

// Add multiple regions ctx.setHitTestRegions([ { element, rect: { x, y } }, { element: anotherElement, rect: { x: anotherX, y: anotherY } } ]); element.onclick = e => ctx.setHitTestRegions([]); // that was a 'once' handler

Should the anotherElement still perform the hit-test?

This kind of problem is sometimes handled in the spec by making a frozen copy of the thing to iterate before starting iteration. Do you think that'd be OK here?

That would certainly be clearer as to what's supposed to happen yes. Now, whether it's the best behavior or not, I don't know and don't have any strong opinion. Both possibilities might come surprising depending on the case. The fact that the timing of hit-testing w.r.t. events propagation isn't well defined doesn't help...

@szager made a good point here which is that really when we start dispatching events hit testing is already done. So rather than making a copy of the list here, the spec here needs to make clear how the list is used in hit testing and that it all happens before event dispatch.

source

progers · 2025-12-04T23:12:07Z

source


  <pre><code class="css">@namespace "http://www.w3.org/1999/xhtml";

+canvas[layoutsubtree] > * { isolation: isolate !important; contain: strict !important; }


I think we just need contain: paint rather than contain: strict.

I'll change that. Are there observable differences that we should write tests for? In particular for layout containment, can the children of the canvas element now have layout interactions with each other?

No interaction between the children of the canvas. If you have strict containment and don't specify a size (e.g., something like <div style="contain: strict;">hello world</div>), the text won't paint, so this is an example where the behavior difference would be visible.

source

progers · 2025-12-05T00:23:29Z

source

+    let transform = ctx.drawElementImage(leftElm, x * devicePixelRatio, y * devicePixelRatio);
+    leftElm.style.transform = transform;
+
+    // purple circle goes in between the elements


I worry this example may be considered an anti-pattern because the purple area isn't hit testable. We could use canvas 2d drawing apis to draw an interesting path that includes both boxes and is painted as the background and which is hard to do with regular css?

Right, something like a peanut shape behind both elements would be hard to do with CSS even if we have border shaping, I'm guessing. I haven't done this just yet, but let me know if peanuts or peas in a pod or something along those lines makes sense to you.

WDYT of this example? This draws a simple pie chart and uses drawElementImage for the labels.

<!doctype html> <style> canvas { width: 250px; height: 250px; } canvas > div { text-align: center; max-width: 40%;} </style> <canvas layoutsubtree="true" role="list" aria-label="Example Pie Chart"> <div role="listitem" tabindex="0" data-val="0.45" data-color="tomato"> <b>45%</b><br>Apple - a crowd favorite. </div> <div role="listitem" tabindex="0" data-val="0.35" data-color="cornflowerblue"> <b>35%</b><br>Blueberry - great for summer. </div> <div role="listitem" tabindex="0" data-val="0.20" data-color="gold"> <b>20%</b><br>Durian - our newest flavor. </div> </canvas> <script> const canvas = document.querySelector('canvas'); const ctx = canvas.getContext('2d'); function drawPieChart() { // 1. Center the coordinate system. const radius = Math.min(canvas.width, canvas.height) / 2; ctx.translate(radius, radius); let angle = 0; for (const label of canvas.children) { const slice = Number(label.dataset.val) * Math.PI * 2; // 2. Draw the wedge. ctx.fillStyle = label.dataset.color; ctx.beginPath(); ctx.moveTo(0, 0); ctx.arc(0, 0, radius, angle, angle + slice); ctx.fill(); // 3. Draw the label element, and update its transform. const mid = angle + slice / 2; const label_width = label.offsetWidth * devicePixelRatio; const label_height = label.offsetHeight * devicePixelRatio; const x = Math.cos(mid) * radius * 0.60 - label_width / 2; const y = Math.sin(mid) * radius * 0.60 - label_height / 2; let transform = ctx.drawElementImage(label, x, y); label.style.transform = transform; angle += slice; } } new ResizeObserver(([entry]) => { // Size the canvas to the device pixel content box. const box = entry.devicePixelContentBoxSize[0]; canvas.width = box.inlineSize; canvas.height = box.blockSize; ctx.reset(); drawPieChart(); }).observe(canvas, {box: ['device-pixel-content-box'], fireOnEveryPaint: true}); </script>

foolip · 2025-12-05T12:35:25Z

Thanks for the review @progers! I've left two comments unresolved where I'd like more feedback.

annevk · 2025-12-08T08:30:47Z

source

+  the same <span>natural dimensions</span> as the element. A <code>canvas</code> element that
+  <span>represents</span> <span>embedded content</span> and has a <code
+  data-x="attr-canvas-layoutsubtree">layoutsubtree</code> attribute specified is additionally
+  <span>expected</span> to be treated as 👋replaced element with subtree layout👋, where children are


We need address the issue that replaced elements generally don't have children. Video does, I think, and canvas can but they are not rendered. I believe that's what we're trying to get at here.

I used lots of 👋 for handwavy things originally, and this one remains. The issue is that https://drafts.csswg.org/css-display/#replaced-element says:

The content of replaced elements is not considered in the CSS formatting model

Something needs to change to cause the children to be laid out and be hit testable, and the approaches I could see are:

Define a new subcategory of replaced elements in CSS, "replaced element with subtree layout" by some other name.

Define it as something other than a replaced element in HTML, something more akin to a div of a certain size where the canvas bitmap is the "background" and the subtree is laid out and hit tested as usual, but is not visible.

@progers it's been a while since we discussed this last, have any other ideas popped up?

annevk · 2025-12-08T08:31:14Z

source

  undefined <span data-x="dom-canvas-toBlob">toBlob</span>(<span>BlobCallback</span> _callback, optional DOMString type = "image/png", optional any quality);
  <span>OffscreenCanvas</span> <span data-x="dom-canvas-transferControlToOffscreen">transferControlToOffscreen</span>();
+
+  [<span>CEReactions</span>] attribute boolean <span data-x="dom-canvas-layoutSubtree">layoutSubtree</span>;


Why doesn't this use Reflect?

I can, fixed. Do you know if there's a good reason that width and height don't? If not, I can send a separate PR to change those too.

source

annevk · 2025-12-08T08:34:35Z

source

+  <p>When drawing elements to a <code>canvas</code>, no information should be used that isn't
+  otherwise observable to author code. Such <dfn export>sensitive information</dfn> includes
+  but isn't limited to:</p>


I think we actually have to spell out how this works, in particular for form controls where this can also impact their sizing and such, but also in general it seems we can do better than this.

annevk · 2025-12-08T08:37:16Z

source

+
+  <p>Descendants of a <code>canvas</code> element with the <code
+  data-x="attr-canvas-layoutsubtree">layoutsubtree</code> attribute specified participate in hit
+  testing.</p>


This also seems like it needs more detail. In particular how user agents probably have to reorder the nodes or some such to make that possible.

That's a good point that we had not considered. i.e that the paint order should somehow match the DOM order for hit testing to work as expected.

schenney-chromium · 2025-12-09T13:56:43Z

source

+  <p><code>HTMLCanvasElement</code>'s <dfn method for="HTMLCanvasElement"><code
+  data-x="dom-canvas-getElementTransform">getElementTransform(<var>element</var>,
+  <var>drawTransform</var>)</code></dfn> method steps are to <span>get element transform</span>
+  with <span>this</span>, <var>element</var>, and <var>drawTransform</var>.</p>


I guess this works for any element, but do we want to restrict it to direct children of the canvas, using the same definition we use for drawElementImage. I suspect it would make the intent clearer.

schenney-chromium · 2025-12-09T13:57:40Z

source

  <span>TextMetrics</span> <span data-x="dom-context-2d-measureText">measureText</span>(DOMString text);
 };

+interface mixin <dfn interface>CanvasDrawElementImage</dfn> {


Need to add the source rect version that just landed in Chromium.

schenney-chromium · 2025-12-09T14:08:39Z

source

+   <li>
+    <p>Paint <var>element</var> to the specified rectangular area, ignoring <var>element</var>'s
+    <span>transformation matrix</span> and without using any <span>sensitive information</span>.
+    Instead, either paint nothing or use static information that is the same for all users.</p>


The "paint nothing" etc should be more clearly tied to the "sensitive information" if possible. Can we link to a description of "sensitive information".

schenney-chromium · 2025-12-09T14:10:59Z

source

+   <li><p>Let <var>T<sub>draw</sub></var> be a copy of the <span>current transformation
+   matrix</span>.</p></li>
+
+   <li><p>Translate <var>T<sub>draw</sub></var> by (<var>dx</var>, <var>dy</var>).</p></li>


Maybe this uses language consistent with the existing spec, in which case fine. But translating a matrix is not really a valid term. "Compose a translate" is more proper in my experience.

schenney-chromium · 2025-12-09T14:11:12Z

source

+   <li><p>Translate <var>T<sub>draw</sub></var> by (<var>dx</var>, <var>dy</var>).</p></li>
+
+   <li><p>Scale <var>T<sub>draw</sub></var> by (<var>dw</var> / <var>layoutWidth</var>,
+   <var>dh</var> / <var>layoutHeight</var>).</p></li>


schenney-chromium · 2025-12-09T14:11:40Z

source

+   <li><p>Scale <var>T<sub>draw</sub></var> by (<var>dw</var> / <var>layoutWidth</var>,
+   <var>dh</var> / <var>layoutHeight</var>).</p></li>
+
+   <li><p>Return the result of <span>get element transform</span> with <var>context</var>'s


"get element transform" needs a link.

schenney-chromium · 2025-12-09T14:13:05Z

source

+
+  <p>Descendants of a <code>canvas</code> element with the <code
+  data-x="attr-canvas-layoutsubtree">layoutsubtree</code> attribute specified participate in hit
+  testing.</p>


That's a good point that we had not considered. i.e that the paint order should somehow match the DOM order for hit testing to work as expected.

schenney-chromium · 2025-12-09T14:15:45Z

source

+  the same <span>natural dimensions</span> as the element. A <code>canvas</code> element that
+  <span>represents</span> <span>embedded content</span> and has a <code
+  data-x="attr-canvas-layoutsubtree">layoutsubtree</code> attribute specified is additionally
+  <span>expected</span> to be treated as 👋replaced element with subtree layout👋, where children are


We need address the issue that replaced elements generally don't have children. Video does, I think, and canvas can but they are not rendered. I believe that's what we're trying to get at here.

Stub out canvas.drawElement()

e181de1

Handwavy things that need fleshing out are marked with 👋

foolip marked this pull request as draft August 21, 2025 12:52

Kaiido reviewed Aug 25, 2025

View reviewed changes

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

source Outdated Show resolved Hide resolved

foolip mentioned this pull request Aug 28, 2025

Drawing only part of an element (cropping) WICG/html-in-canvas#37

Open

foolip added 5 commits August 28, 2025 14:25

Fix typ: y->h

8634ebe

Modernize the style somewhat

afc4a4d

Replace handwavy check with real check and drop "update the rendering"

f151c83

Add the layoutsubtree attribute

74d67ae

Rename to drawHTMLElement()

f7d320a

foolip changed the title ~~Stub out canvas.drawElement()~~ Stub out canvas.drawHTMLElement() Aug 29, 2025

foolip mentioned this pull request Sep 2, 2025

HTML-in-Canvas #10650

Open

Fix the layoutsubtree UA style sheet

3cad6d4

foolip commented Sep 2, 2025

View reviewed changes

foolip changed the title ~~Stub out canvas.drawHTMLElement()~~ Stub out canvas.drawElementImage() Sep 16, 2025

foolip added 6 commits September 16, 2025 17:06

Rename to drawElementImage()

4367aac

Define sensitive information

84ec8e6

Expose children to AT

f351c3d

grammar

aed994e

Use containment to make children containing blocks

8c3ce46

Define setHitTestRegions()

ff066c9

foolip changed the title ~~Stub out canvas.drawElementImage()~~ Add HTML-in-Canvas APIs Sep 17, 2025

Kaiido reviewed Sep 23, 2025

View reviewed changes

foolip commented Sep 30, 2025

View reviewed changes

source Outdated Show resolved Hide resolved

foolip added 5 commits October 7, 2025 15:26

Merge remote-tracking branch 'origin/main' into foolip/html-in-canvas

5e3c3d0

Wrap algorithms

f8f119a

Fix typo and reserved word

202adae

Add a note about not using CTM

be22310

Fix reference to hit test regions

9866034

foolip added 2 commits October 7, 2025 17:14

Skip non-positive width/height regions

4e559a9

Use CSS border box

4838669

foolip mentioned this pull request Oct 22, 2025

WHATUP at TPAC 2025: agenda suggestions #11711

Closed

foolip added 5 commits November 11, 2025 12:19

Replace hit testing API with a simpler model

9fa39e9

Add a hit testing example

f79ed95

Add getElementTransform() and related algorithms

0a243be

Update drawElementImage() to return matrix

1f71b9a

Express the transformations in a more approachable way

5aeb03b

progers reviewed Dec 5, 2025

View reviewed changes

Address progers feedback

951bdee

<data> -> <code>

ba5c560

foolip marked this pull request as ready for review December 5, 2025 13:21

annevk reviewed Dec 8, 2025

View reviewed changes

schenney-chromium reviewed Dec 9, 2025

View reviewed changes

foolip added 2 commits December 9, 2025 15:56

Use [Reflect] for layoutSubtree

4d77e94

Link DOMMatrix

d033976


		<pre><code class="css">@namespace "http://www.w3.org/1999/xhtml";

		canvas[layoutsubtree] > * { isolation: isolate !important; contain: strict !important; }

Add HTML-in-Canvas APIs #11588

Are you sure you want to change the base?

Add HTML-in-Canvas APIs #11588

Uh oh!

Conversation

foolip commented Aug 21, 2025 • edited by pr-preview bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kaiido left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

foolip commented Aug 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

foolip commented Sep 17, 2025

Uh oh!

Kaiido left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

progers Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

foolip commented Dec 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

foolip commented Aug 21, 2025 •

edited by pr-preview bot

Loading

Kaiido left a comment •

edited

Loading

progers Dec 6, 2025 •

edited

Loading