#5081 [GUI] let the side panel's page preview use pointer lock

closed · @storm-ace · 0 Kommentare · Auf GitHub

Setup & installAMD / HIPSecurityWindowsLinux

Beschreibung

### Problem and result

A web page that the agent writes opens in the side panel's preview. That preview is an iframe with `sandbox="allow-scripts"`. A page that captures the mouse calls `requestPointerLock()`, for example a first-person game with "click to play". In that frame the call is refused with a `SecurityError`, so the game never starts in the desktop app. The same file works when it is opened in a browser tab.

This adds `allow-pointer-lock` to that one sandbox. The frame keeps its opaque origin: no same-origin access, forms, popups or top navigation. Pointer lock only starts after a click in the frame, and Esc releases it.

### Related discussion

I searched the open and closed issues and pull requests for "pointer lock" and "allow-pointer-lock" and found nothing that covers this.

### Approach

- `apps/host/src/components/ArtifactPanel.tsx`: `sandbox="allow-scripts"` becomes `sandbox="allow-scripts allow-pointer-lock"` on the website preview iframe. Nothing else changes.
- `apps/host/tests/artifact-preview-sandbox.test.mjs` (new): checks that this iframe's sandbox has exactly `allow-scripts` and `allow-pointer-lock`. A wider sandbox would fail the test as well.
- The Time-skip compare frames (`HTML_PREVIEW_SANDBOX`) are not changed. They show two versions side by side for comparison, not for play.

### Validation

Environment: Windows 11 Pro 10.0.26200, Node from the shipped app, this branch (upstream `ffa80d73` plus this commit).

**Passed**

- The new test fails on `ffa80d73` (`actual: [ 'allow-scripts' ]`, `expected: [ 'allow-pointer-lock', 'allow-scripts' ]`) and passes with the change.
- `npm run typecheck` and `npm run build` in `apps/host`.
- A standalone page in Chromium with two `srcdoc` frames, each with a button that calls `document.body.requestPointerLock()`:
  - the frame with `sandbox="allow-scripts"` gets `SecurityError` and `pointerlockerror`;
  - the frame with `sandbox="allow-scripts allow-pointer-lock"` gets past the sandbox check. In that run the browser window was in the background, so it then got `WrongDocumentError` (no focus), not a lock.

- The desktop app on Windows (Radeon 8060S), with the same one-word change made in its installed bundle: a three.js first-person game in the side panel now captures the mouse after "click to play" and plays. Before the change, the click did nothing.

**Failed**

- `npm test` in `apps/host`: 1,346 passed, 108 skipped, 3 failed. All 3 are outside this change:
  - `new hosts create the default; custom folders and legacy weights remain in place` (`model-directory-default.test.mjs`)
  - `#4783 the composer acts on the choice, …` (`ui-do.test.mjs`)
  - `directory API protects loaded models, active requests and downloads, …` (`model-directory.test.mjs`)

  Running those three files again on `ffa80d73` without this change, the first two fail the same way; the third passed that time.

**Not run**
- macOS and Linux app builds. The attribute is standard HTML and WebKit supports it, but I did not try it there.
- `apps/desktop` (Tauri) was not rebuilt; the change is only in the host's web UI.

### Limits

- Fullscreen (`allowfullscreen`) is not added. It was not needed for the case I hit, and it is a larger change to what a preview may do.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Mehr auf der Site

Links zu Install, Modellen, Releases.