#6491 [Other] Ray, an assistant and coding agent built on Strata: would any part of it help the app?
open · @DamirLukina88 · 0 comentarios · En GitHub
Setup & installServer & APIAMD / HIPModels & quantsLinux
Descripción
### Area
[Other] (anything else, including setup and installation)
### What do you want to do?
First, thank you for taking `GET /v1/models/{id}` in with #5477.
I keep a fork of Strata, https://github.com/DamirLukina88/D.L.-AI---Strata, with an assistant and coding agent called Ray on top of it. Ray was built for one PC and one household, so a lot of it is personal and much of it overlaps with what the app now does through the Harness. I am not asking you to merge the fork. I'd like to ask whether any of the parts below would be useful to Strata. If one is, I'd send it as a small, separate pull request in the app's own stack (Node/TypeScript) or as reference code, whichever you prefer.
The numbers below are measurements from the fork on my PC (RX 7600 XT 16 GB, Ryzen 5 3600, 32 GB RAM, CachyOS, ROCm 7.2), not from upstream builds.
1. **A shell command scanner for an approval gate.** `tools/dl_auditing.py` decides whether a shell command only reads or may change something, and it reads what simple pattern checks miss: heredoc bodies line by line, `$(...)`, `$'...'`, `<<<`, `>|`, line continuations, and spliced quotes such as `c''p` for `cp`. `tools/test_dl_auditing.py` has the cases. It might help the Manual-mode gate (`plugins/strata-gate.mjs`), which asks before anything it cannot prove read-only: the more commands it can prove read-only, the fewer Approve cards the user sees.
2. **MCP servers turned on only when a request needs them.** The agent sees each server as one short line and turns one on when needed. With 27 fixed prompts run three times each (`tools/ray_toolchoice_bench.py`): 36 of 36 runs that needed a server picked a right one, and the median time to the first text fell from 4.6 s to 3.1 s on the prompts that needed none, after the server list shrank from about 5.4k to 2.1k characters.
3. **Voice on the server.** `POST /v1/audio/transcriptions` runs whisper.cpp on the PC, and `POST /v1/audio/speech` runs Piper (0.39 s for 3.4 s of speech). The web page reads the answer aloud sentence by sentence while the model writes; on three spoken prompts the first word came 3 to 67 s sooner than waiting for the whole answer. The server-side transcription might help #4765, where the Linux app's WebKitGTK view has no speech recognition.
4. **An Android app and one shared session.** One agent session lives in a small daemon (`tools/dl_sessiond.py`). The web page, a terminal and an Android app are all clients of it, so a task started on the phone can be followed or continued in the terminal. Sessions are saved and can be rewound or branched.
5. **Undo and memory with jj.** Every round that changes files is committed to a jj history kept outside the project, so any agent edit can be undone, also in folders that are not git repositories, and the user's own git history is never touched. Separately, a small memory store (Markdown pages versioned with jj, searched with SQLite full-text search and an offline CPU embedding model) gives the agent project notes that last between sessions and that other agents can share through MCP.
6. **Engine robustness seen from the client.** If the engine fails during a turn, Ray waits for the model to reload and retries up to twice. When the engine started short of RAM (part of the experts read from disk), `/health` reports `degraded` and the server reloads it once it is idle and the memory is free. While a long prompt is read, the client shows "Reading the prompt… N s". The vision weights stay unloaded until a picture arrives (1.15 GiB of RAM saved here). Stop cancels at once, also while a long prompt is read or the request waits: 1.0 s, where it used to wait for the next 10 s heartbeat.
7. **Agent behaviour tuned for a slow local model.**
- An automatic thinking level: per-turn rules plus a CPU router (0.09 ms) choose none, low, medium or high, raise it when a tool call fails, and cap the reasoning tokens to match (checked with a 75-item set).
- A handoff instead of trimming: near the context limit the agent writes a short brief and reloads into a clean ~4k-token context, which cut the next prompt read from about 45–55 s to about 1.5 s.
- A thinking-loop guard, and a watchdog that nudges an agent that has gone 12 rounds without editing a file.
- Checks before the agent may stop: "tests pass" with no test run after the last edit is sent back, a headless browser check returns page console errors, and a fresh-context review reads the diff.
- A prompt sent while the agent works is queued and picked up at the next step instead of being lost.
8. **More voice.** Interim text while you speak, hands-free auto-send, a cleaner that turns Markdown into speakable text, plain spoken answers (no tables or code), and a wake lock with media controls on phones.
9. **Remote access with no open port**, through a tunnel. The Cloudflare Access login part is already in #5485.
Left out on purpose: my personal integrations (mail, smart home, phone sensors, login), and anything above that would belong to the Harness rather than Strata, if you prefer it there.
Which of these, if any, would you like to see first? A "no thanks" is a fine answer too.
### What do you do today instead?
Ray runs from my fork, which I keep in step with upstream by porting fixes by hand (last whole merge at 0.1.44.1; later fixes that apply to my PC are ported, up to 0.2.16).
En el sitio
Enlaces a install, modelos, releases.