Testing Methodology

Current test configuration and suite scope from application dev at 01c423c, checked on 11 October 2026.

Tools and Commands

The application uses Vitest, @solidjs/testing-library, jsdom, and @vitest/coverage-v8. vitest.config.ts sets the default environment to jsdom, uses the Solid plugin with hot reload disabled, resolves the ~ source alias, and enables browser/development resolve conditions.

Run these in the application repository, not the documentation repository:

npm test
npm test -- src/engine/grader.test.ts
npm run coverage
npm run build

Use existing relevant test-file selectors for a focused change before running broader suites. There is no root lint or typecheck script in the inspected application manifest; do not claim those scripts passed.

Automated Scope

Layer	Existing examples	What it protects
Engine	Simulator, computational engine, marker, parser, test parser, utilities, finalization, grader, trace	Tape semantics, outcomes/cutoffs, normalization and server grading
Compiler	Lexer, parser, analyzer, assembler, completion; machine-store integration	DSL diagnostics and compiler-to-store contracts
Components	DiagramView interaction tests, toolbar, CodeEditor, AIChatPanel, ProblemPanel, DailyGuessGrid, tapes/theme/tests	User-facing controls and interaction/state rendering
Data/domain	Problem repository, daily repository/authoring/clock, player cookie	Persistence shapes, generation, budgets, date rollover and player identity
Routes	Daily route tests and problem submit tests	Validation, public projection, errors and grading contracts
Stores	Problem-store tests and compiler/store integration	Loading and verdict/source state transitions

Do not describe the suite as only render-level: diagram and other components already have interaction tests. Conversely, jsdom tests do not establish production canvas rendering, browser CodeMirror selection, OAuth redirects, or multi-client live-room behavior.

API tests invoke exported route handlers with web-standard requests and inspect responses, with mocked infrastructure where appropriate. They are not tests of hypothetical public parse/run endpoints; those routes are absent from the current tree. Authenticated file/auth routes and dedicated Yjs deployments still need appropriately scoped integration/manual checks.

Coverage Interpretation

Coverage includes src/engine/**/*.ts and src/components/**/*.tsx. Exclusions include Editor, MachineThumbnail, TestModal, TestPanel, GBAFilter, InviteModal, compiler highlighting/AST/token definitions, and test files. Repository, route, and store suites can run without their files counting toward this coverage denominator.

Reporters are text, JSON summary, and HTML. No configured coverage thresholds were found. Do not carry forward a fixed historical test count or percentage: record the commit, command, environment, denominator and timestamp alongside any newly measured result.

The Gitea coverage workflow runs on main pushes and PRs targeting main, uses Node 24, and uploads coverage artifacts. A dev push alone does not prove that workflow executed.

Manual and Integration Regression Checklist

• Editor: valid/invalid source, diagram-to-code edits, port transitions, touch context actions, panning/zoom, Undo/Redo, variant changes, tape and tree.
• Tests: result/tape mismatches, cutoff boundary, multi-tape behavior, computational output, nondeterministic branches, complexity displays.
• Documents: guest login/restore, upload limits, owned saves/rename/delete, latest four versions, download and problem association.
• Sharing: pending invite, accept/decline/cancel, shared list, owner versus contributor controls and revocation.
• Collaboration: two browser clients, simultaneous code/test/position edits, awareness, reconnect/offline fallback, Yjs restart/replay/compaction, stable room path. Explicit file saves must remain distinct from room history.
• Problems: guest unrecorded verdict, signed-in attempts, author case visibility, hidden-detail projection, invalid-machine limits and publication.
• Daily: no scheduled day, black-box reserved inputs, probe/answer budget boundaries, build tests versus graded attempts, solved lock, timezone rollover, cookie persistence, history and reveal policies.
• Admin: 401/403 gates, invalid drafts, preview without write, hidden-rule reachability, calendar replacement and delete/reschedule semantics.
• Groq: mocked missing/invalid key, provider 429/502, model selection, source/message request shape, explicit apply and compiler feedback. Use mocks for automated runs; never send private source or real credentials to Groq as part of documentation validation.

Record observed failures rather than replacing them with assumed passes. Deployment checks require the actual database/storage/Yjs configuration; unit tests cannot certify missing migrations or external service availability.