testing_map

Testing Map

This page maps tin test layers to the confidence they provide. Use it to choose the smallest useful test during development and the right gate before review.

Test Layers

Layer	What it proves	Where to look
HSM authoring tests	State/event syntax, transition selection, hierarchy, history, guards, actions	test/hsm_authoring_semantics_test.cpp
Runtime integration tests	Queues, channels, actors, tasks, timers, dispatch, resource accounting	test/framework_integration_test.cpp
Tooling tests	CLI import/export, YAML/JSON IR, schema, generated artifacts	test/CMakeLists.txt, test/tsm_tool*
Compile-fail tests	Invalid API usage is rejected at compile time	test/CMakeLists.txt
Package consumer tests	Installed/facade targets compile for users	test/package_*, package CMake tests
Platform smoke tests	Bare-metal, FreeRTOS, Zephyr, or other platform-facing examples compile against target APIs	tools/check_profile_compile.py, platform examples
Example run tests	Example binaries still build and execute on host	examples/*/CMakeLists.txt
Evidence and trace tests	Specs, traces, and generated evidence still match the implementation	spec, tools
Performance harness	Size and latency claims are measured on the documented host/toolchain	tools/performance_harness.cpp, docs/performance_roadmap.md

Common Commands

Build and run the normal host gate:

cmake -S . -B build/host -DCMAKE_BUILD_TYPE=Debug
cmake --build build/host
ctest --test-dir build/host --output-on-failure

Run focused host checks:

./build/host/bin/tsm_test [hsm_authoring]
./build/host/bin/tsm_test [runtime]
./build/host/bin/tsm_test [embedded]
ctest --test-dir build/host -L tooling --output-on-failure

Run platform smoke checks when toolchains and workspaces are installed:

export TSM_PLATFORM_ROOT="$HOME/opt/tin-platforms"
ctest --test-dir build/host -L platform --output-on-failure

The platform runner searches $TSM_PLATFORM_ROOT first and accepts explicit overrides such as TSM_FREERTOS_KERNEL_SOURCE and ZEPHYR_BASE.

Coverage By Change Type

Change	Minimum useful coverage
New transition semantics	HSM authoring test plus docs for the semantic rule
New runtime primitive behavior	Runtime integration test plus resource accounting check
New actor or channel behavior	Runtime integration test and a focused guide/example update
New coroutine wait or executor behavior	Runtime/task test covering wakeup, cancellation, and resource impact
New platform profile	Compile smoke test and target-shaped example update
New YAML/JSON IR shape	Schema/tooling test, generated artifact smoke, docs update
New public facade header or target	Package consumer compile test
New performance claim	Re-run the harness and document what was measured

What Platform Smoke Means

Platform smoke coverage proves that target-facing examples and headers compile against the selected platform APIs. Host unit and integration tests cover the primitive behavior.

For example, a FreeRTOS smoke compile can prove that freertos_storage_tag still names the right API surface. A runtime integration test should prove what happens when a bounded queue accepts, rejects, drops, or dispatches work.

What To Report In A Review

When handing off a change, report:

• the focused test you ran first;
• the full local gate result;
• any platform smoke result or unavailable dependency;
• any resource, performance, or generated artifact change;
• any remaining risk that is not covered by automation.