Overview

Bring Your Character to Life.

CosNFX brings cosplay, LARP, and stage-magic characters to life: natural arm movements — a punch, a wrist flick, an overhead raise — fire high-fidelity sound effects and elemental energy bursts straight from the wrist. It runs as a standalone Wear OS app (no phone required to perform), tested on a Galaxy Watch 7.

Each character is an identity with its own colour, sound kit, energy field, and cinematic backdrop — 30 built-ins (heroes, monsters, and mages: Superhero, Cyborg, Pyromancer, Vampire, Reaper, Werewolf, Robot, and more) plus your own. From a phone browser you assign gestures, upload your own .wav/.mp3, chain gestures into combos, and route audio — with nothing to install on the phone. A lightweight phone companion app is optional, mainly so testers without a watch can still try the gestures.

Active beta   Kotlin + Jetpack Compose (Wear Material 3) · ~10,000 lines of Kotlin across the watch app and a phone companion · com.jdcap.cosnfx v2.8.0. Free to install, with an optional one-time Pro in-app purchase.

At A Glance

Watch Core, Phone Mission Control

The watch is the impact core; the phone is the configuration surface and an optional second speaker. A single Android foreground service (HeroService) owns the audio engine, the sensors, and an embedded web server, so they survive the screen sleeping or the app backgrounding.

Watch

HeroService + HeroSynth

A foreground service hosts the self-healing AudioTrack engine, gravity/accel/gyro sensor fusion, and the measured-axis gesture detectors. The watch UI is one page — a Clean HUD: the identity name is the arm toggle, ringed by numbered 1–7 gesture nodes over a cinematic per-identity backdrop.

Phone

Mission Control (8 tabs)

The watch runs an embedded Ktor server on :8080; any phone browser on the same Wi-Fi opens an 8-tab panel — Gestures, Combos, Look, Combat, Library, Props, Calibrate, System — over REST + a WebSocket event stream, gated by a 6-digit pairing PIN shown on the watch. Nothing to install on the phone.

Visuals

Per-Identity Energy Fields

Each identity renders a distinct transparent Canvas energy field (~19 hand-built effects) over its cinematic backdrop, plus audio-reactive elemental bursts per gesture — all frame-throttled so they pause when the screen is off.

Audio Engineering

Low-Latency by Design.

Standard Android media players add 40–100 ms of round-trip latency — unacceptable for a sound triggered by a physical wrist snap. CosNFX bypasses them and streams raw 16-bit PCM straight into a hardware AudioTrack on a MAX_PRIORITY thread, requesting PERFORMANCE_MODE_LOW_LATENCY (the fast HAL path, with a fallback build for devices that reject it).

The engine is self-healing: build, play, and render are wrapped in a retry loop that releases and rebuilds the track after a Bluetooth/audio-focus change (which used to throw uncaught and kill the process). Writes use 256-sample chunks (~6 ms — latency negligible, far fewer underruns). To save power the track pauses when idle, but that pause is arm-gated: it stays warm (~2.5 s) while armed so every gesture in a performance fires instantly, and pauses fast (250 ms) when disarmed.

Sensor Fusion

Measured-Axis Gesture Detection

The hard problem is firing on a real gesture and never on natural conversation. The honest fix was to stop trusting the textbook: on this Galaxy Watch 7 the sensor axes aren't what the docs assume, so each gesture is keyed to the axis it actually drives — read from a live on-watch inspector.

Measured axes

One Axis Per Gesture

Gravity X is the arm up/down axis (gravity Y stays ~0 in every pose), gating the armed state and the overhead Raise. The two wrist flicks (Underhand / Overhand) read gyro X; the thrust and impulse moves — a forward Strike, a superhero-landing down-punch, a hip thrust, and a finger Snap — each key off the accel axis they actually drive, picked by which signal dominates so they can't trip each other.

Anti-cross-talk

Reorientation Suppression

A fast vertical sweep of the arm (Δgravity X > 1.0 — i.e. you're raising it) mutes the twitchy accel detectors for 400 ms, so the swing itself can't trip a thrust. All lanes share a 350 ms global cooldown — one gesture per motion — and a hands-free double-tap arms or disarms above all of it.

Calibration

Teach the Watch Your Motion.

Fixed axes fit how the prop sits on this wrist and how this performer moves; a very different pose can drift. So calibration is built in: record a built-in impulse gesture a few times and the watch auto-aims that gesture's detector at the channel your motion actually drives — fitting detection to you instead of a hard-coded axis.

It's per-gesture (Calibrate / Redo), with Recalibrate-all and Reset-to-defaults, and it's an opt-in backup, never a regression: an uncalibrated gesture keeps its built-in detection unchanged, and an under-sampled capture is rejected rather than saved.

Props & Output

From the Wrist to the Room

A gesture doesn't have to stop at sound — it can also fire an external action and be routed across devices.

Props

External Lights & Devices

An opt-in per-gesture HTTP action (method + URL + body) fires alongside the sound — driving anything with a local web API: WLED strips, Philips Hue, Home Assistant (smoke machines, fans), DIY ESP32/Arduino, or IFTTT. Each row has a Test button. It's device-agnostic — you supply the URL — and scoped in code to private LAN addresses (anything public is forced to HTTPS).

Audio routing

Three-Way Routing & Combat Link

A main/support mix routes independently across three outputs — the watch, the phone companion app, and the Mission Control browser tab — so the main hit can hit a loud phone speaker while a support layer stays on the wrist (any mix, or silence either). An opt-in Combat Link relays strike / shield / damage events between paired watches for co-op or versus play.

Engineering Reality: Battery vs. Latency

A performance prop has to stay instant without draining the watch mid-show. Sensors run full-rate only while armed and downshift when disarmed; the audio track stays warm while armed and pauses fast when not. The screen (kept on while armed) is the real drain — so the watch auto-disarms after ~15 minutes idle (any gesture resets it), letting the screen sleep and the watch fall to its idle floor. CosNFX is in active beta development.