Audio: Beat Tap Rhythm

A live rhythm-scoring demo. The script imports a short looped BGM (bgm_loop.mp3), feeds it to a BeatDetector built through APJS.AudioDetectionModule.getOrCreateAudioDetectionBuilder, and judges every screen tap against the closest unjudged beat — PERFECT inside ±150 ms, GOOD inside ±300 ms, MISS otherwise. A cyan indicator pulses on each detected beat as a visual cue, score and combo update in a 2D HUD, and a one-shot pop SFX fires on each successful hit.

What you'll build

• Two Audio Player SceneObjects, each owning an AudioComponent — BgmPlayer (looping bgm_loop.mp3) and HitSfxPlayer (one-shot sfx_pop.mp3). Both audio files are imported via import_source from the workspace's local CC0 game-asset library.
• A 2D HUD with three Text labels: ScoreText ("Score: N"), ComboText ("Combo: xN (max N)"), and JudgeText ("PERFECT" / "GOOD" / "MISS").
• A cyan BeatIndicator Screen Image at the center that scales up to 1.4× on each detected beat and lerps back over ~250 ms — a visual metronome.
• A GameController empty SceneObject hosting BeatTapScoring. The script builds the BeatDetector in onInit(), polls detector.getResult() every frame, queues each new beat with a timestamp, judges the closest unjudged beat on each tap, and auto-misses any beat past the GOOD window.

Open the demo

↓ beat-tap-rhythm.zip

Unzip and open in Effect House (5.9.0+). The opening scene contains:

• Camera — default 3D perspective camera, untouched (renders the camera feed).
• 2D Camera — auto-created when the first 2D Text was added. Parents the entire HUD.
• BgmPlayer — Audio Player SceneObject. Its AudioComponent has audioRef wired to the imported bgm_loop.mp3 AudioAsset, with playMode: "Infinity" and volume: 60. autoPlay is off — the script calls bgm.play() explicitly in onStart() so playback starts in lock-step with the BeatDetector.
• HitSfxPlayer — second Audio Player. AudioComponent wired to sfx_pop.mp3 with playMode: "Once" and volume: 90. The script calls sfx.play() on every successful tap; play() alone (never stop()-then-play()) restarts the sample cleanly on rapid hits.
• ScoreText / ComboText / JudgeText — three 2D Text labels positioned at top-center, just below, and lower-center.
• BeatIndicator — 220×220 Screen Image, cyan, anchored at (0, 80). Scaled by the script.
• GameController — empty SceneObject hosting BeatTapScoring.

Read the scripts

BeatTapScoring.ts

@component()
export class BeatTapScoring extends APJS.BasicScriptComponent {
  // Audio host SceneObjects — each owns an AudioComponent that gets looked up
  // at runtime. Component types don't register on @serializeProperty in APJS,
  // so we wire SceneObjects and call getComponent("ClassName") in onInit/onStart.
  @serializeProperty bgmPlayer!: APJS.SceneObject;
  @serializeProperty hitSfxPlayer!: APJS.SceneObject;

  // 2D Text labels for the live HUD.
  @serializeProperty scoreText!: APJS.SceneObject;
  @serializeProperty comboText!: APJS.SceneObject;
  @serializeProperty judgeText!: APJS.SceneObject;

  // The cyan square that pulses on each detected beat.
  @serializeProperty beatIndicator!: APJS.SceneObject;

  // Tunables — adjust per game feel.
  @serializeProperty perfectWindowSec: number = 0.15;
  @serializeProperty goodWindowSec: number = 0.30;
  @serializeProperty perfectPoints: number = 100;
  @serializeProperty goodPoints: number = 50;

  private bgm: APJS.AudioComponent | null = null;
  private sfx: APJS.AudioComponent | null = null;
  private detector: APJS.BeatDetector | null = null;
  private scoreComp: APJS.Text | null = null;
  private comboComp: APJS.Text | null = null;
  private judgeComp: APJS.Text | null = null;
  private indicatorST: APJS.ScreenTransform | null = null;

  private touchCallback!: (event: APJS.IEvent) => void;
  private elapsed: number = 0;
  private lastResult: number = -999;
  private pendingBeats: { time: number; beat: number; judged: boolean }[] = [];
  private score: number = 0;
  private combo: number = 0;
  private maxCombo: number = 0;
  private lastJudgmentLabel: string = "";
  private indicatorPulse: number = 0; // 0..1, 1 = just pulsed

  // BeatDetector MUST be built in onInit — builder.build() returns null elsewhere.
  onInit(): void {
    if (!this.bgmPlayer) return;
    this.bgm = this.bgmPlayer.getComponent("AudioComponent") as APJS.AudioComponent;
    if (!this.bgm) return;

    const builder = APJS.AudioDetectionModule
      .getOrCreateAudioDetectionBuilder(APJS.AudioDetectionType.Beat) as APJS.BeatDetectorBuilder | null;
    if (!builder) return;
    builder.setDetectorSource(APJS.AudioSourceType.ExternalFile, this.bgm);
    this.detector = builder.build();
  }

  onStart(): void {
    if (this.scoreText) this.scoreComp = this.scoreText.getComponent("Text") as APJS.Text;
    if (this.comboText) this.comboComp = this.comboText.getComponent("Text") as APJS.Text;
    if (this.judgeText) this.judgeComp = this.judgeText.getComponent("Text") as APJS.Text;
    if (this.beatIndicator) {
      this.indicatorST = this.beatIndicator.getComponent("ScreenTransform") as APJS.ScreenTransform;
    }
    if (this.hitSfxPlayer) {
      this.sfx = this.hitSfxPlayer.getComponent("AudioComponent") as APJS.AudioComponent;
    }

    if (this.bgm) this.bgm.play();

    this.touchCallback = (event: APJS.IEvent) => {
      const t = event.args[0] as APJS.TouchData;
      if (t.phase !== APJS.TouchPhase.Began) return;
      this.handleTap();
    };
    APJS.EventManager.getGlobalEmitter()
      .on(APJS.EventType.Touch, this.touchCallback, this);
    console.log("[BeatTapScoring] ready — listening for beats");
  }

  onDestroy(): void {
    if (this.touchCallback) {
      APJS.EventManager.getGlobalEmitter()
        .off(APJS.EventType.Touch, this.touchCallback, this);
    }
  }

  onUpdate(dt: number): void {
    this.elapsed += dt;

    // Poll the detector. result is the beat number (1-4 in 4/4 time,
    // -1 between beats).
    if (this.detector) {
      const result = this.detector.getResult();
      if (result !== this.lastResult && result > 0) {
        this.pendingBeats.push({ time: this.elapsed, beat: result, judged: false });
        this.indicatorPulse = 1;
      }
      this.lastResult = result;
    }

    // Auto-miss any unjudged beat past the GOOD window.
    for (const b of this.pendingBeats) {
      if (!b.judged && this.elapsed - b.time > this.goodWindowSec) {
        b.judged = true;
        this.combo = 0;
        this.lastJudgmentLabel = "MISS";
      }
    }
    if (this.pendingBeats.length > 50) {
      this.pendingBeats = this.pendingBeats.filter(b => this.elapsed - b.time < 5);
    }

    // Decay the indicator pulse and apply it to the indicator's scale.
    this.indicatorPulse = Math.max(0, this.indicatorPulse - dt * 4);
    if (this.indicatorST) {
      const s = 1 + this.indicatorPulse * 0.4;
      this.indicatorST.scale = new APJS.Vector2f(s, s);
    }

    this.refreshHUD();
  }

  private handleTap(): void {
    let closestBeat: { time: number; beat: number; judged: boolean } | null = null;
    let closestDelta = 999;
    for (const b of this.pendingBeats) {
      if (b.judged) continue;
      const delta = Math.abs(this.elapsed - b.time);
      if (delta < closestDelta && delta < this.goodWindowSec) {
        closestDelta = delta;
        closestBeat = b;
      }
    }
    if (closestBeat) {
      closestBeat.judged = true;
      const isPerfect = closestDelta < this.perfectWindowSec;
      this.score += isPerfect ? this.perfectPoints : this.goodPoints;
      this.combo += 1;
      if (this.combo > this.maxCombo) this.maxCombo = this.combo;
      this.lastJudgmentLabel = isPerfect ? "PERFECT" : "GOOD";
      // play() alone restarts the SFX cleanly on rapid taps —
      // never stop()-then-play().
      if (this.sfx) this.sfx.play();
    } else {
      this.combo = 0;
      this.lastJudgmentLabel = "MISS-tap";
    }
  }

  private refreshHUD(): void {
    if (this.scoreComp) this.scoreComp.text = "Score: " + this.score;
    if (this.comboComp) {
      this.comboComp.text = "Combo: x" + this.combo + "  (max " + this.maxCombo + ")";
    }
    if (this.judgeComp && this.lastJudgmentLabel) {
      this.judgeComp.text = this.lastJudgmentLabel;
    }
  }
}

The Audio-namespace calls of interest:

• APJS.AudioDetectionModule.getOrCreateAudioDetectionBuilder(AudioDetectionType.Beat) is the gateway to the seven detector types (Beat, Pitch, Onset, Spectrum, Volume, SoundEvent, Keyword). The BeatDetectorBuilder returned exposes a builder for the most common rhythm-game case. A common gotcha: the JSDoc in APJS.d.ts calls it getAudioDetectionBuilder — the actual export is getOrCreateAudioDetectionBuilder (declared in AudioComponent.d.ts). Using the JSDoc'd name fails compilation.
• builder.setDetectorSource(AudioSourceType.ExternalFile, audioComponent) binds the detector to a specific AudioComponent clip (verified working in TTEH preview). The other source types are Microphone (no-op in preview), and Music (the device's currently-playing music — strategic for TikTok, unverified in preview).
• builder.build() returns a usable BeatDetector only when called inside onInit(). Calling it in onStart() or onUpdate() silently returns null. The script's structure mirrors this — every AudioComponent / Text / ScreenTransform lookup happens in onStart() or later, but the detector is constructed in onInit() so the builder slot is honored.
• detector.getResult() returns the current beat number (1–4 in 4/4 time) when a beat is active, or -1 between beats. The script detects a new beat by comparing the result against lastResult — any change to a positive value is a fresh beat.
• AudioComponent.play() / .pause() / .resume() / .stop() — standard playback control. Volume is 0–100 (set via DSL or audio.volume = N). For rapid-fire SFX, call play() alone — it restarts the sample. stop()-then-play() in the same frame collapses to "do nothing" and the SFX never fires.
• AudioComponent.playMode — "Once" (default), "Loop" (with loopCount), or "Infinity" (loop forever). Set on the BgmPlayer's AudioComponent in the editor; the script doesn't need to touch it.
• @serializeProperty constraint: SceneObject and asset-resource types persist; AudioComponent and AudioAsset references must be wired through the host SceneObject + runtime getComponent(). That's why the script wires bgmPlayer: APJS.SceneObject instead of bgm: APJS.AudioComponent.

Customize

On GameController → BeatTapScoring:

• perfectWindowSec (default 0.15) — half-width of the PERFECT judge window. Tighten to 0.08 for a hard-mode game; widen to 0.25 for casual.
• goodWindowSec (default 0.30) — half-width of the GOOD window. Beats outside this window auto-miss and reset the combo.
• perfectPoints / goodPoints — score values; the default 100 / 50 2× ratio reads cleanly on the HUD.

On the BgmPlayer AudioComponent:

• audioRef — drag in any imported AudioAsset (must be MP3). The detector keys off whatever clip is wired here, so changing the BGM gives a new beat pattern automatically.
• volume — 0–100; 60 is the default.
• playMode — keep "Infinity" for a continuous loop; switch to "Once" for a single-pass demo that ends at the audio's end.

On the HitSfxPlayer AudioComponent:

• audioRef — pick any short percussive .mp3. The local game-asset library has sfx_click.mp3, sfx_pop.mp3, sfx_success.mp3, sfx_fail.mp3.

Suggestions for further play:

• Spawn a falling note Screen Image on each detected beat and require the player to tap when it crosses a target line. Use Vector3f.lerp (see the Math tutorial) to drive the note's ScreenTransform.anchoredPosition from spawn to target.
• Switch the detector to AudioDetectionType.Onset for a non-quantized rhythm trigger — useful for percussive tracks where the 4/4 beat assumption doesn't hold.
• Trade the bundled BGM for AudioSourceType.Music (the device's currently-playing music) so the effect adapts to whichever song the TikTok creator picks. Note that Music source behavior in TTEH preview is unverified — test on-device.
• Add a sfx_fail.mp3 AudioPlayer and play it on MISS-tap / auto-miss for negative feedback.

What you learned

This tutorial used:

• AudioComponent — play, playMode: "Infinity"/"Once", volume, runtime lookup via getComponent("AudioComponent").
• AudioAsset — imported via edit_by_dsl/import_source from the local game-asset library (returns the AudioAsset; no host SceneObject is auto-created — pair it with an Audio Player builtin object or a manually-attached AudioComponent).
• AudioDetectionModule.getOrCreateAudioDetectionBuilder — the factory entry point for all 7 detector types; remember the getOrCreate prefix (the JSDoc'd getAudioDetectionBuilder doesn't exist).
• BeatDetectorBuilder — setDetectorSource(AudioSourceType, audioComponent)
  • build() (must run in onInit()).
• AudioSourceType.ExternalFile — the source type for a bundled AudioComponent clip; Microphone and Music are alternatives.
• AudioDetectionType.Beat — the detector type; siblings are Pitch, Onset, Spectrum, Volume, SoundEvent, Keyword.
• BeatDetector.getResult() polled per frame; returns the current beat number (1–4) or -1 between beats.
• 2D Text + ScreenTransform for the HUD; Text.text, fontSize, color; ScreenTransform.scale for the indicator pulse.

Read the full AudioComponent reference, the AudioAsset reference, the AudioDetectionModule reference, the AudioDetectionType reference, the AudioSourceType reference, the BeatDetector reference, the BeatDetectorBuilder reference, the BaseAudioDetector reference, the AudioDetectorBuilder reference, and the Audio namespace overview.