speech-habit-lens

Two commands

1. shl analyze speech.wav --out report.md Recognize the audio, extract ESAS, run three Claude layers, and write a grounded Markdown report — about 3–4 minutes end to end.
2. shl serve Launch the Streamlit UI at localhost:8501 with in-browser mic recording, Plotly time series and cross-layer cards.
3. --model claude-opus-4-7 Swap the analysis model per run: Sonnet 4.6 (default), Opus 4.7 for the toughest cross patterns, Haiku 4.5 to cut cost.

Who it’s for

• Speakers & founders audit the pitch you are about to give — before the room is watching
• Product managers find the sentence where your update stops landing, then fix that one
• Coaches & educators ground feedback in measured evidence instead of "you sounded a bit flat"
• Curious engineers see your own speech habits the way an instrument sees them, in one minute of audio

When to reach for it

• Rehearsing a pitch or a talk: hear the opening habit before it goes live
• After a demo that fell flat: find the moment the voice left the words behind
• Coaching a speaker: replace impressions with a time-stamped, parameter-cited report
• Auditing your own delivery: a monthly reading of the habits you did not know you had

Install

git clone https://github.com/kenimo49/speech-habit-lens
cd speech-habit-lens
pip install -e .

cp .env.example .env   # AMIVOICE_API_KEY + ANTHROPIC_API_KEY

Needs an AmiVoice API key (free up to 60 minutes per month) and an Anthropic API key. One analysis costs roughly ¥18 of Claude Sonnet usage — around 60 runs per month for ¥1,080.

The three layers

• Acoustic layer: ESAS 20 parameters over time — tension curve, intonation bias, opening energy, ending fade
• Text layer: fillers, conclusion position, repeated words, logic flow from the recognized transcript
• Cross layer: body–language correlations like "vocal energy drops right before the conclusion"
• Grounding Rule: every claim must cite time, parameter and quote, which suppresses hallucinated observations
• ESAS is language-independent: acoustic habits surface sharply even when text recognition is noisy
• All 20 ESAS field names verified against the live API — the official docs list only 5 examples
• Streamlit UI: in-browser mic recording, interactive Plotly time series, expandable cross-layer cards, Markdown report download

Measured across 4 pitches

Same tool, four 60-second speech clips from Jobs, Son (SoftBank), Nishino and Ochiai. Numbers below are the peak values printed by the tool.

A single indicator does not tell you who is a great speaker: Jobs’ peak energy is 21, Son’s is 44, and both land. The tool surfaces where each speaker’s habit lives — Nishino’s five "chotto" fillers, Son’s pitch delivered entirely from the front, Ochiai’s recognition confidence dropping to 0.70 while his acoustic pattern stays intense. The point is not the score; it is the specific habit named on the timeline.

Full 4-pitch case study →

The two moves that matter most

These are the two things the tool routinely surfaces that impressions do not.

Usage

# analyze: recognize → ESAS → Claude × 3 layers → Markdown report
shl analyze examples/sample.wav --out report.md

# pick the Claude model per run
shl analyze examples/sample.wav --model claude-opus-4-7 --out report.md

# browser UI with mic recording at localhost:8501
shl serve

Web UI

shl serve starts a Streamlit UI at localhost:8501: record from the browser mic or drop a .wav, read the ESAS time series interactively in Plotly, open the cross-layer cards, and download the Markdown report. API keys are read server-side from .env and never reach the browser.

What real runs return

Three patterns the tool has surfaced on real speeches. Every line ships with time + parameter + quote.

• Stallman opener (0.0s, anticipation=65, energy=2, confidence=15): maximum anticipation with minimum vocal energy. A run-up that never leaves the ground — the specific second that reads as "he seemed off" in impressions.
• intensive_thinking=62 × energy=3: the moments of hardest thinking coincide with the lowest vocal output. Present in every long-form talk the tool has been run against so far.
• passionate=0 while fluent: words flow without pause, ESAS passionate stays at zero. Fluent but flat — a common failure mode of well-rehearsed pitches, only visible on the cross layer.

The tool is a mirror, not a coach. It names the habit on the timeline; what you do with it is yours.

v0.2, built for Zennfes Spring 2026. Recommended clip length 30–90 seconds (60 minutes/month is the free AmiVoice tier). Analysis runs about 3–4 minutes end to end. The ESAS parameter semantics are reverse-engineered from measurements plus the official parameter list — treat the cross-layer findings as coaching hints, not clinical measurements.

Why this tool exists

Most delivery feedback lives at the level of impression: "you seemed nervous", "the closing felt weak", "you lost me in the middle". By the time the impression is put into words, the specific second that produced it is gone. The three layers here exist to hold on to that second: the acoustic layer measures it whether you recognized the words or not, the text layer records what was actually said, and the cross layer aligns them so a habit becomes a citation instead of a vibe. Debug the delivery like you would debug code.

Alongside Whisper and pyAudioAnalysis

Different jobs. Whisper transcribes; pyAudioAnalysis classifies acoustic frames. speech-habit-lens is the layer on top: it aligns a per-parameter emotional time series with the transcript and asks an LLM to name the habits, under a strict citation rule.

A dash means the tool does not offer that surface out of the box. speech-habit-lens sits on top of a transcript engine and an acoustic signal (AmiVoice ESAS in this build) — swap either upstream layer and the cross-layer analysis still runs.

FAQ

About the author

Built by Ken Imoto: 300+ technical articles across Zenn, Qiita, Dev.to and this site, 40+ books in 4 languages, 400K+ pageviews on Zenn and Qiita, 4 research papers on Zenodo, and creator of the LLMO Framework.

More about Ken →