Semantic Telemetry
Semantic Telemetry is a real-time measurement system for conversational meaning. It answers the question: "Is this conversation going somewhere meaningful?"
Core Metrics
SGI — Semantic Grounding Index
Measures how well the assistant's response semantically grounds with the user's query.
- Range: 0.0 to 4.0+
- Coherent region: 0.5 to 2.0
- Formula:
cos_sim(user_embedding, assistant_embedding) / mean_embedding_norm
| SGI Value | Interpretation |
|---|---|
| < 0.7 | Drifting from user's topic |
| 0.8 - 1.5 | Healthy grounding (coherent) |
| > 2.0 | Question-focused (assistant asking clarifying questions) |
Orbital Velocity
Rate of semantic movement in embedding space — how fast the conversation is changing direction.
- Range: 0 to 180 degrees
- Coherent region: 0 to 45 degrees
- Formula:
arccos(cos_sim(msg_t, msg_{t-1})) * (180 / pi)
| Velocity | Interpretation |
|---|---|
| < 25 degrees | Stable, on-topic |
| 25 - 45 degrees | Moderate exploration (still coherent) |
| > 45 degrees | Chaotic movement, topic jumping |
Coherence Region
A conversation enters the Coherence Region when:
- SGI is between 0.5 and 2.0
- Velocity is at or below 45 degrees
Conversations that enter and remain in this region show genuine transformational depth. This is the central finding from the S64 research papers.
How It Works
- Each message pair (user + assistant) is embedded using a vector model (Nomic, OpenAI, Cohere, etc.)
- The SDK computes SGI and velocity from the embedding vectors
- Results are returned in real-time (~500ms per ingestion)
- Aggregate statistics build a "conversational profile" over time
Try It
- Semantic Telemetry App — Have a live conversation or load a research baseline to see metrics in action
- SDK Quickstart — Integrate telemetry into your own application
Research Baselines
The S64 validation suite includes 10 research baselines:
| ID | Name | Category |
|---|---|---|
| B01 | Surface Deception | Synthetic |
| B02 | Implicit Transformation | Synthetic |
| B03 | Rapid Oscillation | Synthetic |
| B04 | Stuck States | Synthetic |
| B05 | Nested Complexity | Synthetic |
| B06 | Outcome Ambiguity | Synthetic |
| B07 | Failed Transformation | Synthetic |
| B08 | False Completion | Synthetic |
| B09 | Self-Discovery (Human) | Naturalistic |
| B10 | Self-Discovery (AI) | Naturalistic |
These baselines cover a spectrum of conversational patterns, from surface-level exchanges to deep transformational dialogues, and are available in the Semantic Telemetry App's "Load Conversation" mode.
SDK Documentation
For technical integration details, see the SDK Reference.