mirror of
https://dev.azure.com/globalhealthx/EMR/_git/helix-engage
synced 2026-04-14 20:12:25 +00:00
docs: barge/whisper/listen design spec
SIP-only supervisor barge with DTMF mode switching (4=listen, 5=whisper, 6=barge). Live monitor split layout with context panel. Agent indicator on whisper/barge only. Auto-disconnect on call end. Dynamic SIP from Ozonetel pool. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
385
docs/superpowers/specs/2026-04-12-barge-whisper-listen-design.md
Normal file
385
docs/superpowers/specs/2026-04-12-barge-whisper-listen-design.md
Normal file
@@ -0,0 +1,385 @@
|
|||||||
|
# Supervisor Barge / Whisper / Listen — Design Spec
|
||||||
|
|
||||||
|
**Date:** 2026-04-12
|
||||||
|
**Branch:** `feature/barge-whisper`
|
||||||
|
**Prereq:** QA validates barge flow in Ozonetel's own admin UI first
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Enable supervisors to monitor and intervene in live agent calls directly from Helix Engage's live monitor. Three modes: **Listen** (silent), **Whisper** (agent hears supervisor, patient doesn't), **Barge** (both hear supervisor). Supervisor connects via SIP WebRTC in the browser. Mode switching via DTMF tones.
|
||||||
|
|
||||||
|
## Design Decisions
|
||||||
|
|
||||||
|
| Decision | Choice | Rationale |
|
||||||
|
|----------|--------|-----------|
|
||||||
|
| Connection method | SIP only (PSTN later) | Supervisors are already on browser with headset |
|
||||||
|
| Agent indicator | Whisper/barge only (listen is silent) | Spec says show indicator; listen should be undetectable |
|
||||||
|
| SIP number | Dynamic from Ozonetel pool (apiId 139) | No need to pre-assign per supervisor. 3 SIP IDs available. |
|
||||||
|
| Barge UI location | Live monitor + context panel + barge controls | Supervisor needs call context to intervene effectively |
|
||||||
|
| Access control | Any admin can barge any agent | Flat RBAC, no team hierarchy |
|
||||||
|
| Call end behavior | Auto-disconnect supervisor | No orphaned sessions |
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────┐
|
||||||
|
│ Supervisor Browser │
|
||||||
|
│ │
|
||||||
|
│ ┌──────────────┐ ┌────────────────────────────────┐ │
|
||||||
|
│ │ Live Monitor │ │ Context Panel + Barge Controls│ │
|
||||||
|
│ │ │ │ │ │
|
||||||
|
│ │ Agent list │ │ Patient summary / AI insight │ │
|
||||||
|
│ │ Active calls │──│ Appointments / Recent calls │ │
|
||||||
|
│ │ Click → │ │ ─────────────────────────────│ │
|
||||||
|
│ │ │ │ [Connect] │ │
|
||||||
|
│ │ │ │ [Listen] [Whisper] [Barge] │ │
|
||||||
|
│ │ │ │ [Hang up] │ │
|
||||||
|
│ └──────────────┘ └────────────────────────────────┘ │
|
||||||
|
│ │ │ │
|
||||||
|
│ │ poll /active-calls │ SIP WebRTC (kSip) │
|
||||||
|
│ │ every 5s │ DTMF 4/5/6 │
|
||||||
|
└─────────┼───────────────────────┼────────────────────────┘
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
┌─────────────────────┐ ┌──────────────────────────┐
|
||||||
|
│ Sidecar │ │ Ozonetel SIP Gateway │
|
||||||
|
│ │ │ (blr-pub-rtc4.ozonetel) │
|
||||||
|
│ POST /api/supervisor│ │ │
|
||||||
|
│ /barge │ │ SIP INVITE → supervisor │
|
||||||
|
│ /barge-mode │ │ audio mixing │
|
||||||
|
│ │ │ DTMF routing │
|
||||||
|
│ → Ozonetel admin API│ └──────────────────────────┘
|
||||||
|
│ dashboardApi │
|
||||||
|
│ apiId 63, 139 │
|
||||||
|
└─────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────┐
|
||||||
|
│ Ozonetel Cloud │
|
||||||
|
│ api.cloudagent. │
|
||||||
|
│ ozonetel.com │
|
||||||
|
│ │
|
||||||
|
│ /dashboardApi/ │
|
||||||
|
│ monitor/api │
|
||||||
|
│ apiId 63 → barge │
|
||||||
|
│ apiId 139 → SIP# │
|
||||||
|
│ /auth/login → JWT │
|
||||||
|
└─────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Components
|
||||||
|
|
||||||
|
### 1. Sidecar — Ozonetel Admin Auth Service
|
||||||
|
|
||||||
|
**New file:** `src/ozonetel/ozonetel-admin-auth.service.ts`
|
||||||
|
|
||||||
|
Manages a persistent Ozonetel admin session for supervisor APIs. Credentials from TelephonyConfig.
|
||||||
|
|
||||||
|
**Config extension** (`telephony.defaults.ts`):
|
||||||
|
```typescript
|
||||||
|
ozonetel: {
|
||||||
|
// ...existing fields
|
||||||
|
adminUsername: string; // NEW
|
||||||
|
adminPassword: string; // NEW
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. On startup, read `adminUsername` + `adminPassword` from TelephonyConfig
|
||||||
|
2. `GET /api/auth/public-key` → `{ publicKey, keyId }`
|
||||||
|
3. RSA-encrypt credentials using `jsencrypt`
|
||||||
|
4. `POST /auth/login` → JWT token
|
||||||
|
5. Cache token in memory, decode expiry via `jwt-decode`
|
||||||
|
6. Auto-refresh before expiry
|
||||||
|
7. Expose `getAuthHeaders()` for other services
|
||||||
|
|
||||||
|
**Auth headers for all admin API calls:**
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
'Authorization': `Bearer ${jwt}`,
|
||||||
|
'userId': userId,
|
||||||
|
'userName': userName,
|
||||||
|
'isSuperAdmin': 'true',
|
||||||
|
'dAccessType': 'false'
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Sidecar — Supervisor Barge Endpoints
|
||||||
|
|
||||||
|
**New file:** `src/supervisor/supervisor-barge.controller.ts`
|
||||||
|
|
||||||
|
Three endpoints proxying to Ozonetel admin API:
|
||||||
|
|
||||||
|
#### `POST /api/supervisor/barge`
|
||||||
|
|
||||||
|
Initiates barge-in on an active call.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Request
|
||||||
|
{ ucid: string, agentNumber: string }
|
||||||
|
|
||||||
|
// Sidecar calls:
|
||||||
|
POST https://api.cloudagent.ozonetel.com/dashboardApi/monitor/api
|
||||||
|
{
|
||||||
|
apiId: 63,
|
||||||
|
ucid: "<ucid>",
|
||||||
|
action: "CALL_BARGEIN",
|
||||||
|
isSip: true,
|
||||||
|
phoneno: "<dynamic SIP number from pool>",
|
||||||
|
agentNumber: "<agent phone>",
|
||||||
|
cbURL: "<sidecar hostname>"
|
||||||
|
}
|
||||||
|
|
||||||
|
// Response
|
||||||
|
{ status: "success", sipNumber: "19810", sipPassword: "19810", sipDomain: "blr-sbc1.ozonetel.com", sipPort: "442" }
|
||||||
|
```
|
||||||
|
|
||||||
|
Before calling barge, fetches an available SIP number:
|
||||||
|
|
||||||
|
#### `GET /api/supervisor/barge/sip-credentials`
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Sidecar calls:
|
||||||
|
POST https://api.cloudagent.ozonetel.com/ca-admin-Api/CloudAgentAPI/endpoint/sipnumber/sipSubscribe
|
||||||
|
{ apiId: 139, sipURL: "<sip gateway>" }
|
||||||
|
|
||||||
|
// Response
|
||||||
|
{ sip_number: "19810", password: "19810", pop_location: "blr-sbc1.ozonetel.com" }
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `POST /api/supervisor/barge/end`
|
||||||
|
|
||||||
|
Cleanup: disconnect SIP, clear Redis tracking.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Request
|
||||||
|
{ agentId: string, sipId: string }
|
||||||
|
|
||||||
|
// Sidecar calls:
|
||||||
|
POST https://api.cloudagent.ozonetel.com/dashboardApi/monitor/api
|
||||||
|
{ apiId: 158, Action: "delete", AgentId: "<agentId>", Sip: "<sipId>" }
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Frontend — Supervisor SIP Client
|
||||||
|
|
||||||
|
**New file:** `src/lib/supervisor-sip-client.ts`
|
||||||
|
|
||||||
|
Lightweight SIP client for supervisor barge sessions. Modeled on Ozonetel's `kSip.tsx` — separate from the agent's `sip-client.ts`.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
type SupervisorSipClient = {
|
||||||
|
init(domain: string, port: string, number: string, password: string): void;
|
||||||
|
register(): void;
|
||||||
|
isRegistered(): boolean;
|
||||||
|
isCallActive(): boolean;
|
||||||
|
sendDTMF(digit: string): void; // "4"=listen, "5"=whisper, "6"=barge
|
||||||
|
hangup(): void;
|
||||||
|
close(): void;
|
||||||
|
on(event: string, callback: Function): void;
|
||||||
|
off(event: string, callback: Function): void;
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
**Events emitted:**
|
||||||
|
- `registered` — SIP registration successful
|
||||||
|
- `registrationFailed` — SIP registration error
|
||||||
|
- `callReceived` — incoming call from Ozonetel (auto-answer)
|
||||||
|
- `callConnected` — barge session active
|
||||||
|
- `callEnded` — call terminated (agent hung up or supervisor hung up)
|
||||||
|
|
||||||
|
**Audio:** Remote audio plays through a hidden `<audio>` element (same pattern as agent SIP). Supervisor's microphone is captured via `getUserMedia`.
|
||||||
|
|
||||||
|
**DTMF mode mapping:**
|
||||||
|
- `"4"` → Listen (supervisor hears all, nobody hears supervisor)
|
||||||
|
- `"5"` → Whisper/Training (agent hears supervisor, patient doesn't)
|
||||||
|
- `"6"` → Barge (both hear supervisor)
|
||||||
|
|
||||||
|
### 4. Frontend — Live Monitor Redesign
|
||||||
|
|
||||||
|
**Modified file:** `src/pages/live-monitor.tsx`
|
||||||
|
|
||||||
|
Current: full-width table with disabled barge buttons.
|
||||||
|
New: split layout — call list on the left, context panel + barge controls on the right.
|
||||||
|
|
||||||
|
**Layout:**
|
||||||
|
```
|
||||||
|
┌─────────────────────────────┬──────────────────────────────┐
|
||||||
|
│ Active Calls (left, 60%) │ Context + Barge (right, 40%)│
|
||||||
|
│ │ │
|
||||||
|
│ ┌─ KPI cards ────────────┐ │ (nothing selected) │
|
||||||
|
│ │ Active: 3 Hold: 1 │ │ "Select a call to monitor" │
|
||||||
|
│ └────────────────────────┘ │ │
|
||||||
|
│ │ ── OR ── │
|
||||||
|
│ ┌─ Table ────────────────┐ │ │
|
||||||
|
│ │ Agent Caller Type Dur│ │ ┌─ Patient Summary ───────┐ │
|
||||||
|
│ │ rekha +9180.. In 2:34│ │ │ Name / Phone / Type │ │
|
||||||
|
│ │ ▶ selected row │ │ │ AI Insight │ │
|
||||||
|
│ │ ganesh +9199.. Out 0:45│ │ │ Appointments │ │
|
||||||
|
│ └────────────────────────┘ │ │ Recent calls │ │
|
||||||
|
│ │ └─────────────────────────┘ │
|
||||||
|
│ │ │
|
||||||
|
│ │ ┌─ Barge Controls ───────┐ │
|
||||||
|
│ │ │ [Connect] │ │
|
||||||
|
│ │ │ │ │
|
||||||
|
│ │ │ (after connect:) │ │
|
||||||
|
│ │ │ [Listen] [Whisper] [Barge]│
|
||||||
|
│ │ │ status: Connected 1:23 │ │
|
||||||
|
│ │ │ [Hang up] │ │
|
||||||
|
│ │ └─────────────────────────┘ │
|
||||||
|
└─────────────────────────────┴──────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**Selection flow:**
|
||||||
|
1. Supervisor clicks a call row → row highlights
|
||||||
|
2. Right panel populates with caller context (fetched from platform via lead phone match)
|
||||||
|
3. "Connect" button becomes active
|
||||||
|
4. Click Connect → sidecar fetches SIP credentials → calls barge API → supervisor SIP client registers → auto-answers incoming call
|
||||||
|
5. Status: CONNECTING → CONNECTED
|
||||||
|
6. Mode tabs appear: Listen (default) / Whisper / Barge
|
||||||
|
7. Tab click sends DTMF tone via supervisor SIP client
|
||||||
|
8. Hang up → disconnect SIP, clean up, right panel resets
|
||||||
|
|
||||||
|
### 5. Frontend — Agent Barge Indicator
|
||||||
|
|
||||||
|
**Modified file:** `src/components/call-desk/active-call-card.tsx`
|
||||||
|
|
||||||
|
When supervisor switches to whisper or barge mode, the agent sees an indicator.
|
||||||
|
|
||||||
|
**Detection:** The sidecar's supervisor service emits SSE events. Add a new event type:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// New SSE event from /api/supervisor/agent-state/stream
|
||||||
|
{ state: "supervisor-whisper", timestamp: "..." }
|
||||||
|
{ state: "supervisor-barge", timestamp: "..." }
|
||||||
|
{ state: "supervisor-left", timestamp: "..." }
|
||||||
|
```
|
||||||
|
|
||||||
|
**UI:** Small badge on the active call card:
|
||||||
|
- Whisper mode: "Supervisor coaching" badge (blue)
|
||||||
|
- Barge mode: "Supervisor on call" badge (brand)
|
||||||
|
- Listen mode: no indicator (silent)
|
||||||
|
|
||||||
|
**Implementation:** The sidecar tracks barge state per agent. When a supervisor connects and switches mode, the sidecar emits the appropriate SSE event to the agent's stream. The agent's `use-agent-state.ts` hook picks it up and sets a Recoil atom. The `active-call-card.tsx` renders the badge conditionally.
|
||||||
|
|
||||||
|
### 6. Sidecar — Barge State Tracking
|
||||||
|
|
||||||
|
**Modified file:** `src/supervisor/supervisor.service.ts`
|
||||||
|
|
||||||
|
Track which supervisor is barged into which agent, and in what mode.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
type BargeSession = {
|
||||||
|
supervisorId: string;
|
||||||
|
agentId: string;
|
||||||
|
sipNumber: string;
|
||||||
|
mode: 'listen' | 'whisper' | 'barge';
|
||||||
|
startedAt: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
// In-memory map (single sidecar per hospital)
|
||||||
|
private readonly bargeSessions = new Map<string, BargeSession>();
|
||||||
|
```
|
||||||
|
|
||||||
|
When mode changes, emit SSE event to the agent:
|
||||||
|
- `listen` → no event (silent)
|
||||||
|
- `whisper` → emit `supervisor-whisper` to agent's SSE stream
|
||||||
|
- `barge` → emit `supervisor-barge` to agent's SSE stream
|
||||||
|
- disconnect → emit `supervisor-left` to agent's SSE stream
|
||||||
|
|
||||||
|
**New endpoint for mode update:**
|
||||||
|
```typescript
|
||||||
|
POST /api/supervisor/barge/mode
|
||||||
|
{ agentId: string, mode: "listen" | "whisper" | "barge" }
|
||||||
|
```
|
||||||
|
|
||||||
|
This updates the in-memory session and emits the SSE event. The actual audio routing happens via DTMF on the SIP connection (frontend handles that).
|
||||||
|
|
||||||
|
## Data Flow — Full Barge Sequence
|
||||||
|
|
||||||
|
```
|
||||||
|
1. Supervisor clicks call row in live monitor
|
||||||
|
└→ Frontend fetches caller context from platform (lead by phone match)
|
||||||
|
└→ Right panel shows patient summary
|
||||||
|
|
||||||
|
2. Supervisor clicks "Connect"
|
||||||
|
└→ Frontend: POST /api/supervisor/barge/sip-credentials
|
||||||
|
└→ Sidecar: calls Ozonetel apiId 139 → gets SIP number/password/domain
|
||||||
|
└→ Frontend: initializes supervisor-sip-client with credentials
|
||||||
|
└→ Frontend: POST /api/supervisor/barge { ucid, agentNumber }
|
||||||
|
└→ Sidecar: calls Ozonetel apiId 63 (CALL_BARGEIN, isSip: true)
|
||||||
|
└→ Ozonetel: bridges SIP number into active call
|
||||||
|
└→ Supervisor SIP client receives incoming call → auto-answers
|
||||||
|
└→ Status: CONNECTED, default mode: Listen (DTMF "4" sent)
|
||||||
|
└→ Sidecar: creates BargeSession in memory
|
||||||
|
|
||||||
|
3. Supervisor clicks "Whisper" tab
|
||||||
|
└→ Frontend: supervisor-sip-client.sendDTMF("5")
|
||||||
|
└→ Ozonetel: routes supervisor audio to agent only
|
||||||
|
└→ Frontend: POST /api/supervisor/barge/mode { agentId, mode: "whisper" }
|
||||||
|
└→ Sidecar: emits SSE { state: "supervisor-whisper" } to agent
|
||||||
|
└→ Agent: sees "Supervisor coaching" badge
|
||||||
|
|
||||||
|
4. Supervisor clicks "Barge" tab
|
||||||
|
└→ Frontend: supervisor-sip-client.sendDTMF("6")
|
||||||
|
└→ Ozonetel: routes supervisor audio to both
|
||||||
|
└→ Frontend: POST /api/supervisor/barge/mode { agentId, mode: "barge" }
|
||||||
|
└→ Sidecar: emits SSE { state: "supervisor-barge" } to agent
|
||||||
|
└→ Agent: sees "Supervisor on call" badge
|
||||||
|
|
||||||
|
5. Call ends (agent or patient hangs up)
|
||||||
|
└→ Supervisor SIP client: "callEnded" event fires
|
||||||
|
└→ Frontend: auto-disconnects, calls POST /api/supervisor/barge/end
|
||||||
|
└→ Sidecar: clears BargeSession, emits SSE { state: "supervisor-left" }
|
||||||
|
└→ Agent: badge disappears
|
||||||
|
└→ UI: right panel resets to "Select a call to monitor"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Files to Create/Modify
|
||||||
|
|
||||||
|
### New Files
|
||||||
|
| File | Purpose |
|
||||||
|
|------|---------|
|
||||||
|
| `helix-engage-server/src/ozonetel/ozonetel-admin-auth.service.ts` | Ozonetel admin JWT management |
|
||||||
|
| `helix-engage-server/src/supervisor/supervisor-barge.controller.ts` | Barge proxy endpoints |
|
||||||
|
| `helix-engage/src/lib/supervisor-sip-client.ts` | Supervisor SIP client (modeled on kSip) |
|
||||||
|
|
||||||
|
### Modified Files
|
||||||
|
| File | Change |
|
||||||
|
|------|--------|
|
||||||
|
| `helix-engage-server/src/config/telephony.defaults.ts` | Add `adminUsername`, `adminPassword` |
|
||||||
|
| `helix-engage-server/src/supervisor/supervisor.service.ts` | Add barge session tracking + SSE events |
|
||||||
|
| `helix-engage/src/pages/live-monitor.tsx` | Split layout, context panel, barge controls |
|
||||||
|
| `helix-engage/src/components/call-desk/active-call-card.tsx` | Supervisor indicator badge |
|
||||||
|
| `helix-engage/src/hooks/use-agent-state.ts` | Handle supervisor SSE events |
|
||||||
|
| `helix-engage/src/components/setup/wizard-step-telephony.tsx` | Add admin credential fields |
|
||||||
|
|
||||||
|
### Reference Files (from Ozonetel source — study, don't copy)
|
||||||
|
| File | What to learn |
|
||||||
|
|------|--------------|
|
||||||
|
| `CA-Admin/.../BargeInDrawer/BargeInDrawer.tsx` | Normal barge flow, status states |
|
||||||
|
| `CA-Admin/.../BargeinDrawerSip/BargeinDrawerSip.tsx` | SIP barge, DTMF, continuous barge, session storage |
|
||||||
|
| `CA-Admin/.../utils/ksip.tsx` | SIP client wrapper pattern |
|
||||||
|
| `CA-Admin/.../services/api-service.ts:827-890` | Barge API payloads |
|
||||||
|
| `CA-Admin/.../services/auth-service.ts` | Admin auth flow |
|
||||||
|
| `cloudagent/.../services/websocket.service.js:367-460` | Agent-side barge event handling |
|
||||||
|
|
||||||
|
## Testing Plan
|
||||||
|
|
||||||
|
1. **Prereq:** QA validates barge in Ozonetel's own admin UI with the 3 SIP IDs
|
||||||
|
2. **Sidecar unit tests:** Admin auth service (login, token refresh, expiry)
|
||||||
|
3. **Sidecar integration test:** Barge endpoint → Ozonetel API (mock or live)
|
||||||
|
4. **Frontend manual test:** Connect → listen → whisper → barge → hang up
|
||||||
|
5. **Agent indicator test:** Verify badge appears on whisper/barge, disappears on listen/disconnect
|
||||||
|
6. **Auto-disconnect test:** Agent ends call → supervisor auto-disconnects
|
||||||
|
7. **Edge cases:** Supervisor navigates away mid-barge, network drop, agent goes to ACW
|
||||||
|
|
||||||
|
## Out of Scope (Future)
|
||||||
|
|
||||||
|
- PSTN barge (call supervisor's phone instead of SIP)
|
||||||
|
- Continuous barge (auto-reconnect to next call same agent handles)
|
||||||
|
- Barge audit logging (who barged whom, when, duration)
|
||||||
|
- Gemini AI whisper (separate feature, separate branch)
|
||||||
|
- Multi-supervisor on same call
|
||||||
Reference in New Issue
Block a user