feat: LiveKit AI answering agent (Gemini 2.5 Flash native audio)

- Hospital receptionist agent "Helix" with Gemini realtime speech-to-speech
- Tools wired to platform: lookupDoctor, bookAppointment, collectLeadInfo, transferToAgent
- Loads hospital context (doctors, departments) from platform GraphQL on startup
- Connects to LiveKit Cloud, joins rooms when participants connect
- Silero VAD for voice activity detection
- @livekit/agents + @livekit/agents-plugin-google + @livekit/agents-plugin-silero

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-03-25 20:30:41 +05:30
parent fcc7c90e84
commit 3c06a01e7b
5 changed files with 3498 additions and 65 deletions

View File

@@ -23,6 +23,9 @@
"@ai-sdk/anthropic": "^3.0.58",
"@ai-sdk/openai": "^3.0.41",
"@deepgram/sdk": "^5.0.0",
"@livekit/agents": "^1.2.1",
"@livekit/agents-plugin-google": "^1.2.1",
"@livekit/agents-plugin-silero": "^1.2.1",
"@nestjs/common": "^11.0.1",
"@nestjs/config": "^4.0.3",
"@nestjs/core": "^11.0.1",
@@ -34,7 +37,8 @@
"ioredis": "^5.10.1",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.1",
"socket.io": "^4.8.3"
"socket.io": "^4.8.3",
"zod": "^4.3.6"
},
"devDependencies": {
"@eslint/eslintrc": "^3.2.0",