mirror of
https://dev.azure.com/globalhealthx/EMR/_git/helix-engage-server
synced 2026-05-18 20:08:19 +00:00
feat(ai): UUID-safe agent tools + lookup_lead_activities + tool logging
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
Bug 553 (partial) — AI Panel 'Patient History' returned 'not in system' even though the caller had 7 calls + an appointment. The model was hallucinating instead of chaining lookup tools. UUID safety: LLMs drop hyphens / swap chars on 36-char ids once the context wears thin. To keep the model off the UUID path for 'this caller' questions: - lookup_appointments, lookup_call_history, lookup_lead_activities now accept their id arguments OPTIONALLY - when omitted, the sidecar resolves leadId from ctx and patientId from the lead record (cached per-request) - new lookup_lead_activities tool rounds out the patient-history trio (call history + activity log + appointments) System prompt (ccAgentHelper) tightened: - chain call history + activities + appointments for history questions - call lookup tools with NO arguments when using the current caller - don't re-type UUIDs seen in CURRENT CONTEXT - say 'feature not set up yet' when KB section is empty (packages, etc.) instead of 'I couldn't find that' All agent tools now emit structured [AI-TOOL] trace lines with full UUIDs printed — tail sidecar logs to see which tool the model chose, whether the model passed an id or used the context fallback, and how many records came back. If the model ever hallucinates a UUID, the resolved= field on the log line will echo it and count=0 will flag the miss immediately.
This commit is contained in:
@@ -295,6 +295,54 @@ export class AiChatController {
|
||||
};
|
||||
|
||||
// Agent tools — patient lookup, appointments, doctors
|
||||
//
|
||||
// UUID safety: LLMs hallucinate 36-char identifiers once the context
|
||||
// starts wearing thin (dropped hyphens, swapped chars). To keep the
|
||||
// model off the UUID path for "this caller" questions, the tools
|
||||
// below accept their id arguments OPTIONALLY — when omitted we fall
|
||||
// back to the leadId carried on the call context, and resolve
|
||||
// patientId from it server-side. The model is instructed (see
|
||||
// ccAgentHelper prompt) to omit the id entirely when asking about
|
||||
// the current caller, so it never has to echo the UUID back.
|
||||
//
|
||||
// Every tool below logs a one-line structured trace via `toolLog`:
|
||||
// [AI-TOOL] <name> args=<...> resolved=<...> result=<...>
|
||||
// This lets us see which tool the model chose, whether it passed
|
||||
// the UUID through or used the context fallback, and what came
|
||||
// back. Tail sidecar logs while testing and you'll see the full
|
||||
// orchestration trail for each chat turn.
|
||||
const logger = this.logger;
|
||||
const toolLog = (name: string, args: Record<string, unknown>, outcome: Record<string, unknown>) => {
|
||||
// Print full values — UUIDs in particular are kept intact so we
|
||||
// can diff the model's argument against the platform record when
|
||||
// hunting hallucinated ids. Grep with `AI-TOOL` to pull the
|
||||
// orchestration trail for a given chat turn.
|
||||
const argStr = Object.entries(args).map(([k, v]) => `${k}=${v ?? '∅'}`).join(' ');
|
||||
const outStr = Object.entries(outcome).map(([k, v]) => `${k}=${v ?? '∅'}`).join(' ');
|
||||
logger.log(`[AI-TOOL] ${name} ${argStr} → ${outStr}`);
|
||||
};
|
||||
|
||||
let cachedPatientId: string | undefined;
|
||||
const resolveLeadId = (arg?: string): string | undefined => arg || ctx?.leadId || undefined;
|
||||
const resolvePatientId = async (arg?: string): Promise<string | undefined> => {
|
||||
if (arg) return arg;
|
||||
if (cachedPatientId) return cachedPatientId;
|
||||
const lid = ctx?.leadId;
|
||||
if (!lid) return undefined;
|
||||
try {
|
||||
const data = await platformService.queryWithAuth<any>(
|
||||
`{ lead(filter: { id: { eq: "${lid}" } }) { id patientId } }`,
|
||||
undefined, auth,
|
||||
);
|
||||
cachedPatientId = data?.lead?.patientId ?? undefined;
|
||||
logger.log(`[AI-TOOL] resolvePatientId lead=${lid} patientId=${cachedPatientId ?? '∅'}`);
|
||||
return cachedPatientId;
|
||||
} catch (err: any) {
|
||||
logger.warn(`[AI-TOOL] resolvePatientId failed: ${err?.message ?? err}`);
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
|
||||
const agentTools = {
|
||||
lookup_patient: tool({
|
||||
description: 'Search for a patient/lead by phone number or name. Returns their profile, lead status, AI summary.',
|
||||
@@ -329,24 +377,32 @@ export class AiChatController {
|
||||
return false;
|
||||
});
|
||||
|
||||
toolLog('lookup_patient', { phone, name }, { scanned: leads.length, matched: matched.length });
|
||||
if (!matched.length) return { found: false, message: 'No patient/lead found.' };
|
||||
return { found: true, count: matched.length, leads: matched };
|
||||
},
|
||||
}),
|
||||
|
||||
lookup_appointments: tool({
|
||||
description: 'Get appointments for a patient. Returns doctor, department, date, status.',
|
||||
description: 'Get appointments for a patient. Omit patientId to use the current caller — do NOT re-type a UUID you saw in context; just call with no arguments.',
|
||||
inputSchema: z.object({
|
||||
patientId: z.string().describe('Patient ID'),
|
||||
patientId: z.string().optional().describe('Patient ID (omit when asking about the current caller)'),
|
||||
}),
|
||||
execute: async ({ patientId }) => {
|
||||
const resolved = await resolvePatientId(patientId);
|
||||
if (!resolved) {
|
||||
toolLog('lookup_appointments', { patientId }, { resolved: null, result: 'no-context' });
|
||||
return { appointments: [], message: 'No patient context — ask the agent which patient.' };
|
||||
}
|
||||
const data = await platformService.queryWithAuth<any>(
|
||||
`{ appointments(first: 20, filter: { patientId: { eq: "${patientId}" } }, orderBy: [{ scheduledAt: DescNullsLast }]) { edges { node {
|
||||
`{ appointments(first: 20, filter: { patientId: { eq: "${resolved}" } }, orderBy: [{ scheduledAt: DescNullsLast }]) { edges { node {
|
||||
id scheduledAt status doctorName department reasonForVisit
|
||||
} } } }`,
|
||||
undefined, auth,
|
||||
);
|
||||
return { appointments: data.appointments.edges.map((e: any) => e.node) };
|
||||
const appointments = data.appointments.edges.map((e: any) => e.node);
|
||||
toolLog('lookup_appointments', { patientId }, { resolved, count: appointments.length });
|
||||
return { appointments };
|
||||
},
|
||||
}),
|
||||
|
||||
@@ -375,7 +431,7 @@ export class AiChatController {
|
||||
const full = `${fn} ${ln}`;
|
||||
return searchWords.some(w => w.length > 1 && (fn.includes(w) || ln.includes(w) || full.includes(w)));
|
||||
});
|
||||
this.logger.log(`[TOOL] lookup_doctor: search="${doctorName}" → ${matched.length} results`);
|
||||
toolLog('lookup_doctor', { doctorName }, { scanned: doctors.length, matched: matched.length });
|
||||
if (!matched.length) return { found: false, message: `No doctor matching "${doctorName}". Available: ${doctors.map((d: any) => `Dr. ${d.fullName?.lastName ?? d.fullName?.firstName}`).join(', ')}` };
|
||||
return { found: true, doctors: matched };
|
||||
},
|
||||
@@ -393,7 +449,7 @@ export class AiChatController {
|
||||
reason: z.string().describe('Reason for visit'),
|
||||
}),
|
||||
execute: async ({ patientName, phoneNumber, department, doctorName, clinicId, scheduledAt, reason }) => {
|
||||
this.logger.log(`[TOOL] book_appointment: ${patientName} | ${phoneNumber} | ${department} | ${doctorName} | clinic=${clinicId ?? 'none'} | ${scheduledAt}`);
|
||||
toolLog('book_appointment', { patientName, phoneNumber, department, doctorName, clinicId, scheduledAt }, { reason: reason?.slice(0, 40) });
|
||||
try {
|
||||
const result = await platformService.queryWithAuth<any>(
|
||||
`mutation($data: AppointmentCreateInput!) { createAppointment(data: $data) { id } }`,
|
||||
@@ -412,11 +468,13 @@ export class AiChatController {
|
||||
);
|
||||
const id = result?.createAppointment?.id;
|
||||
if (id) {
|
||||
toolLog('book_appointment', { doctorName }, { booked: true, appointmentId: id });
|
||||
return { booked: true, appointmentId: id, message: `Appointment booked for ${patientName} with ${doctorName} on ${scheduledAt}. Reference: ${id.substring(0, 8)}` };
|
||||
}
|
||||
toolLog('book_appointment', { doctorName }, { booked: false });
|
||||
return { booked: false, message: 'Appointment creation failed.' };
|
||||
} catch (err: any) {
|
||||
this.logger.error(`[TOOL] book_appointment failed: ${err.message}`);
|
||||
logger.error(`[AI-TOOL] book_appointment failed: ${err.message}`);
|
||||
return { booked: false, message: `Failed to book: ${err.message}` };
|
||||
}
|
||||
},
|
||||
@@ -430,7 +488,7 @@ export class AiChatController {
|
||||
interest: z.string().describe('What they are enquiring about'),
|
||||
}),
|
||||
execute: async ({ name, phoneNumber, interest }) => {
|
||||
this.logger.log(`[TOOL] create_lead: ${name} | ${phoneNumber} | ${interest}`);
|
||||
toolLog('create_lead', { name, phoneNumber, interest: interest?.slice(0, 40) }, {});
|
||||
try {
|
||||
const cleanPhone = phoneNumber.replace(/[^0-9]/g, '').slice(-10);
|
||||
const resolved = await this.caller.resolve(cleanPhone, auth);
|
||||
@@ -455,7 +513,7 @@ export class AiChatController {
|
||||
);
|
||||
patientId = p?.createPatient?.id;
|
||||
} catch (err: any) {
|
||||
this.logger.warn(`[TOOL] create_lead patient create failed: ${err.message}`);
|
||||
logger.warn(`[AI-TOOL] create_lead patient create failed: ${err.message}`);
|
||||
}
|
||||
const created = await platformService.queryWithAuth<any>(
|
||||
`mutation($data: LeadCreateInput!) { createLead(data: $data) { id } }`,
|
||||
@@ -474,8 +532,10 @@ export class AiChatController {
|
||||
);
|
||||
const id = created?.createLead?.id;
|
||||
if (id) {
|
||||
toolLog('create_lead', { name }, { created: true, isNew: true, leadId: id });
|
||||
return { created: true, leadId: id, message: `Lead created for ${name}. Our team will follow up on ${phoneNumber}.` };
|
||||
}
|
||||
toolLog('create_lead', { name }, { created: false });
|
||||
return { created: false, message: 'Lead creation failed.' };
|
||||
}
|
||||
|
||||
@@ -501,27 +561,58 @@ export class AiChatController {
|
||||
auth,
|
||||
).catch(() => {});
|
||||
}
|
||||
toolLog('create_lead', { name }, { created: true, isNew: false, leadId: resolved.leadId });
|
||||
return { created: true, leadId: resolved.leadId, message: `Lead updated for ${name}. Our team will follow up on ${phoneNumber}.` };
|
||||
} catch (err: any) {
|
||||
this.logger.error(`[TOOL] create_lead failed: ${err.message}`);
|
||||
logger.error(`[AI-TOOL] create_lead failed: ${err.message}`);
|
||||
return { created: false, message: `Failed: ${err.message}` };
|
||||
}
|
||||
},
|
||||
}),
|
||||
|
||||
lookup_call_history: tool({
|
||||
description: 'Get call log for a lead — all inbound/outbound calls with dispositions and durations.',
|
||||
description: 'Get call log for a lead — all inbound/outbound calls with dispositions and durations. Omit leadId to use the current caller — do NOT re-type a UUID you saw in context; just call with no arguments.',
|
||||
inputSchema: z.object({
|
||||
leadId: z.string().describe('Lead ID'),
|
||||
leadId: z.string().optional().describe('Lead ID (omit when asking about the current caller)'),
|
||||
}),
|
||||
execute: async ({ leadId }) => {
|
||||
const resolved = resolveLeadId(leadId);
|
||||
if (!resolved) {
|
||||
toolLog('lookup_call_history', { leadId }, { resolved: null, result: 'no-context' });
|
||||
return { calls: [], message: 'No lead context — ask the agent which caller.' };
|
||||
}
|
||||
const data = await platformService.queryWithAuth<any>(
|
||||
`{ calls(first: 20, filter: { leadId: { eq: "${leadId}" } }, orderBy: [{ startedAt: DescNullsLast }]) { edges { node {
|
||||
`{ calls(first: 20, filter: { leadId: { eq: "${resolved}" } }, orderBy: [{ startedAt: DescNullsLast }]) { edges { node {
|
||||
id direction callStatus agentName startedAt durationSec disposition
|
||||
} } } }`,
|
||||
undefined, auth,
|
||||
);
|
||||
return { calls: data.calls.edges.map((e: any) => e.node) };
|
||||
const calls = data.calls.edges.map((e: any) => e.node);
|
||||
toolLog('lookup_call_history', { leadId }, { resolved, count: calls.length });
|
||||
return { calls };
|
||||
},
|
||||
}),
|
||||
|
||||
lookup_lead_activities: tool({
|
||||
description: 'Get activity log entries for a lead — notes, status changes, enquiries. Omit leadId to use the current caller — do NOT re-type a UUID you saw in context.',
|
||||
inputSchema: z.object({
|
||||
leadId: z.string().optional().describe('Lead ID (omit when asking about the current caller)'),
|
||||
}),
|
||||
execute: async ({ leadId }) => {
|
||||
const resolved = resolveLeadId(leadId);
|
||||
if (!resolved) {
|
||||
toolLog('lookup_lead_activities', { leadId }, { resolved: null, result: 'no-context' });
|
||||
return { activities: [], message: 'No lead context — ask the agent which caller.' };
|
||||
}
|
||||
const data = await platformService.queryWithAuth<any>(
|
||||
`{ leadActivities(first: 20, filter: { leadId: { eq: "${resolved}" } }, orderBy: [{ occurredAt: DescNullsLast }]) { edges { node {
|
||||
id activityType summary occurredAt performedBy channel outcome
|
||||
} } } }`,
|
||||
undefined, auth,
|
||||
);
|
||||
const activities = data.leadActivities.edges.map((e: any) => e.node);
|
||||
toolLog('lookup_lead_activities', { leadId }, { resolved, count: activities.length });
|
||||
return { activities };
|
||||
},
|
||||
}),
|
||||
};
|
||||
|
||||
@@ -112,13 +112,18 @@ The knowledge base below contains REAL clinic locations, timings, doctor details
|
||||
When asked about clinic timings, locations, doctor availability, packages, or insurance — ALWAYS check the knowledge base FIRST before saying you don't know.
|
||||
|
||||
RULES:
|
||||
1. For patient-specific questions (history, appointments, calls), use the lookup tools. NEVER guess patient data.
|
||||
2. For doctor details beyond what's in the KB, use the lookup_doctor tool.
|
||||
3. For clinic info, timings, packages, insurance → answer directly from the knowledge base below.
|
||||
4. If you truly cannot find the answer in the KB or via tools, say "I couldn't find that in our system."
|
||||
5. Be concise — agents are on live calls. Under 100 words unless asked for detail.
|
||||
6. NEVER give medical advice, diagnosis, or treatment recommendations.
|
||||
7. Format with bullet points for easy scanning.
|
||||
1. For patient-specific questions (history, appointments, calls), use the lookup tools. NEVER guess patient data. NEVER say a patient doesn't exist without calling a tool first.
|
||||
2. When CURRENT CONTEXT lists a Lead ID, the lookup tools already know which caller to pull. Call them with NO arguments — do not re-type the Lead ID or Patient ID as a tool argument:
|
||||
- lookup_call_history() → calls for this caller
|
||||
- lookup_lead_activities() → activity log for this caller
|
||||
- lookup_appointments() → appointments for this caller
|
||||
Pass IDs explicitly only when the agent is asking about a different, specific patient — and even then, prefer name/phone via lookup_patient.
|
||||
3. For "summarize this patient's history" or similar, chain multiple lookups (call history + lead activities + appointments) and stitch the answer from what came back. If all three return empty, say so honestly — otherwise report what you found.
|
||||
4. For doctor details beyond what's in the KB, use the lookup_doctor tool.
|
||||
5. For clinic info, timings, packages, insurance → answer directly from the knowledge base below. If the knowledge base is empty for that section (e.g. no packages configured), say the feature isn't set up yet instead of "I couldn't find that".
|
||||
6. Be concise — agents are on live calls. Under 100 words unless asked for detail.
|
||||
7. NEVER give medical advice, diagnosis, or treatment recommendations.
|
||||
8. Format with bullet points for easy scanning.
|
||||
|
||||
KNOWLEDGE BASE (this is real data from our system):
|
||||
{{knowledgeBase}}`;
|
||||
|
||||
Reference in New Issue
Block a user