Context Is King: Better Conversations, Lower Costs

Better Conversations, Lower Costs: A look under the hood

One of the best things about Clark Street Health, that we really try to nail, is that you can turn on AI agents that pick up the phone in a matter of minutes without much set up. You'll have a working system with full context on your practice, your preferences, your patients' needs, all as easy as a coordinating call with the IT team and a few emails to us. Behind the scenes, however, we do a tremendous amount of configuring and constantly tweak the agents to improve your patient's experience.

I want to break down some of what we do behind the scenes to try to make every call better, and what specifically we're changing month over month to do so. Some of it's technical, some of it's easy, and all of it, at least to us, is interesting, so I hope you find this useful.

Two levers to improve

We can simplify how we think of configuration in two big buckets. The first, is ensuring that the agents have enough information to do their job properly. The second is making sure, as they execute their job, they do it in a way that is efficient, helpful, and effective for the patient at the other end of the phone, and for you the provider team, who is receiving the info eventually. Both are important, both are improving, and both actually don't have much to do with the underlying AI models.

Context is king

Providing the agent with appropriate information about each conversation what our goals are is critical. The technical word for this in the AI world is context. There's dozens of PhD level papers written on context management, but in an essence it's the art of giving the agent enough information to do whatever it is they need to do but not too much more.

Try to imagine showing up at a new job and someone drops a big stack of papers down on your desk to read. That sounds daunting, but for AI, that sort of information dump is perfect. If you lay it out the right way they can process thousands of pages of information in seconds.

The challenge and the risk to this is just giving the agent as much info as you can possibly throw at it about what to do on a call. This actually can backfire and lead us down into traps where, for example, a patient calling about a prescription refill might sidetrack into a long monologue about insurance or something else. Just because the agent has too much information about too many different things. The technical term for this is called context rot. If you share too much info with the AI models and so that they can, and usually do, perform worse. Even though the models are trained on billions of pages of text and know almost all modern drug names in all languages, we can't overload them with our specific instructions.

So does this change what we do? Yes. Specifically, we have recently provided the agent with more dynamic instructions about how to verify and procure information on the fly for patients. One example happens when you configure an EHR Clark Street integration. You now can have the option of letting the Agents automatically (it takes seconds) look up patient records based on phone number. This can save the back and forth of name spelling, pronunciation, and really just makes all the conversations shorter, more efficient, better for both sides.

We are also now able to pull in relevant information based on what the high level root and category the patient's question falls into. If we're just talking about scheduling and need to transfer, that subset of information becomes available. If we are diving into a new patient intake form, the nuances and notes become available. This dynamic context management and the operating procedure the system we build around the agents makes them more efficient and reliable.

Style or vibes

The second big category that we think about is improving the overall style and manner in which the agents speak to callers. If you asked your teenage nephew what this is called, they might say vibes. It's a helpful analogy, but this is an important and serious challenge to setting up AI agents for success.

Small conversational tweaks can make every interaction better. We have been able to iterate towards improvements through listening to thousands of calls but there are always opportunities to improve.

Lately, we are focusing on having the agent ask one question at a time making it easy for patients to answer and stay on track. We also value efficiency, and never want the AI to ramble on longer than they need to which these models love doing. Success is getting the patients what they need in as short a call as is appropriate. We also have the AI slow its pace down for elderly patients, while if that same 18 year old nephew calls we can speed up the pace and questions. We run dozens of experiments a week, test new tweaks to our systems daily.

We rolled out changes that shortened the time our agents are speaking by about 20% while improving their accuracy. This gives you a decrease in cost with an increase in reliability.

Context management, and conversation style are both central to a successful AI deployment. They make up a lot of work we do to provide safe, effective and affordable solutions that improve every week.