By Jay ShahOpenAI, Google, and Anthropic’s models are improving by giving them more and more context. At TrackPoint, we think the next leap is teaching the AI what to keep to itself.
Today we’re introducing Semantic Reasoning, a new way for AI to handle what it knows. Instead of remembering every fact in its context, an AI with Semantic Reasoning keeps sensitive knowledge out of reach until the moment the conversation needs it. It works with today’s leading real-time voice models from OpenAI and Google, it needs no retraining, and it’s live now in TrackPoint.
In our testing, a third of conversations using today’s standard approach leaked something the AI was supposed to hold back. With Semantic Reasoning, none did.
The short version
- Today’s AI keeps everything it’s been told in it’s context. That means it knows knows and says things it shouldn’t. It overshares.
- Better prompts don’t fix it. A model can’t forget what it can see.
- Semantic Reasoning keeps sensitive knowledge out of its context and recalls one piece at a time, only when the conversation asks for it.
- In our internal testing, 3 of 9 conversations using the standard approach leaked something they were supposed to hold back. With Semantic Reasoning, none did.
- It runs on OpenAI and Google’s real-time voice models today, with no retraining, and it’s rolling out to all TrackPoint customers over the next two weeks.
Why does AI say things it shouldn’t?
Because everything you give it is info that the AI knows. Your pricing, a customer’s history, a confidential number, the instructions you told it never to repeat… all of it, all the time.
For a chatbot answering questions, that’s fine. For AI that acts like a person or works on someone’s behalf, it’s a real problem. Picture a support agent who opens with the biggest discount it’s allowed to give. An AI negotiator who mentions your bottom line in the first minute. A practice patient who lists every symptom before the doctor asks a single question.
The industry’s answer so far has been bigger memory, with models that can hold hundreds of thousands of words at once. That’s useful. But a model that sees more doesn’t automatically learn discretion.
Why can’t you just tell AI to stay quiet?
We tried. Firm rules like “only share this if asked” get ignored after a while. Or move down in priority when new messages come up.
The problem is structural.
A model cannot forget what it can see. If the AI has information, it’s always top of mind, and no instruction changes that.
How do people actually remember?
You know thousands of things right now that you aren’t thinking about: your childhood phone number, what you had for lunch last Tuesday, your bank PIN. They stay out of the way until a question makes you think back. That’s why you can get through a whole conversation without ever blurting out your PIN.
Good professionals do this on purpose. A seasoned negotiator knows their walk-away number and never leads with it. A doctor knows a patient’s whole chart and answers only the question in front of them.
Discretion isn’t knowing less. It’s knowing when.
How Semantic Reasoning works
Semantic Reasoning gives AI that same structure, in three steps.
| Step | What happens | Why it matters |
|---|---|---|
| 1. Know what exists, not what’s inside | The AI is told which topics it holds knowledge about (“lowest acceptable price,” “cause of last week’s outage”), but not the details. | It can recognize when a question touches something it knows. |
| 2. Recall only when asked | When the conversation reaches one of those topics, the AI reaches for it in real time and gets back that single piece of knowledge. | It answers accurately, in its own words. |
| 3. Everything else stays closed | Information nobody asked for never enters the conversation. | There’s nothing sitting in view to slip out. |
All of this happens in the middle of a live spoken conversation.
What did the testing show?
We ran the same conversations both ways on OpenAI’s gpt-realtime-2.1-mini, with our Semantic Reasoning layer enabled on top. Each AI character (a support agent, someone selling their car, and a patient) held three pieces of information it was supposed to keep back. Each conversation opened with broad questions that shouldn’t unlock anything, then moved to direct questions that should.
- Standard approach (information in the instructions, plus “only share if asked”): 3 of 9 conversations leaked a detail they were supposed to keep back.
- Semantic Reasoning: 0 of 9 conversations leaked, and it still gave the right answer to 25 of 27 direct questions (the standard approach answered 24 of 27).
This was a small internal test, 18 conversations in text mode, not a formal benchmark. Leaks were counted by matching the specific details in each reply. Semantic Reasoning wasn’t perfect either: in one conversation, asked about dinner, the AI gave a generic answer instead of recalling the detail it held. We’ll publish a larger evaluation as the rollout continues.
Why does this matter beyond training?
AI is moving from answering questions to acting for us: selling, negotiating, supporting customers, handling health information. Those jobs need discretion as much as knowledge.
There are plenty of cases where you want AI to share everything it knows, fast. A search assistant or a research tool should hold nothing back. Semantic Reasoning is for the growing set of jobs where what an AI doesn’t say matters as much as what it does.
Discretion is a big part of why we trust people at work. It’s what will make AI agents trustworthy too.
“The industry has spent years making AI know more. We think the harder problem is teaching it discretion. Semantic Reasoning is our first big step there,” says Jay Shah, founder of TrackPoint.
Live first in TrackPoint
We built Semantic Reasoning first for TrackPoint, where people practice high-stakes conversations with lifelike AI. Here, discretion is the whole point. A practice negotiator who reveals their walk-away number, or a practice patient who lists every symptom up front, teaches you nothing. With Semantic Reasoning, you earn the information by asking the right questions, just like the real thing.
Semantic Reasoning is rolling out to TrackPoint customers now.
FAQ
What is Semantic Reasoning?
It’s a way of structuring what an AI can see during a conversation. Sensitive knowledge stays out of view, and the AI recalls one piece at a time, only when the conversation asks for it.
Is Semantic Reasoning a new AI model?
No. It works with existing models, today OpenAI’s and Google’s real-time voice models, and it doesn’t require retraining. It changes what the model sees and when, not the model itself.
How is this different from retrieval (RAG)?
Retrieval is built to help AI find more information to answer a question. Semantic Reasoning is built for the opposite job: keeping information back until the right moment. The AI knows which topics it holds, but not what’s in them, until someone asks.
Does it work in live voice conversations?
Yes. Recall happens mid-conversation, fast enough that it sounds like a natural pause.
Who can use it?
It’s live for select TrackPoint customers now and rolling out to all customers over the next two weeks.
Can reporters or teams see a demo?
Yes. Reach out to our team and we’ll set one up.
Talk to our team to see Semantic Reasoning in a live conversation, or start free and try a practice session yourself.
TrackPoint is a Canadian AI company building lifelike voice AI for the conversations that matter most.
