web
You’re offline. This is a read only version of the page.
close
Skip to main content
Community site session details

Community site session details

Session Id :
Copilot Studio - Building Copilot Studio Chatbo...
Suggested Answer

Problems Using a Copilot Studio Agent with Custom Knowledge (JSON Wiki Export)

(1) ShareShare
ReportReport
Posted on by 5

Hi everyone,

 

I’m currently using a Copilot Studio agent with a custom knowledge base. The knowledge is provided as a JSON file exported from our internal wiki. Each entry contains the fields title, url, and content.

In the agent’s instructions, I explicitly tell it to output the corresponding URL whenever it uses information from the knowledge base. This works in some cases, but I’m still running into several issues:


  1. Sometimes the agent can’t find an answer at all, even though the relevant information definitely exists in the JSON.

  2. Sometimes it hallucinates an answer and even invents a link, despite the instructions stating that it is not allowed to do this.

  3. Sometimes it finds the correct information, but the returned link is wrong — instead of pointing to the specific referenced article, it links only to a parent or unrelated subpage.

Because of these issues, I’m wondering:

Should I use a different format instead of JSON for the custom knowledge base?

Is there a recommended or more reliable format for Copilot Studio to improve grounding and reduce hallucinations?

I have the same question (0)
  • Suggested answer
    venturemavenwill Profile Picture
    1,187 Super User 2025 Season 2 on at
    Problems Using a Copilot Studio Agent with Custom Knowledge (JSON Wiki Export)
    When you upload a large JSON file directly as a knowledge source, Copilot attempts to index it. However, if the content is dense, the semantic search might fail to retrieve the exact chunk, or the Large Language Model (LLM) might get "distracted" by irrelevant data in its context window.

    LLM doesn't filter per say, but retrieves sections of the content that it deems relevant.
     
    If you already have the output you want in JSON format, I recommend working with agent flow first to filter the items to limit the scope, and then feed it to Copilot to interpret the final answer.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Coming soon: forum hierarchy changes

In our never-ending quest to improve we are simplifying the forum hierarchy…

Chiara Carbone – Community Spotlight

We are honored to recognize Chiara Carbone as our Community Spotlight for November…

Leaderboard > Copilot Studio

#1
Romain The Low-Code Bearded Bear Profile Picture

Romain The Low-Code... 304 Super User 2025 Season 2

#2
Michael E. Gernaey Profile Picture

Michael E. Gernaey 224 Super User 2025 Season 2

#3
Sam_Fawzi Profile Picture

Sam_Fawzi 78 Super User 2025 Season 2

Last 30 days Overall leaderboard

Featured topics