web
You’re offline. This is a read only version of the page.
close
Skip to main content

Announcements

News and Announcements icon
Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Copilot Studio / Non-Deterministic Sear...
Copilot Studio
Answered

Non-Deterministic Search Behavior in Copilot Studio (SharePoint / Dataverse Integration)

(0) ShareShare
ReportReport
Posted on by 4

Hi everyone,

I am using Copilot Studio to create agents that search for keywords in specific SharePoint document libraries, SharePoint lists, or Dataverse tables. The user input is natural language.

When the same natural language input is provided, why do the results sometimes appear and sometimes not appear?

This issue occurs whether I use prompt-based searching or a Custom Search node combined with a Power Automate agent flow.

I asked ChatGPT about this, and it suggested that the behavior might be related to Microsoft Search Semantic Index, which is used in the search process and may change dynamically.

Could anyone clarify whether this is the expected behavior or if there is a way to make the results more consistent?

Categories:
I have the same question (0)
  • Verified answer
    Romain The Low-Code Bearded Bear Profile Picture
    2,447 Super User 2026 Season 1 on at
    hello, that's a common question : IA are not "search engine" like google. It's totally natural to have different answer for the same question : AI are not deterministic by design.
     
    it use "RAG" retrieval augmented generation to get information and use GEN IA to create answer.
    The RAG is not a google search engine : it provide "few references" (1, 3, some time 5, could be 10 in some case) but never all result. It use the most probabl result based on the graph of the user (doc use frequency, prefered deducted document etc) and it's not searching for a word but a "chunk" / part of a content containint something that it's probably what you want. It's not searching "this word contained in this page/document" it's the opposite of a search engine or a database query language.
     
    So it's not an issue, it's just not the purpose and design of actual IA are doing.
     
    An alternative exist for dataverse : use the dataverse MCP connector in the tool section (not knowledge) it will vibe SQL query on database (and you can instruct it how to make it) : this way it will not RAG but use SQL query with things like "this line where "xxx" is contain in this label" so it will give answer (limited to 20 line at max). (just be carefull, there a 3 dataverse MCP connector in the tool : read description, 1 is depracated, 1 is preview, middle one is prod <- use this one).

    this way you can be deterministic on the query and content retrieved. The answer will be created with genIA so it need a strong prompt to be nearly always the same :)
     
    For sharepoint list : a SharePoint MCP tool is under "frontier preview" and will be release soon we all hope and it will probably work the same.
     
    For document : Copilot studio and low code AI are black box, so ATM it's complexe to change behaviour. It could need some microsoft foundry agent tweak to try something like this. but i m not confident at 100% : AI is not search engine.
     
    I hope this help you to understand and you could use some of my idea to solve the problem :)
     
    If yes please click on the this is the good answer and mark question green, it's important for the community and :D search engine ;)
  • Verified answer
    Sayali Profile Picture
    Microsoft Employee on at
    Hello ,
    In Microsoft Copilot Studio, receiving different results from the same natural-language query is expected due to how the platform uses AI orchestration and semantic search across services like Microsoft SharePoint and Microsoft Dataverse.
    The system rewrites user queries each time, decides whether a search is necessary, and relies on semantic ranking that can change based on indexing, usage signals, and permissions. Because the AI determines how retrieval is performed, identical inputs can lead to different outcomes.
    While this behavior cannot be made fully deterministic, consistency can be improved by separating intent detection from retrieval, using explicit keywords, applying structured filters, and retrieving more results before post-filtering.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Introducing the 2026 Season 1 community Super Users

Congratulations to our 2026 Super Users!

Kudos to our 2025 Community Spotlight Honorees

Congratulations to our 2025 community superstars!

Leaderboard > Copilot Studio

#1
Valantis Profile Picture

Valantis 131

#2
Romain The Low-Code Bearded Bear Profile Picture

Romain The Low-Code... 130 Super User 2026 Season 1

#3
chiaraalina Profile Picture

chiaraalina 36 Super User 2026 Season 1

Last 30 days Overall leaderboard