Skip to main content

Notifications

Community site session details

Community site session details

Session Id :
Copilot Studio - Model Context Protocol
Unanswered

What's going on, why models seem dumber than through the api?

(0) ShareShare
ReportReport
Posted on by 2

I've been working in n8n for a while creating myself an agent, simple loop, gpt4.1 with an elaborate system prompt and some mcp tools connected.

Mcp tools provide agent with access to crm/wms to ask about orders/products/status/tracking
 

IT WORKS FANTASTIC. I thought I've got staff some copilot licences, let's try and deploy it behind auth as a copilot agent so staff can use it.


I've tried to recreate the same thing on copilot studio. Same model, same system prompt, same mcp tools. Generative AI enabled.


It's dumber. It's missing context that was provided by tool responses. It's coming back with very dry and short responses. It delivers bare minimum in response. It always adds "If you need further assistance, feel free to reach out to customer support on the respective platform." with each response. It's not eager to call multiple tools to get the final answer.
 

Basically this, when asked for can you tell me what's the situation with this order [order number]:

n8n agent

goes and get's order content, matching pos for etas, looks up suppliers, suggest alternatives to non in stock items . tracking numbers and writes a complete nicely formatted here is the situation response. Suggest potential next steps to take to resolve any issues.
 

Copilot agent

The order is awaiting shipping. This is the items that are on this order LIST OF PRODUCT CODES, and that's it, not even qty - code - description(which is all provided), just codes.


It's like temperature is all wrong and there seems to be additional layer between what the agent generates and what the final response is going to be.

I've adjusted prompt, moderation level, response formatting, tried modifying the topics to see if it changes it. Nothing works.

 

And it's not like n8n is doing something special to make it happen, I've recreated the n8n behaviour in langchain, system prompt + mcp tools, literally 30 lines. Works as expected.

  • GGGreg Profile Picture
    2 on at
    What's going on, why models seem dumber than through the api?
    And don't get me started on some other weird behaviours like.
    • If the agent is deployed in teams, it will not try the tools at all. Tools work in copilot but not in the chat.
    • I've got the tools behind on premise gateway -> custom connector -> custom connection. Every so often I have to reconnect as the connection goes stale? THIS SHOULD NOT BE SOMETHING USER DOES after first auth. It should just Unstale the connection.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Michael Gernaey – Community Spotlight

We are honored to recognize Michael Gernaey as our June 2025 Community…

Congratulations to the May Top 10 Community Leaders!

These are the community rock stars!

Announcing the Engage with the Community forum!

This forum is your space to connect, share, and grow!

Leaderboard > Copilot Studio

#1
Romain The Low-Code Bearded Bear Profile Picture

Romain The Low-Code... 132

#2
Michael E. Gernaey Profile Picture

Michael E. Gernaey 81 Super User 2025 Season 1

#3
Pablo Roldan Profile Picture

Pablo Roldan 61