creating a questioning strucure that produces great answers

(1) Share

Report

Posted on by CU13051508-0

Hi, I’m creating an agent that searches multiple documents. Before searching, I want it to ask clarifying questions to understand the user’s specific context and intent.

Our documentation is highly context-dependent. The same question may have different answers depending on the process, department, or situation.

Has anyone developed an instruction structure that reliably guides users through clarifying questions before the agent searches its knowledge sources? If so, would you be willing to share the structure or approach you used?

Thanks,

Brad

Categories:

Building Copilot Studio chatbots in Microsoft Teams

General topics

Copilot Studio skills development

Topic creation & management

All responses (4)

Answers (0)

Sort by

Suggested answer

11manish 4,609 Super User 2026 Season 2 on at

Like
a
(0)

Report
Copy link

Link copied!

use Clarify → Capture context → Validate context → Retrieve → Answer, rather than Question → Search → Hope the model figures out the context.

For your scenario, this will also make the agent considerably easier to test because you can create test cases such as same question + different department = different expected answer.
Suggested answer

Haque 4,189 Super User 2026 Season 2 on at

Like
a
(0)

Report
Copy link

Link copied!
Hi @CU13051508-0,

That's the beauty of the agent that asks the questions to narrow down the context so that intent can be significanlty precise to make the answers. We can follow some approaches where we can decorate the agent that will ask to clarify before searching multiple documents - more specifically when documentation or knowledge source is highly context-dependent.

Step-1: Lets make intellegent the agent by defining contextual dimensions - meaning, if the agent can identify the contextual factors that influence the answers, like as Process Type, Department, Situation, User role and even location would be useful.

Step-2: Nice to have a goal oriented topic flow - initial conversation flow can be simple and narrow done the context. Like greets the user, based on the contextual dimension identified clarify the targeted questions. Finally, collects user responses to narrow down the context. To set an example:

"Which department are you referring to?"

"Is this related to process A or process B?"

"Can you specify the situation or scenario?"

Step-3: Decorating context variable will be advantageous. Let's store the user's answers to these clarifying questions in context variables or memeory slots within the agent.

Step-4: As we have proposed to decorate context varaible, what we can do now is invoke knowledge source conditionally meaning we can configure knowledge source or documents searches to be triggered only after the relevant context varaibles are set. This can be done by using topic conditions that check for context variables before invoking document searce and setting knowledge sources to "Only when referenced by topics" and referencing them in the context-aware topics.

Step-5: Construct query dynamically - let's use the collected context to dynamically tailor the search query or filter the documents, ensuring the retrieval is relevant to the user’s specific context. Dynamic chaining is one area that can help.

Step-6: If the initial search results are ambiguous or insufficient, design the agent to ask follow-up clarifying questions to refine the context further. Build an iterative evaluation framework in four stages.

So the general structure should be like this:

Welcome Topic: "Hi! To help you better, I need to ask a few questions."

Clarification Topic

Ask: "Which department is this about?"

Store answer in context.department

Ask: "Which process are you referring to?"

Store answer in context.process

Ask: "What situation or scenario applies?"

Store answer in context.scenario

Search Invocation Topic

Condition: Only trigger if all the context variables are set. Use these variables to filter or scope the document search

Answer Delivery: Present the retrieved, context-aware answer to the user

References:

Design a question node - ask a question.

Disambiguate customer intent

Design effective language understanding

I am sure some clues I tried to give. If these clues help to resolve the issue brought you by here, please don't forget to check the box Does this answer your question? At the same time, I am pretty sure you have liked the response!
Suggested answer

chiaraalina 2,553 Super User 2026 Season 2 on at

Like
a
(1)

Report
Copy link

Link copied!
Hi @CU13051508-0

I would use a topic with inputs for the important context: Department, Process or Situation.

The reason is that with generative orchestration, the agent can try to get these values from the user's message first.

For example, if the user asks:

"I work in Finance. How do I get a purchase approved?"

the agent can already identify Finance (Department) and does not need to ask for the department again

But if the user only says:

"How do I get a purchase approved?"

the department is missing. The agent can then ask a clarifying question before continuing.

Another example:

"I work in Finance and want to buy software for CHF 3000. Who approves it?"

In this case, the agent can identify all three inputs directly:

Department = Finance

Purchase Type = Software

Amount = CHF 3,000

This avoids building a long fixed sequence of "Ask a question" nodes and makes the conversation more natural because the agent can reuse information the user has already provided.

Here is an example of the implementation:

kind: AdaptiveDialog

inputs:

- kind: AutomaticTaskInput

propertyName: Department

description: The department that the purchase relates to, for example Finance, HR, or IT. Use the department already mentioned by the user if available.

shouldPromptUser: true

- kind: AutomaticTaskInput

propertyName: PurchaseType

description: The type of purchase the user is asking about, for example Software, Consulting, Equipment, or another purchase category. Use the purchase type already mentioned by the user if available.

shouldPromptUser: true

- kind: AutomaticTaskInput

propertyName: Amount

description: The purchase amount in CHF. Use the amount already mentioned by the user if available.

shouldPromptUser: true

modelDescription: |

Use this topic when the user asks about purchase approvals,

purchase approval requirements, approval thresholds,

or who needs to approve a purchase.

The correct answer may depend on the department,

purchase type, and purchase amount.

Use information already provided by the user whenever possible.

Ask only for missing information that is necessary to determine

the correct approval requirements.

beginDialog:

kind: OnRecognizedIntent

id: main

intent: {}

actions:

- kind: SearchAndSummarizeContent

id: npDxmT

userInput: "=Concatenate(\"Find the applicable purchase approval requirements. Department: \", Topic.Department, \". Purchase type: \", Topic.PurchaseType, \". Amount: \", Text(Topic.Amount), \" CHF.\")"

inputType:

properties:

Amount:

displayName: Amount

description: The purchase amount in CHF. Use the amount already mentioned by the user if available.

type: Number

Department:

displayName: Department

description: The department that the purchase relates to, for example Finance, HR, or IT. Use the department already mentioned by the user if available.

type: String

PurchaseType:

displayName: Purchase Type

description: The type of purchase, for example Software, Consulting, Equipment, or another purchase category. Use the purchase type already mentioned by the user if available.

type: String

outputType: {}
Suggested answer

sannavajjala87 1,109 Super User 2026 Season 2 on at

Like
a
(0)

Report
Copy link

Link copied!

I agree with @chiaraalina Using topic inputs for key context elements such as Department, Process, or Scenario is a great approach because the agent can automatically extract values already provided by the user and ask follow-up questions only when information is missing. This keeps the conversation more natural, reduces unnecessary prompts, and helps ensure knowledge retrieval is performed with the right context. For highly context-dependent documentation, combining targeted inputs with context-aware retrieval usually produces more reliable answers than a fixed question flow.

Thanks & Regards,

Manoj Annavajjala

www.powerplatformengineer.com