Building Enterprise AI Agents with Azure AI Search & Copilot Studio

Problem: Copilot Studio can answer questions, but reliable citations are inconsistent - especially when you’re using MCP resources where native citations aren’t yet exposed.
Goal: Ship a lightweight pattern that produces grounded answers with clear, file‑level citations like:
Source: Annual_leave_and_public_holiday_policy.pdf
This post shows how to implement a retrieval‑only copilot backed by Azure AI Search with semantic ranking, chunk‑level enrichment, and a custom prompt pipeline in Copilot Studio.
What you’ll build

• Documents in Azure Blob (PDF policies)
• Indexer pipeline in Azure AI Search that chunks content and keeps source metadata
• Semantic ranker on the index for better relevance (captions/answers optional)
• Copilot Studio agent that calls Search via HTTP → injects top chunks into the LLM with a “cite the source” instruction

• Fixed‑size or layout‑aware chunking works.
• Add small overlap (e.g., 10–15%) to avoid cutting useful context.
• Store chunk text, file name, page, and chunk id in the index.

• id (key)
• content (searchable, retrievable)
• source_file (retrievable, filterable)
• page (int, retrievable)
• title (optional; good for semantic “titleField”)

{
  "name": "policy-index",
  "fields": [
    { "name": "id", "type": "Edm.String", "key": true, "filterable": false, "searchable": false, "retrievable": true },
    { "name": "content", "type": "Edm.String", "searchable": true, "retrievable": true, "analyzer": "en.lucene" },
    { "name": "title", "type": "Edm.String", "searchable": true, "retrievable": true },
    { "name": "source_file", "type": "Edm.String", "searchable": false, "filterable": true, "retrievable": true },
    { "name": "page", "type": "Edm.Int32", "searchable": false, "filterable": true, "retrievable": true }
  ]
}

• To split by pages or sections, use Text Split or Document Layout; both are documented with JSON examples.
• If you need custom metadata to flow alongside chunks, add it in the skillset or via a custom Web API skill.

• Define titleField (optional), contentFields (e.g., content), and keywordFields (e.g., tags).
• You can have multiple semantic configurations and set a default.
• No index rebuild is required to add or update the configuration.

Example: semantic configuration JSON

{
  "name": "policies-semantic",
  "semantic": {
    "configurations": [
      {
        "name": "policies-semantic",
        "prioritizedFields": {
          "titleField": { "fieldName": "title" },
          "contentFields": [
            { "fieldName": "content" }
          ],
          "keywordsFields": []
        }
      }
    ]
  }
}

Step 5 — Query pattern from Copilot Studio (HTTP action)
In Copilot Studio, add a Send HTTP request node to call your Azure AI Search endpoint. Select GET (or POST for complex queries), include the api‑key header or use Entra ID where supported, and parse the JSON response into a variable for your prompt.
POST https://<your-search>.search.windows.net/indexes/policy-index/docs/search?api-version=2024-07-01
api-key: <key>
Content-Type: application/json

{
  "queryType": "semantic",
  "semanticConfiguration": "policies-semantic",
  "search": "{userQuestion}",
  "select": "id, content, source_file, page",
  "top": 5,
  "queryLanguage": "en-us"
}

Step 6 — Prompt the LLM to summarize + cite
In your Copilot flow, take the top N chunks from the HTTP response and pass them to the LLM with a strict instruction, for example:
SYSTEM INSTRUCTIONS
You are a policy assistant. Use ONLY the supplied chunks to answer.
For each statement of fact, ensure it is supported by the chunks.
At the end, print:
"Source: <source_file>" (one line per unique file).
If page numbers are present, include them: "Source: <file>, p.<page>".
If you cannot find the answer in the chunks, say you don’t know.

• No results / poor grounding? Inspect the index: are content fields searchable & retrievable? Did the skillset actually split the PDFs? Use Search Explorer to verify.
• Inconsistent page numbers? Use Text Split in pages mode or combine with Document Layout → add page to the projection.
• HTTP node mapping in Copilot Studio: define a sample JSON schema so the response is typed in Power Fx and easy to bind into your prompt.
• Security: Use role‑based access (Entra ID) instead of keys where possible, per the semantic quickstart guidance.

🔗 Want to explore or reuse the full implementation?
I’ve published the complete Azure AI Search + Copilot Studio project here:
https://github.com/RichaPandit/Azure-AI-Search-Copilot-Studio-citations-prototype.git
Conclusion
Until Copilot Studio exposes uniform, platform‑level citations across all resources and orchestrations, this pattern gives you reliable, auditable answers with human‑readable sources and it’s deployable in an afternoon.