web
You’re offline. This is a read only version of the page.
close
Skip to main content

Announcements

News and Announcements icon
Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Copilot Studio / Can a Website knowledg...
Copilot Studio
Suggested Answer

Can a Website knowledge source read HTML tag data, not just visible page text?

(0) ShareShare
ReportReport
Posted on by 79
Hi all,
 
I'm trying to get an agent to answer questions using data that only exists in a page's HTML <meta> tags, not in the visible, rendered content a user would see.
 
Example: https://www.solidigm.com/products/data-center/d7/ps1010.html
 
If you view the page source, there's a line like:
<meta name="solidigm:eccn" content="5A992C,5A992"/>
 
So the expected answer for "What is the ECCN for D7-PS1010?" would be 5A992C, but that value isn't shown anywhere in the visible page (the on-page "Compliance Information" table appears to populate separately via JavaScript, and doesn't show this value either). It's only present in the raw HTML source (screenshot attached).
 
Does a Website knowledge source in Copilot Studio index this kind of meta tag content, or does it only pick up rendered/visible text? If it doesn't, what's the recommended way to get an agent to access data like this?
 
Screenshot of page source behind https://www.solidigm.com/products/data-center/d7/ps1010.html
Categories:
I have the same question (1)
  • Suggested answer
    sannavajjala87 Profile Picture
    826 Super User 2026 Season 2 on at
    Hi,
     
    Great question. From my experience with Copilot Studio website knowledge sources, you shouldn't assume that HTML <meta> tags are indexed or available to grounding. The website ingestion process is generally optimized around page content that can be rendered and consumed as document text, and metadata embedded in the HTML source may not be consistently extracted or exposed to the agent.
    In your example, the ECCN value exists in the raw page source (<meta name="solidigm:eccn" content="5A992C,5A992"/>) but isn't clearly exposed as visible page content. If the agent needs to reliably answer questions using that value, I would not depend on website knowledge ingestion alone.
    A few alternatives that tend to work better:
    • Surface the metadata as visible content on the page so it becomes part of the indexed content.
    • Extract the metadata using a scheduled process (Power Automate, Fabric, Azure Function, etc.) and store it in a structured data source that the agent can query.
    • Expose the data through an API and use an action/plugin so the agent retrieves the current value at runtime.
    • Maintain the metadata in a SharePoint list, Dataverse table, or other structured source and use that as a knowledge source instead of relying on HTML source parsing.
    If anyone has tested whether Website knowledge sources currently index custom <meta> tags specifically, I'd be interested in hearing the results. My recommendation for production scenarios would be to treat metadata embedded only in HTML source as non-guaranteed and move critical values into a structured, queryable source.
     
    Thanks & Regards,
    Manoj Annavajjala
  • Suggested answer
    Dua Sadiq Profile Picture
    10 on at

    Hi,

    I tested your premise directly (Screenshots at the end) that a Website knowledge source only reads rendered/visible text and skips <meta> content — and it turns out meta-tag data can surface. Details below.

    Setup note first: you can't add the deep page URL directly — Copilot Studio rejects it with "The website can't end with a file extension" if the URL ends in .html. [screenshot of the error] So I added the parent instead: https://www.solidigm.com/products/. The Public Website source grounds across publicly viewable content under that path, including child pages like the ps1010 page.

    Test 1 — hallucination control. I asked for the ECCN of a fake model, D7-PS9999. The agent didn't invent anything — it said the product doesn't appear to exist. So it genuinely grounds rather than fabricating. [screenshot]

    Test 2 — meta-only values. Asking about the D7-PS1010, the agent returned identifiers that only exist in the raw <meta> tags, not in the visible page (the Compliance table populates via JS and doesn't show these): [screenshot]


    • Compliance: ECCN 5A992C, HTS 8523510000, Self-Classified, License Exception 740.17(b)(1) (from solidigm:eccn / solidigm:usHts / solidigm:ccats)

    • PCN Reference: 0000046032-00 / 62474 – D7-PS1010/D7-PS1030 CookPoint U.2 (from solidigm:pcn)

    •  

    Those aren't visible body text — so the source did index and retrieve meta content.

    Why it works: the Public Website source grounds on Bing's index rather than crawling raw HTML itself, and Bing crawls meta tags. So whether a field surfaces depends on what Bing indexed for that page. That's also a likely reason your original attempt failed — the page or that specific field may not have been in Bing's index yet.

    Important caveat: it's not reliable field-to-field (a specific MMID field didn't come through for me; ECCN/HTS/PCN did), and it's entirely dependent on Bing's crawl. So for compliance-critical values I wouldn't rely on the Website knowledge source in production.

    Reliable approach: read the source deterministically instead — a Power Automate flow (or custom connector) that does an HTTP GET on the page, parses the specific <meta name="solidigm:eccn"> value from the raw HTML, and returns just that string. Add the flow as a tool and instruct the agent to call it for these lookups. That bypasses Bing and reads the tag directly every time.

    Hope this helps !

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Season of Sharing Community Challenge Winners!

Congratulations to our community stars!

Kudos to our 2025 Community Spotlight Honorees

Expanding mentorship, skilling, and AI innovation

Congratulations to the July Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Copilot Studio

#1
Haque Profile Picture

Haque 115 Super User 2026 Season 2

#2
sannavajjala87 Profile Picture

sannavajjala87 114 Super User 2026 Season 2

#3
11manish Profile Picture

11manish 108 Super User 2026 Season 2

Last 30 days Overall leaderboard