Answered

Pagination issue for Dataverse connector

(1) Share

Report

Posted on by NT-29050956-0

I am using the Dataverse connector in my Power Automate cloud flows, specifically the List Rows action.

Since I need to fetch more than 5,000 records from my environment, I enabled pagination and set the threshold to 20,000. However, despite this configuration, each time I run the flow, I still receive only 5,000 records. I also tried setting the threshold to 1,500, but the result remains the same(fetching 5,000 records).

Note:- I have around 6,000 records in my table

Any guidance or suggestions on how to retrieve all records beyond 5,000 would be greatly appreciated.

Categories:

General topics

I have the same question (0)

All responses (7)

Answers (1)

Suggested answer

Tomac 3,966 Moderator on at

Like (0)

Report

Pagination for the Dataverse connector rounds to the next highest default page size (5000). So if you set Row Count to 1500 it will return up to 5000. If you set it to 6000 it will return up to 10000.

To get more than the default page size you need to go into the action's Settings and set the Threshold higher:

You can read more about this in Microsoft's official documentation.

If this comment resolved your issue, please remember to mark it as the answer.

1 people found this reply helpful.

Was this reply helpful? Yes No
NT-29050956-0 10 on at

Like (0)

Report

In one of the configurations, I set it to 20,000, and in that case, the Threshold should be 20,000 intact, but even then, I get only 5,000 records, even when I have 6,000+ records. Am I doing anything incorrectly here? I turned the pagination on and set the threshold to 20,000

Was this reply helpful? Yes No
Suggested answer

Tomac 3,966 Moderator on at

Like (0)

Report

@NT-29050956-0 Please try it again, setting both the Row Count and Threshold to 10,000, then use a Compose action to run a count of the returned records and post the results. This works in my testing:

Outputs:

1 people found this reply helpful.

Was this reply helpful? Yes No
NT-29050956-0 10 on at

Like (0)

Report

Observation: This abnormal behavior is observed only when I use FetchXml to filter the records (even if After-Filter has more than 5,000 records as the result); if I clear out the FetchXml, then it works as expected.
The fetchXml I am using:-

<fetch version="1.0" mapping="logical"><entity name="elcn_demoentity"><attribute name="statecode"/><attribute name="elcn_demoentityid"/><attribute name="elcn_name"/><attribute name="createdon"/><order attribute="elcn_name" descending="false"/><filter type="and"><condition attribute="statecode" operator="eq" value="0"/></filter></entity></fetch>

Was this reply helpful? Yes No
Suggested answer

Tomac 3,966 Moderator on at

Like (0)

Report

@NT-29050956-0 In the Microsoft documentation page, it recommends adding a Count argument to your fetch:

1 people found this reply helpful.

Was this reply helpful? Yes No
NT-29050956-0 10 on at

Like (0)

Report

With count set to 100,000, pagination turned on, and threshold set to 15,000, I still get 5,000 records when I use fetchXml (without fetchXml and same config, I get 5998 records.

and if I do not use FetchXml (all other configuration as is):-

Was this reply helpful? Yes No
Verified answer

Tomac 3,966 Moderator on at

Like (0)

Report

@NT-29050956-0

Looks like FetchXML has its own set of pagination requirements.

1 people found this reply helpful.

Was this reply helpful? Yes No