web
You’re offline. This is a read only version of the page.
close
Skip to main content

Announcements

News and Announcements icon
Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Power Automate / Power Automate Desktop...
Power Automate
Suggested Answer

Power Automate Desktop Image Recognition Misselects List Items – Requires N Retries for Nth Row

(1) ShareShare
ReportReport
Posted on by 399

As shown in the attached image, the target is the HK logo or stock - Hong Kong in line N.

📝 Issue Description

When using “Move mouse to image” in Power Automate Desktop, if the target image is located in the Nth row of a dropdown list, the action consistently misselects earlier rows. The flow only succeeds after N retries, effectively requiring one retry per row above the target.

Attempts to adjust parameters (similarity threshold, search scope, multiple image variants, wait delays) do not resolve the issue. Even when capturing both the left-side icon and the text (“Stock – Hong Kong”), the misselection persists.

 

🔍 Expected Behavior

The image recognition should globally search the screen and directly return the most accurate match of the target image, regardless of its row position in the list.

 

⚠️ Actual Behavior

  • First row target → selected correctly.

  • Target in Nth row → requires N retries before correct selection.

  • Behavior resembles “stepwise row elimination” rather than true image matching.

 

📌 Impact

  • Increases automation complexity and runtime.

  • Forces users to design error loops, reducing reliability.

  • Cannot be corrected through configuration, indicating a deeper algorithmic flaw.

 

💡 Suggested Improvements

  • Fix the image recognition algorithm to perform global best-match search instead of row-by-row priority.

  • Provide an option for multi-image simultaneous matching (e.g., both icon and text must match before returning).

  • Enhance recognition stability in dynamic list scenarios.

螢幕擷取畫面 2026-06-05...
螢幕擷取畫面 2026-06-06...

Your file is currently under scan for potential threats. Please wait while we review it for any viruses or malicious content.

I have the same question (0)
  • Suggested answer
    eetuRobo Profile Picture
    4,545 Super User 2026 Season 1 on at
    Your images did not load yet so I don't have all the context. 

    But if I understand you correctly you have rows that have drop down list which you want your automation to click. And you have used "Move mouse to image" -action on it. But since there are many rows with identical image that you are trying to click then your automation always selects the first it finds. Most likely its because your action is set up to find the first occurrence of the image that it finds.



    So if you change that to a different number like 2 then it will Move mouse to the second image it can find.
    Unfortunately you cant insert variable into that (atleast in my 2.68 version of PAD). Just numbers. So by its basic features it doesn't really let you make it dynamic like referring to a variable.
    You can add variable to it if you copy the action to text editor and add the variable name there after "Occurence:". So replace the 1 with the name of the variable.


    Then just copy and paste  that action back to your PAD editor. This point if you open the action the Occurrence -field is empty but it does actually contain the variable. So this would find in first loop the first occurrence of the image and then in the second loop the second occurrence:


    Notice that if the image you are moving over is an element that changes like gets highlighted or tool tip is popping over it then that image is not the same anymore. So then there would be one less image for the action to find. So that might mess with it too. Hard to explain but hopefully you get what I mean :D 

    Tip:
    If possible always use actions that refer to element rather than image recognition. It will be much more reliable. So if it a web site use "Click link on web page" or "Press button on web page" -actions or "Set drop-down list value on web page". Or if its stand alone app then use similar UI automation -actions.
  • alee001 Profile Picture
    399 on at
     Thank you for your attention.

    I have tried different actions, settings, and parameters, but the problem persists.

    The issue lies in the behavior of this action. For example, in a test with a loop: move the mouse to the image (input box) > press the send button (enter number) > move the mouse to the image (find the target in the displayed list and press it), even though the contents of the list are the same each time, it executes the action back and forth from the top to the Nth row of the list instead of directly finding the target row and executing it.

    Of course, I know there are other methods to replace moving the mouse to the image, but this strange behavior seems to only occur in the list. I cannot understand its unexpected operation; perhaps it is a bug?
  • Suggested answer
    Valantis Profile Picture
    6,025 on at
     
    This isn't a bug in the image recognition algorithm, it's how "Move mouse to image" works by design. When multiple identical images exist on screen (every row in a list has the same icon), PAD matches them in screen order top-to-bottom and picks occurrence 1 by default. That's why the Nth row requires N retries.

    The variable occurrence workaround eetuRobo described works but is fragile if items get highlighted or tooltips appear.
    The better solution for a dropdown list: instead of image recognition, use the UI element approach.

    For web-based dropdowns: Use "Set drop-down list value on web page" or "Click link on web page" targeting the specific list item by its text value.
    For native Windows dropdowns: Use "Select list item" or "Send keys" after clicking the dropdown open.

    If those don't work because PAD misidentifies the element type (button vs dropdown as you mentioned earlier): click the dropdown to open it, then use "Move mouse to text on screen" targeting the specific text of the stock name rather than its icon. Text matching is more unique than icon matching in a list.
     

     

    Best regards,

    Valantis

     

    ✅ If this helped solve your issue, please Accept as Solution so others can find it quickly.

    ❤️ If it didn’t fully solve it but was still useful, please click “Yes” on “Was this reply helpful?” or leave a Like :).

    🏷️ For follow-ups  @Valantis.

    📝 https://valantisond365.com/

    💼 LinkedIn

    ▶️ YouTube

     
     
  • alee001 Profile Picture
    399 on at
    Valantis, thank you for your response.

    The fact that the action can recognize the HK flag in rows N of the list and execute the loop from top to bottom indicates that it's not an image recognition issue but a bug in the operation.

    Since the flags, names, etc., on the list are all independent UI elements, finding the target and executing the click action would be difficult or complex. Therefore, image recognition (moving the mouse over the image) is used.

    I tried using "Retrieve Data from Webpage" to convert the list into an index, but it failed because the Ordinal of the selector in "Click a Link on a Webpage" does not accept index variable input.

    Still trying other methods…

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Season of Sharing Community Challenge Launch!

Jump in, show your community spirit, and win prizes!

Kudos to our 2025 Community Spotlight Honorees

Expanding mentorship, skilling, and AI innovation

Congratulations to the April Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Power Automate

#1
Vish WR Profile Picture

Vish WR 450

#2
Haque Profile Picture

Haque 366

#3
Valantis Profile Picture

Valantis 349

Last 30 days Overall leaderboard