FinalRun
  • Intro
  • Installation Guide
    • macOS
    • App Permission
  • How to connect to a device?
  • Create steps
    • AI Check / Verify
    • Create local variable
    • Extract text to a variable
      • Using regex for text extraction
    • Add Variable Assertion
  • Advanced
    • Element identifier
    • Create identifier
  • Contact Us
Powered by GitBook
On this page
  • Overview
  • Writing Effective Prompts
  • Supported Evaluations with Examples
  • Unsupported or Problematic Evaluations
  • Summary checklist for your prompt
  1. Create steps

AI Check / Verify

Validate on any element or page content

PreviousCreate stepsNextCreate local variable

Last updated 19 days ago

Overview

FinalRun uses AI to evaluate mobile app screens. When you write a prompt, you are asking the FinalRun AI to confirm whether a particular element or property is present or correct in a given screen. It processes and returns an evaluation based on what is visible in the screenshot.

Here's a video demonstration:-

Writing Effective Prompts

When crafting your prompts, keep the following in mind:

  • Be Specific: Clearly state what element or text you want to check. For example, use "login button" instead of just "button" when possible.

  • Exact Matching: For text-based evaluation, the text must match exactly unless you use qualifiers like "contains". For instance, "Verify that the screen contains 'Welcome back'" will search for the exact phrase.

  • Property Checks: If you are checking for properties such as color, visibility, or badges, ensure the property is clearly visible.

    • Example: "Verify that the button is red" (if the color is visibly red)

  • Ambiguity and Limitations: If your prompt might refer to multiple elements or lacks detail, the system errs on the side of caution. In these cases, the AI will fail, and you will receive a failure message explaining the ambiguity or missing information.

Supported Evaluations with Examples

1. Element Visibility

  • Example: "Verify that the login button is visible"

    • Evaluation:

      • Success: When a single button labeled "login" is clearly visible.

      • Failure: If the button is not visible or if multiple buttons create confusion.

2. Text Content

  • Example: "Verify that the screen contains 'Welcome back'"

    • Evaluation:

      • Success: When the exact phrase "Welcome back" is present in the screenshot.

      • Failure: If the text is missing or appears in a different form.

3. Badge or Numeric Indicators

  • Example: "Verify that the cart icon has a badge with '3'"

    • Evaluation:

      • Success: When the cart icon shows a badge displaying the number 3.

      • Failure: If the badge is absent or displays a different number.

4. Color Checks

  • Example: "Verify that the button is red"

    • Evaluation:

      • Success: When the button’s color is clearly red.

      • Failure: If the color is not red or if the screenshot lacks sufficient color detail.

Unsupported or Problematic Evaluations

  • Ambiguous References: If your prompt uses vague terms (e.g., "the button" when several buttons exist), the system will fail the verification and provide an explanation regarding the ambiguity.

  • Insufficient Screenshot Information: Verification that require data not visibly present in the app screen (e.g., exact color values when the image does not support this detail) will also fail.

  • Irrelevant Check: Any check unrelated to the mobile screen (e.g., "Verify that the sky is blue") will be marked as failed because it does not apply to the mobile UI context.

Summary checklist for your prompt