AI Assertion

Assert or validate on any element or page content

Overview

FinalRun uses AI to evaluate assertions on mobile app screens. When you write an assertion prompt, you are asking the FinalRun AI to confirm whether a particular element or property is present or correct in a given screen. It processes the assertion and returns an evaluation based on what is visible in the screenshot.

Here's a video demonstration:-

---

Writing Effective Assertion Prompts

When crafting your assertion prompts, keep the following in mind:

• Be Specific: Clearly state what element or text you want to check. For example, use "login button" instead of just "button" when possible.

• Exact Matching: For text-based assertions, the text must match exactly unless you use qualifiers like "contains". For instance, "Assert that the screen contains 'Welcome back'" will search for the exact phrase.

• Property Checks: If you are checking for properties such as color, visibility, or badges, ensure the property is clearly mentioned.

  • Supported: "Assert that the button is red" (if the color is visibly red)

  • Not Supported: "Assert that the button is green" when the screenshot does not provide explicit color data will result in failure.

• Ambiguity and Limitations: If your prompt might refer to multiple elements or lacks detail, the system errs on the side of caution. In these cases, the assertion will fail, and you will receive a failure message explaining the ambiguity or missing information.

---

Supported Assertions with Examples

1. Element Visibility

• Example: "Assert that the login button is visible"

  • Evaluation:

    • Success: When a single button labeled "login" is clearly visible.

    • Failure: If the button is not visible or if multiple buttons create confusion.

2. Text Content

• Example: "Assert that the screen contains 'Welcome back'"

  • Evaluation:

    • Success: When the exact phrase "Welcome back" is present in the screenshot.

    • Failure: If the text is missing or appears in a different form.

3. Badge or Numeric Indicators

• Example: "Assert that the cart icon has a badge with '3'"

  • Evaluation:

    • Success: When the cart icon shows a badge displaying the number 3.

    • Failure: If the badge is absent or displays a different number.

4. Color Checks

• Example: "Assert that the button is red"

  • Evaluation:

    • Success: When the button’s color is clearly red.

    • Failure: If the color is not red or if the screenshot lacks sufficient color detail.

---

Unsupported or Problematic Assertions

• Ambiguous References: If your prompt uses vague terms (e.g., "the button" when several buttons exist), the system will fail the assertion and provide an explanation regarding the ambiguity.

• Insufficient Screenshot Information: Assertions that require data not visibly present in the screenshot (e.g., exact color values when the image does not support this detail) will also fail.

• Irrelevant Assertions: Any assertion unrelated to the mobile screen (e.g., "Assert that the sky is blue") will be marked as failed because it does not apply to the mobile UI context.

---

Summary Checklist for Your Assertion Prompts

• Specific Element Identification: Clearly identify the element(s) you are referencing.

• Exact Text and Properties: Use precise wording for text matching and property verification.

• Consider Ambiguity: Avoid ambiguous terms; if multiple similar elements exist, refine your prompt.

• Verify Relevance: Ensure the assertion is directly related to the mobile screen content.

• Accept Limitations: Understand that if the screenshot lacks necessary detail, the assertion might fail.