Extract Text from Screen using AI
Leverage the power of AI to extract specific pieces of information from the screen using natural language commands. This is particularly useful when the text you need is part of a larger block or not a distinct UI element.
How to Extract Text with AI:
Describe Your Goal: In the "Extract Text using AI" section, you'll find a text box. Describe what information you want to extract from the current screen and what you'd like to name the variable that will store it.
For example, to get the number of centuries from the article about Virat Kohli and save it to a variable named noOfCenturies, you would type:
Extract the number of centuries by Virat Kohli to variable "noOfCenturies"
Add the Step: Click the Create button at the bottom right.
This will add a new AI-powered step to your test case.
After Execution:
Once the test step is executed, the AI will scan the screen, find the requested information, and populate the following fields:
Variable name: The name of the variable where the text is stored (e.g., noOfCenturies).
Captured text: The actual text that was extracted from the screen.
You can then use this variable in subsequent steps of your test, for example, to validate that the number is correct or to perform other actions.
Crafting Your Prompt
Keep it simple:
Specify what to extract (e.g., username, price, error message)
Name your variable (e.g., currentUser, itemPrice, errorText)
Optional: Ask for processing (e.g.,
uppercase,numbers only)
Format: Extract the [element] to variable [variableName].
Example:
Extract the price and convert to numbers only to variable itemPrice.
3. Real-World Examples
Extract the username to variable currentUser.
currentUser: john.doe
Get the total amount to variable orderTotal.
orderTotal: $149.99
Extract the error message to variable errorText.
errorText: Invalid email
Extract the price and convert to numbers only to variable p.
p: 29.99
Extract the welcome title to variable title in uppercase.
title: WELCOME BACK!
Each example shows exactly what the AI finds and how it assigns the value.
4. Tips for Success
Be specific: If there are multiple similar labels, mention context (e.g., profile section).
Ask processing properly: Perform calculation (for ex. add all items.) or strip symbols in the same request.
Last updated