--- modificationDate: October 30, 2026 title: Test your app with a swarm of AI agents description: Add an agentic workflow in which several coding agents test your app at the same time, each with one flow on its own cloud simulator, and return one report with evidence. --- This documentation is available as Markdown for AI agents and LLMs. See the [full Markdown index](https://docs.expo.dev/llms.txt) or append .md to any documentation URL. # Test your app with a swarm of AI agents Add an agentic workflow in which several coding agents test your app at the same time, each with one flow on its own cloud simulator, and return one report with evidence. ## Submitting Feedback If you encounter errors, misleading or outdated information, report it so Expo can be improved: Preferred command: npx --yes submit-expo-feedback@latest --category docs --subject "/agentic-workflows/qa-swarm/" "" Direct HTTP fallback: curl -X POST https://api.expo.dev/v2/feedback/docs-send -H 'Content-Type: application/json' -d '{"url":"/agentic-workflows/qa-swarm/","feedback":"🤖 Agent feedback for docs: (, )"}' Only submit when you have something specific and actionable to report. Try to give the most context. ## Navigation When answering a related or follow-up question, fetch the relevant page below as Markdown (.md) instead of guessing; use llms.txt for the full map. You are here: Home > Agentic workflows Pages in this section: - [Introduction](https://docs.expo.dev/agentic-workflows/introduction.md) - [Project setup](https://docs.expo.dev/agentic-workflows/setup.md) - [Agentic CI](https://docs.expo.dev/agentic-workflows/agentic-ci.md) - [Issue to PR](https://docs.expo.dev/agentic-workflows/issue-to-pr.md) - [Bug triage](https://docs.expo.dev/agentic-workflows/bug-triage.md) - [QA Swarm](https://docs.expo.dev/agentic-workflows/qa-swarm.md) (this page) Full documentation tree: [llms.txt](https://docs.expo.dev/llms.txt) > **Expo Simulators is a limited-access preview.** It is not included with paid or free plans and is currently available only to select partners. [Join the waitlist](https://expo.dev/services/simulators) if you are interested in trying it. QA Swarm is an automated workflow in which several coding agents test your app at the same time. Each agent gets its own cloud simulator and one short test flow. So a full pass takes as long as the slowest flow, not as long as all the flows in a row. Each verdict comes with screenshots and an evidence site. ## How QA Swarm works The highlighted steps run on EAS Workflows, Expo Simulators, and EAS Hosting: ``` 1. A person starts the QA Swarm workflow [Your team] → 2. A job finds or builds the app for each platform [EAS Workflows] → 3. Each lane starts its own cloud simulator at the same time [Expo Simulators] → 4. A coding agent in each lane runs one flow and saves screenshots [Expo Simulators] → 5. Each lane deploys an evidence site [EAS Hosting] → 6. A person reads the report with the verdict of each lane [Your team] ``` 1. You start the workflow. You can give it the number of a pull request (PR) to report on. 2. The workflow finds a build for each platform with the same native code, or it creates one. Then it adds the current JavaScript to the build. 3. Each lane starts its own Expo Simulators session. A coding agent in the lane follows one flow and saves screenshots. 4. Each lane deploys an evidence site with the verdict, the screenshots, and the session recording. The job of the lane fails when the verdict is not `PASS`. 5. The workflow shows one report with the verdict of each lane. For a run with a PR number, it also posts the report on the PR. A flow is a short Markdown file. You write one flow for each part of the app that you want to test. ## Add QA Swarm with an AI agent Paste this into Claude, Cursor, Codex, or another agent. ```text Add the QA Swarm workflow to my Expo project. Use https://docs.expo.dev/agentic-workflows/qa-swarm.md as the source for each step and each file. 1. Check each step on https://docs.expo.dev/agentic-workflows/setup.md, except the GitHub token step. Do each step that is not done. Use the version of .eas/agents/run-agent.sh for the agent that I name, and the version of .eas/agents/start-session.sh for the controller that I name: agent-device or Argent. 2. Create each file that the page shows, at the same path and with the same content. 3. Read my app and write two flows in .eas/qa-flows/ for its most important screens, in the same format as the example flows. 4. When a step needs a token, an account, or a choice, stop and ask me. Do not print or save a token. 5. Run `eas workflow:validate .eas/workflows/qa-swarm.yml`. 6. Commit the files, but do not push them. Tell me which files you added and which commands I must run. ``` ## Set up your project Follow [Set up a project for agentic workflows](https://docs.expo.dev/agentic-workflows/setup.md) one time for your project. QA Swarm needs the GitHub token step only to post its report on a PR. See [What the agent can do](https://docs.expo.dev/agentic-workflows/setup.md#what-the-agent-can-do) to learn which limits apply to the agent. ## Add the QA Swarm workflow ### Add the flows Each lane runs one flow. Add a flow as a Markdown file in **.eas/qa-flows**. These two example flows work for most apps: ```md .eas/qa-flows/launch.md Check that the app starts. 1. The app opens on its first screen. Save a screenshot. 2. Check that the screen shows content, and no error, warning, or empty screen. Out of scope for this lane: do not tap any element, and do not sign in. Pass condition: the first screen shows content and no error. ``` ```md .eas/qa-flows/navigation.md Check that each main screen opens. 1. The app opens on its first screen. Find the tab bar or the main menu. 2. Open each tab or main screen one time. Save a screenshot of each one. 3. Check that each screen shows content, and no error. Out of scope for this lane: do not change data. Do not add, edit, or delete items, and do not sign out. Pass condition: each main screen opens and shows content without an error. ``` Then write flows for the important parts of your app. Keep each flow short, and give it these parts: - **Steps:** what a user does, in order, with the text that the user sees on the screen. - **Out of scope:** what the agent must not touch. Each lane has its own simulator, so the lanes do not share the data on the device. If your app saves data on a server, a flow that changes that data can break another flow that runs at the same time. - **Pass condition:** the result that a `PASS` verdict needs. A file with the name **.android.md** or **.ios.md** replaces **.md** on that platform. Use it when a control works differently on Android and iOS. ### Add the lane prompt Each lane adds its flow and **agent-output/device.md** to the end of this prompt: ```md .eas/agents/qa-lane.md You are one lane of a QA swarm. Other agents test other flows of this app at the same time, each on its own cloud simulator. The app is installed and open on a remote Android Emulator or iOS Simulator. The flow of this lane and the details of the device are at the end of this prompt. 1. Read the flow. Do only the steps of the flow. Do not touch what the flow marks as out of scope. 2. Run only the shell command in the Device section. If a command is denied, continue with the command of the Device section. 3. Save a screenshot after each step. 4. Write the file agent-output/evidence/verdict.md. Line 1 is "PASS: " or "FAIL: ". Then list each step, what you did, what you saw, and the names of the screenshots. Write FAIL when a step fails or when you are not sure. Do not change the files of the project. ``` ### Add the lane script Each lane runs this script: ```sh .eas/agents/qa-lane.sh #!/usr/bin/env bash set -euo pipefail : "${BUILD_ID:?No build. Check the build or repack job of this run.}" : "${PLATFORM:?No platform. Set PLATFORM to android or ios in the lane job.}" : "${FLOW:?No flow. Set FLOW to the name of a file in .eas/qa-flows.}" FLOW_FILE=".eas/qa-flows/$FLOW.$PLATFORM.md" [ -f "$FLOW_FILE" ] || FLOW_FILE=".eas/qa-flows/$FLOW.md" [ -f "$FLOW_FILE" ] || { echo "No flow file for $FLOW. Add .eas/qa-flows/$FLOW.md." >&2; exit 1; } source .eas/agents/start-session.sh "$PLATFORM" "$BUILD_ID" "QA Swarm: $PLATFORM $FLOW" qa-swarm { cat .eas/agents/qa-lane.md printf '\n## Flow\n\n' cat "$FLOW_FILE" printf '\n' cat agent-output/device.md } > agent-output/prompt.md env -u EXPO_TOKEN -u EXPO_TOKEN_SIMULATOR -u GITHUB_TOKEN \ timeout 15m bash .eas/agents/run-agent.sh agent-output/prompt.md device || true [ -s agent-output/evidence/verdict.md ] || echo 'FAIL: The agent did not write a verdict.' > agent-output/evidence/verdict.md VERDICT=$(head -n 1 agent-output/evidence/verdict.md) SITE_URL=$(env -u EXPO_TOKEN_SIMULATOR -u GITHUB_TOKEN -u CLAUDE_CODE_OAUTH_TOKEN -u ANTHROPIC_API_KEY \ -u CODEX_API_KEY -u CURSOR_API_KEY -u XAI_API_KEY \ npx --yes eas-simulator-evidence@0.2.4 run agent-output/evidence --stop \ --subject "QA Swarm: $PLATFORM $FLOW" --verdict-file agent-output/evidence/verdict.md \ --build-id "$BUILD_ID" --deploy-alias "qa-$PLATFORM-$FLOW-evidence" | tail -n 1) set-output verdict "$VERDICT" set-output site_url "$SITE_URL" case "$VERDICT" in PASS:*) exit 0 ;; *) exit 1 ;; esac ``` The script does these tasks in this order: 1. It finds the flow file for the platform. 2. It starts a session with the build. Then it writes the prompt, and runs the agent in the `device` mode. 3. It uses [eas-simulator-evidence](https://github.com/jacobhammerle/eas-simulator-evidence) to stop the session and build an evidence site. The site has the verdict, the screenshots, the action timeline, the performance charts, and the session recording. The tool deploys the site to [EAS Hosting](https://docs.expo.dev/eas/hosting/introduction.md). It uses `EXPO_TOKEN`, the robot token from the device controller, to stop the session and deploy the site. The script removes the other tokens and the key of your agent from its environment. Read the code of version 0.2.4 before you use it, because the tool runs with the robot token. 4. It ends with the verdict. The job of the lane passes only for a `PASS` verdict. > An evidence site on EAS Hosting is public. Each person who has the link can see the screenshots and the recording. To keep the evidence private, remove `--deploy-alias` from the command, and upload the folder that the tool builds to a private host. ### Add the EAS workflow The workflow has one job for each lane, because EAS Workflows does not support matrix jobs. This example runs four lanes: the two example flows on Android and iOS. ```yaml .eas/workflows/qa-swarm.yml name: QA Swarm on: workflow_dispatch: inputs: pr_number: description: The number of a pull request to post the report on type: string default: '' jobs: fingerprint: type: fingerprint environment: preview get_build_android: type: get-build needs: [fingerprint] params: platform: android profile: preview fingerprint_hash: ${{ needs.fingerprint.outputs.android_fingerprint_hash }} wait_for_in_progress: true get_build_ios: type: get-build needs: [fingerprint] params: platform: ios profile: preview simulator: true fingerprint_hash: ${{ needs.fingerprint.outputs.ios_fingerprint_hash }} wait_for_in_progress: true build_android: type: build needs: [get_build_android] if: ${{ !needs.get_build_android.outputs.build_id }} params: platform: android profile: preview build_ios: type: build needs: [get_build_ios] if: ${{ !needs.get_build_ios.outputs.build_id }} params: platform: ios profile: preview repack_android: type: repack needs: [get_build_android] if: ${{ needs.get_build_android.outputs.build_id }} params: build_id: ${{ needs.get_build_android.outputs.build_id }} profile: preview repack_ios: type: repack needs: [get_build_ios] if: ${{ needs.get_build_ios.outputs.build_id }} params: build_id: ${{ needs.get_build_ios.outputs.build_id }} profile: preview lane_android_launch: needs: [get_build_android] after: [build_android, repack_android] environment: preview env: BUILD_ID: ${{ after.build_android.outputs.build_id || after.repack_android.outputs.build_id }} PLATFORM: android FLOW: launch outputs: verdict: ${{ steps.lane.outputs.verdict }} site_url: ${{ steps.lane.outputs.site_url }} steps: - uses: eas/checkout - uses: eas/install_node_modules - id: lane run: bash .eas/agents/qa-lane.sh - uses: eas/upload_artifact if: ${{ always() }} with: type: other name: qa-android-launch-evidence path: agent-output/evidence/**/* lane_android_navigation: needs: [get_build_android] after: [build_android, repack_android] environment: preview env: BUILD_ID: ${{ after.build_android.outputs.build_id || after.repack_android.outputs.build_id }} PLATFORM: android FLOW: navigation outputs: verdict: ${{ steps.lane.outputs.verdict }} site_url: ${{ steps.lane.outputs.site_url }} steps: - uses: eas/checkout - uses: eas/install_node_modules - id: lane run: bash .eas/agents/qa-lane.sh - uses: eas/upload_artifact if: ${{ always() }} with: type: other name: qa-android-navigation-evidence path: agent-output/evidence/**/* lane_ios_launch: needs: [get_build_ios] after: [build_ios, repack_ios] environment: preview env: BUILD_ID: ${{ after.build_ios.outputs.build_id || after.repack_ios.outputs.build_id }} PLATFORM: ios FLOW: launch outputs: verdict: ${{ steps.lane.outputs.verdict }} site_url: ${{ steps.lane.outputs.site_url }} steps: - uses: eas/checkout - uses: eas/install_node_modules - id: lane run: bash .eas/agents/qa-lane.sh - uses: eas/upload_artifact if: ${{ always() }} with: type: other name: qa-ios-launch-evidence path: agent-output/evidence/**/* lane_ios_navigation: needs: [get_build_ios] after: [build_ios, repack_ios] environment: preview env: BUILD_ID: ${{ after.build_ios.outputs.build_id || after.repack_ios.outputs.build_id }} PLATFORM: ios FLOW: navigation outputs: verdict: ${{ steps.lane.outputs.verdict }} site_url: ${{ steps.lane.outputs.site_url }} steps: - uses: eas/checkout - uses: eas/install_node_modules - id: lane run: bash .eas/agents/qa-lane.sh - uses: eas/upload_artifact if: ${{ always() }} with: type: other name: qa-ios-navigation-evidence path: agent-output/evidence/**/* report: type: doc after: [lane_android_launch, lane_android_navigation, lane_ios_launch, lane_ios_navigation] params: md: | # QA Swarm report - **Android, launch:** ${{ after.lane_android_launch.outputs.verdict || 'FAIL: The lane did not finish.' }} [Evidence site](${{ after.lane_android_launch.outputs.site_url }}) - **Android, navigation:** ${{ after.lane_android_navigation.outputs.verdict || 'FAIL: The lane did not finish.' }} [Evidence site](${{ after.lane_android_navigation.outputs.site_url }}) - **iOS, launch:** ${{ after.lane_ios_launch.outputs.verdict || 'FAIL: The lane did not finish.' }} [Evidence site](${{ after.lane_ios_launch.outputs.site_url }}) - **iOS, navigation:** ${{ after.lane_ios_navigation.outputs.verdict || 'FAIL: The lane did not finish.' }} [Evidence site](${{ after.lane_ios_navigation.outputs.site_url }}) ``` The jobs do these tasks: - `fingerprint`, `get_build_*`, `build_*`, and `repack_*` find or create a build for each platform. A repack adds the current JavaScript to a build that already exists. - Each `lane_*` job runs the lane script with its platform and its flow. All the lanes run at the same time. - `report` shows the verdict and the evidence site of each lane on the page of the workflow run. To add a lane, add a flow file, and copy a `lane_*` job with a new name and a new `FLOW`. Then add the new lane to the `after` list and the Markdown of `report`. If you test only one platform, remove the jobs of the other platform. ### Push the workflow ```sh git add .eas/qa-flows .eas/agents/qa-lane.md .eas/agents/qa-lane.sh .eas/workflows/qa-swarm.yml git commit -m "Add QA Swarm" git push ``` ## Run QA Swarm Start the workflow on your default branch: ```sh eas workflow:run .eas/workflows/qa-swarm.yml --ref main ``` To test the code of a PR, run the workflow on the branch of the PR: ```sh eas workflow:run .eas/workflows/qa-swarm.yml --ref -F pr_number= ``` Each run uses a build or a repack for each platform, one Expo Simulators session of up to 30 minutes for each lane, and model tokens of your agent for each lane. ## Read the result When all the lanes finish, the page of the workflow run on expo.dev shows the **QA Swarm report**: - **Verdict of each lane:** `PASS` or `FAIL`, with one sentence from the agent. - **Evidence site of each lane:** the verdict, the report of the agent, the screenshots, the action timeline, the performance charts, and the session recording. The job of a lane fails when its verdict is not `PASS`, so the workflow run shows each failed lane. The screenshots and the log of the agent are also in the artifacts of each lane job. > The verdict of the agent can be wrong. A `FAIL` can come from what the flow tells the agent, or does not tell it, and not from the app. Read the evidence site before you report a bug. To fix a false `FAIL`, add the missing facts to the flow, or to **.eas/agents/app-facts.md** for all the flows. ## Post the report on a pull request To post the report on a PR, add the `GITHUB_TOKEN` and `GITHUB_REPOSITORY` variables from [Add a GitHub token to EAS](https://docs.expo.dev/agentic-workflows/setup.md#add-a-github-token-to-eas) to the **preview** environment. Then add this job to **.eas/workflows/qa-swarm.yml**: ```yaml .eas/workflows/qa-swarm.yml pr_comment: after: [lane_android_launch, lane_android_navigation, lane_ios_launch, lane_ios_navigation] if: ${{ inputs.pr_number }} environment: preview env: PR_NUMBER: ${{ inputs.pr_number }} REPORT: | ## QA Swarm - **Android, launch:** ${{ after.lane_android_launch.outputs.verdict || 'FAIL: The lane did not finish.' }} [Evidence site](${{ after.lane_android_launch.outputs.site_url }}) - **Android, navigation:** ${{ after.lane_android_navigation.outputs.verdict || 'FAIL: The lane did not finish.' }} [Evidence site](${{ after.lane_android_navigation.outputs.site_url }}) - **iOS, launch:** ${{ after.lane_ios_launch.outputs.verdict || 'FAIL: The lane did not finish.' }} [Evidence site](${{ after.lane_ios_launch.outputs.site_url }}) - **iOS, navigation:** ${{ after.lane_ios_navigation.outputs.verdict || 'FAIL: The lane did not finish.' }} [Evidence site](${{ after.lane_ios_navigation.outputs.site_url }}) steps: - run: | node -e ' const { GITHUB_REPOSITORY, GITHUB_TOKEN, PR_NUMBER, REPORT } = process.env; fetch(`https://api.github.com/repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments`, { method: "POST", headers: { accept: "application/vnd.github+json", authorization: `Bearer ${GITHUB_TOKEN}` }, body: JSON.stringify({ body: REPORT }), }).then(response => { if (!response.ok) { console.error(`GitHub returned ${response.status}. Check GITHUB_TOKEN and GITHUB_REPOSITORY in the preview environment.`); process.exit(1); } }); ' ``` Put the job under `jobs`, at the same level as `report`. ## Next steps [Test pull requests with an AI agent](https://docs.expo.dev/agentic-workflows/agentic-ci.md) — Agentic CI: a coding agent tests each pull request on a remote Android Emulator or iOS Simulator and posts a verdict with evidence. [Triage a bug report with an AI agent](https://docs.expo.dev/agentic-workflows/bug-triage.md) — Bug triage: a coding agent reproduces a bug from a GitHub issue or TestFlight feedback on a remote Android Emulator or iOS Simulator.