AI Test User API (v1)

Table of Contents

Getting Started

API Endpoints

Method Endpoint Description
POST /api/v1/job Create Job - Creates a new job for your account
GET /api/v1/job?job_id=job-xxxxxxxx Get Job Status - Returns job status and bugs found
POST /api/v1/job/abort?job_id=job-xxxxxxxx Abort Job - Marks a job as failed

Authentication

Job Lifecycle & Statuses

Jobs can have the following status values:

Status Definitions

Abort Behavior

When you abort a job via the /api/v1/job/abort endpoint, the job status is set to failed with an appropriate error message.

Configuration

Browser Types

The browser_type parameter accepts these exact values:

Playwright Code Generation

When using playwright-codegen-firefox, the job generates Playwright code, available on the job summary page.

Summary & Reports

Job responses include comprehensive reporting in multiple formats:

Summary Markdown

Summary URL

Integration

Azure DevOps Integration

🚀 Azure DevOps Pipeline Template

Automatically run AI Test User jobs in your Azure DevOps pipelines. Download our ready-to-use template and start testing in minutes.

Download Template

Quick Setup

1. Download the template above and place it in your repository

2. Set up secret variables in your Azure DevOps project (mark as secret):

3. Create a pipeline using this example:

# azure-pipelines.yml
# Uses the local `template.yml` to run one or more AI Test User jobs.
# Set the secret AITU_BEARER_TOKEN in pipeline variables (mark secret).
name: AITestUser_TestRun_$(Date:yyyyMMdd)$(Rev:.r)

trigger: none

variables:
  # Default values; override at queue-time as needed
  AITU_INITIAL_URL: "https://your-website.com"
  AITU_BROWSER_TYPE: "firefox"
  # Secret variables (set these in your pipeline settings and mark as secret):
  # AITU_BEARER_TOKEN
  # AITU_USERNAME  
  # AITU_PASSWORD
  # AITU_TWOFA_SECRET

pool:
  vmImage: ubuntu-latest

stages:
- stage: RunAITestUser
  displayName: Run AI Test User
  jobs:
  # Parallel runs: duplicate template blocks to scale concurrency
  - template: template.yml
    parameters:
      bearerToken: "$(AITU_BEARER_TOKEN)"
      initialUrl: "$(AITU_INITIAL_URL)"
      userPrompt: "Test the main navigation and ensure all pages load correctly"
      browserType: "$(AITU_BROWSER_TYPE)"
      jobName: "AITestUser_Run_Navigation"
      displayName: "Test Navigation"

  - template: template.yml
    parameters:
      bearerToken: "$(AITU_BEARER_TOKEN)"
      initialUrl: "$(AITU_INITIAL_URL)"
      userPrompt: "Test the login flow with 2FA and then fill out the contact form"
      browserType: "$(AITU_BROWSER_TYPE)"
      credentials: "username: $(AITU_USERNAME), password: $(AITU_PASSWORD)"
      twoFASecret: "$(AITU_TWOFA_SECRET)"
      jobName: "AITestUser_Run_LoginAndContact"
      displayName: "Test Login with 2FA and Contact Form"

  # Add more template blocks as needed for additional test scenarios

API Reference

Endpoints

Create Job POST /api/v1/job

Creates a new job for your account.

Request body (all fields optional; defaults are applied from your account profile):

{
  "initial_url": "https://example.com",
  "browser_type": "firefox",
  "user_prompt": "optional extra instructions",
  "credentials": "username: test@example.com, password: secret123",
  "twofa_secret": "JBSWY3DPEHPK3PXP"
}

Field Descriptions:

Response 201:

{
  "status": "ok",
  "job": {
    "job_id": "job-0d8bc23de98fabcd",
    "status": "ready",
    "created_at": "2025-01-01T12:00:00Z",
    "started_at": null,
    "finished_at": null,
    "error_message": "",
    "summary_markdown": "# ✅\n\n## Job\n- Status: ready\n- Duration: not started\n\n## Bugs\n- None\n\n",
    "summary_url": null
  }
}

Example:

curl -X POST 'https://api.ms.aitestuser.com/api/v1/job' \
  -H 'Authorization: Bearer bearer-<your-token>' \
  -H 'Content-Type: application/json' \
  -d '{
        "initial_url": "https://example.com",
        "browser_type": "firefox",
        "user_prompt": "Please test checkout flow",
        "credentials": "username: test@example.com, password: secret123",
        "twofa_secret": "JBSWY3DPEHPK3PXP"
      }'

Get Job Status GET /api/v1/job?job_id=job-xxxxxxxx

Returns the job's public fields and the bugs found in that run (duplicates included). Includes a single summary_markdown string suitable for Azure Pipelines markdown summary UI. The summary contains a link to the full HTML report when available.

Query params: job_id=job-xxxxxxxxxxxxxxxx (required)

Response 200:

{
  "status": "ok",
  "job": {
    "job_id": "job-0d8bc23de98fabcd",
    "status": "running",
    "created_at": "2025-01-01T12:00:00Z",
    "started_at": "2025-01-01T12:01:00Z",
    "finished_at": null,
    "error_message": "",
    "summary_markdown": "[View full HTML report](https://backend.aitestuser.com/results/xxxxxxxx)\n\n## Job\n- Status: running\n- Duration: 1m23s\n\n## Bugs\n- None\n\n## LLM Report\n...",
    "summary_url": "https://backend.aitestuser.com/results/xxxxxxxx"
  },
  "bugs": [
    {
      "title": "Login button not responsive on mobile",
      "description": "The login button becomes unclickable on screen widths below 768px",
      "category": "UI/UX",
      "severity": 2,
      "page_url": "https://example.com/login"
    }
  ]
}

Bug Fields:

Example:

curl -X GET 'https://api.ms.aitestuser.com/api/v1/job?job_id=job-0d8bc23de98fabcd' \
  -H 'Authorization: Bearer bearer-<your-token>'

Abort Job POST /api/v1/job/abort?job_id=job-xxxxxxxx

Marks a ready or running job as failed. Does not delete it.

Query params: job_id=job-xxxxxxxxxxxxxxxx (required)

Response 200:

{
  "status": "ok",
  "job": {
    "job_id": "job-0d8bc23de98fabcd",
    "status": "failed",
    "created_at": "2025-01-01T12:00:00Z",
    "started_at": "2025-01-01T12:01:00Z",
    "finished_at": "2025-01-01T12:02:30Z",
    "error_message": "Aborted by customer",
    "summary_markdown": "# ❌\n\n## Job\n- Status: failed\n- Duration: 1m30s\n\n## Bugs\n- None\n\n## Error\nAborted by customer",
    "summary_url": null
  }
}

Errors return non-200 status code and JSON:

{ "status": "error", "error": "message" }

Example:

curl -X POST 'https://api.ms.aitestuser.com/api/v1/job/abort?job_id=job-0d8bc23de98fabcd' \
  -H 'Authorization: Bearer bearer-<your-token>'