Skip to content

Conversation

@quanru
Copy link
Collaborator

@quanru quanru commented Dec 10, 2025

Problem

The previous clearInput implementation used tripleTap on element coordinates, which failed with dynamic input fields like eBay's search box that shows a new input overlay when clicked.

Root cause: When aiInput was called after the dynamic input appeared:

  1. AI located the original search box coordinates (stale)
  2. clearInput used tripleTap on those stale coordinates
  3. The tap hit the wrong element, breaking focus on the dynamic input field
  4. Text input failed

Solution

Replaced the coordinate-based clearing approach with WebDriver's standard Clear API:

// New approach:
1. Get currently focused element using /element/active endpoint
2. Clear it using /element/{id}/clear endpoint

Benefits

Works with dynamic input fields - Uses element references instead of coordinates
No unwanted events - WebDriver's clear is an atomic operation
W3C standard - Uses official WebDriver protocol
Simpler code - Removed tripleTap and backspace fallbacks

Changes

packages/ios/src/ios-webdriver-client.ts

Added three new methods:

  • getActiveElement() - Get WebDriver ID of focused element
  • clearElement(elementId) - Clear element by ID
  • clearActiveElement() - Combined method for easy use

packages/ios/src/device.ts

Simplified clearInput() to use clearActiveElement()

Test files

Updated test files across platforms to use consistent search action descriptions

Testing

Tested with eBay's dynamic search box which previously failed consistently.


Note: This PR cherry-picks the changes from PR #1403 (merged to 1.0 branch) to the main branch.

* fix(ios): use WebDriver Clear API for dynamic input fields

## Problem
The previous clearInput implementation used tripleTap on element coordinates,
which failed with dynamic input fields (e.g., eBay search box that shows a
new input overlay when clicked). The stale coordinates caused the tap to hit
the wrong element, breaking focus on the dynamic input field.

## Solution
Replaced tripleTap-based clearing with WebDriver's standard Clear API:
- Get the currently focused element using /element/active endpoint
- Clear it using /element/{id}/clear endpoint
- Works reliably regardless of element position changes
- Doesn't trigger unwanted web page events

## Changes
- Added getActiveElement(), clearElement(), clearActiveElement() methods to IOSWebDriverClient
- Updated clearInput() in IOSDevice to use WebDriver Clear API
- Updated ebay.test.ts to reflect the fix

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* test(ebay): unify search action descriptions across platforms

Standardize the search action description across all platforms from
"hit Enter" to "click search button" for consistency.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
@netlify
Copy link

netlify bot commented Dec 10, 2025

Deploy Preview for midscene ready!

Name Link
🔨 Latest commit ba6a889
🔍 Latest deploy log https://app.netlify.com/projects/midscene/deploys/6938e0923cb85a0008b77223
😎 Deploy Preview https://deploy-preview-1568--midscene.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@yuyutaotao yuyutaotao merged commit 128776f into main Dec 11, 2025
11 checks passed
@yuyutaotao yuyutaotao deleted the feat/fix-ios-input branch December 11, 2025 03:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants