Skip to content

[Workflow] macOS chromedriver test works in local but fails in GitHub runner-images #408

@ferpasri

Description

@ferpasri

Description
The macOS Chromedriver test works in a local desktop environment but fails in the GitHub runner images. TESTAR is not able to detect the widgets from the GUI state.

The Chromedriver workflow tests on Windows and Ubuntu runner images are functioning properly with the latest TESTAR changes. However, the macOS runner image has been experiencing failures in the Chromedriver workflow test over the past few weeks.

I have been debugging the Chromedriver workflow test on a local macOS laptop. In this environment, TESTAR successfully runs both Spy and Generate modes with the Parabank web application, and the step-by-step execution of the Chromedriver test works as expected.

TESTAR workflow changes
I have decided to mark the Ubuntu and macOS workflow tests as non-mandatory for blocking a pull request.

If a Windows workflow test fails, changes and corrections will be required. However, if an Ubuntu or macOS Chromedriver test fails, we can still gather feedback without blocking the development process of TESTAR.

macOS local working behavior

14.6.1
macos-14-arm64
ChromeDriver 130

macOS runner-images failure behavior

macOS 11 failing

macos-14-arm64/20241022.361

Current runner version: '2.320.0'
14.7 - 23H124
Image: macos-14-arm64
Version: 20241022.361
Google Chrome 130.0.6723.59
ChromeDriver 130.0.6723.58

Last macOS working workflow

macOS 11 working

macos-14-arm64/20240903.5

Current runner version: '2.319.1'
14.6.1 - 23G93
Image: macos-14-arm64
Version: 20240903.5
Google Chrome 128.0.6613.120
ChromeDriver 128.0.6613.119

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions