Skip to content

It should be possible to (roughly) prioritize tests. #1541

@onsi

Description

@onsi

By default, Ginkgo treats all tests equally. Ish. Tests decorated with Serial are guaranteed to run at the end of a parallel suite - after all the parallel tests have ended. But aside from that, ginkgo -randomize-all -p will interleave all specs and run them in parallel.

There are cases where this behavior is suboptimal. For example, in a parallelized suite known to have very some slow expensive tests it generally works better to run the slower tests sooner in the process to ensure they are parallelized wrt other tests than to wait till the end and effectively reduce the parallelization of the suite (think "boulders first, then sand").

To support this we could introduce a new Priority(int) decorator that takes an integer. By default all specs have priority 0. By annotating jobs with Priority(N) with N>0 we could then nudge Ginkgo's spec sorter to group specs by priority first then randomize them.

With -randomize-all this is straightforward. Without -randomize-all the specs are grouped by outer container. In this case the priority of the group will be the priority of the highest priority spec in the group. This will also apply to Ordered containers which are always treated as a group even with -randomize-all.

When Priority is defined on both a parent container and a child the innermost Prioirty will win.

/cc @pohly wdyt?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions