Skip to content

Allow ignoring individual vars #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 21, 2025
Merged

Allow ignoring individual vars #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3221be4
Allow ignoring individual vars
dpsutton Aug 19, 2025
afe589a
docs
dpsutton Aug 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,23 @@ And adding it to namespaces like
...)
```

## Skipping vars with options
Sometimes you don't want to modify source to skip a few tests. You can easily specify to skip individual tests with:

```clojure
clj -X:dev:test :only metabase.util.queue-test :ignored '{:vars ["metabase.util.queue-test/bounded-transfer-queue-test" "metabase.util.queue-test/take-batch-test"]}'
```

This allows for exact matching on var names to prevent tests. A typical example might be a test that is failing due to a
leap year. A typical fix might be someone prepares a patch, gets it through CI, and then all jobs hoping to pass CI must
rebase on this change. An alternative is that some central store is updated with

```json
{"ignored": {"vars": ["some.ns/date-test"]}}
```

And then all of CI can carry on while you fix this. Once fixed, remove this var from the central store and continue.

## Only running tests against namespaces or vars with tags

The opposite of `:exclude-tags` -- you can only run tests against a certain set of tags with `:only-tags`. If multiple
Expand Down
15 changes: 14 additions & 1 deletion src/mb/hawk/core.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,17 @@
:only))))]
(or excluded-tag? missing-tag?)))

(defn- ignored-var?
[v options]
(letfn [(var-name [v] (format "%s/%s" (-> v meta :ns ns-name) (-> v meta :name)))]
(contains? (-> options :ignored :vars) (var-name v))))

(defn- find-tests-for-namespace-symbol
[ns-symb options]
(load-test-namespace ns-symb)
(when-not (skip-by-tags? (find-ns ns-symb) options)
(remove #(skip-by-tags? % options)
(remove (some-fn #(skip-by-tags? % options)
#(ignored-var? % options))
(eftest.runner/find-tests ns-symb))))

;; a test namespace or individual test
Expand Down Expand Up @@ -216,11 +222,18 @@
(printf "Starting test iteration #%d\n" i)
(run-tests test-vars options)))))

(defn- normalize-options
"Ensure that ignored vars are a set for quick membership checks."
[options]
(cond-> options
(-> options :ignored :vars seq) (update-in [:ignored :vars] set)))

(defn- find-and-run-tests-with-options
"Entrypoint for the test runner. `options` are passed directly to `eftest`; see https://github.com/weavejester/eftest
for full list of options."
[options]
(let [start-time-ms (System/currentTimeMillis)
options (normalize-options options)
test-vars (find-tests-with-options options)
_ (hawk.hooks/before-run options)
[summary fail?] (try
Expand Down
21 changes: 21 additions & 0 deletions test/mb/hawk/core_test.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
(ns ^:exclude-tags-test ^:mic/test mb.hawk.core-test
(:require
[clojure.set :as set]
[clojure.test :refer :all]
[mb.hawk.core :as hawk]))

(deftest simple
;; need a test that already exists without moving them or circularity
(is (= 1 1)))

(deftest ^:exclude-this-test find-tests-test
(testing "symbol naming"
(testing "namespace"
Expand All @@ -28,6 +33,22 @@
(is (seq tests))
(is (every? var? tests))
(is (contains? (set tests) (resolve 'mb.hawk.core-test/find-tests-test)))))
(testing "ignore var options"
;; note this is a set here. but that gets normalized and the cli can pass a sequence
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of passing a seq of ignored vars in the test, since it mirrors the usual usage more closely?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i want it to be the same, but the entrypoint here means it would convert this to a set many times rather than just once. I normalize the options at the beginning of the find-tests bit, but this multi-method (defmethod find-tests clojure.lang.Symbol gets called once for each symbol; so if a vector comes in i have to turn it into a set each time and that is just very wasteful.

(let [tests (set (hawk/find-tests nil {:ignored {:vars #{"mb.hawk.core-test/simple"}}}))
expected (into #{}
(comp
(keep (fn [[_l v]]
(when (and (-> v meta :test)
(not= (-> v meta :name) 'simple))
(symbol (format "%s/%s"
(-> v meta :ns ns-name)
(-> v meta :name))))))
(map resolve))
(ns-publics 'mb.hawk.core-test))]
(is (set/subset? expected tests))
(is (not (contains? tests #'mb.hawk.core-test/simple)))
(is (contains? tests #'mb.hawk.core-test/find-tests-test))))
(testing "sequence"
(let [tests (hawk/find-tests ['mb.hawk.assert-exprs-test
'mb.hawk.assert-exprs-test/partial=-test
Expand Down
Loading