Skip to content

bug in trec_eval introduced by #2274 #2329

@lintool

Description

@lintool

The problem is with something like:

% python -m pyserini.eval.trec_eval -c -m judged.20 tools/topics-and-qrels/qrels.hc4-neuclir22-fa.test.txt runs/run.index.neuclir22-fa-en.test_title.bm25-default 
runid                 	all	Anserini
num_q                 	all	50
num_ret               	all	50000
num_rel               	all	327
num_rel_ret           	all	238
...

"Anserini" isn't a float, so we get an error:

...
Traceback (most recent call last):
  File "/Users/jimmylin/anaconda3/envs/pyserini-dev6/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/jimmylin/anaconda3/envs/pyserini-dev6/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/jimmylin/workspace/pyserini/pyserini/eval/trec_eval.py", line 166, in <module>
    trec_eval(sys.argv)
  File "/Users/jimmylin/workspace/pyserini/pyserini/eval/trec_eval.py", line 155, in trec_eval
    lines = {line.split("\t")[1]: float(line.split("\t")[2]) for line in output}
  File "/Users/jimmylin/workspace/pyserini/pyserini/eval/trec_eval.py", line 155, in <dictcomp>
    lines = {line.split("\t")[1]: float(line.split("\t")[2]) for line in output}
ValueError: could not convert string to float: 'Anserini'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions