Skip to content

Integration tests for the visualisation commands. #356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 79 commits into from
Closed

Integration tests for the visualisation commands. #356

wants to merge 79 commits into from

Conversation

istefano82
Copy link
Contributor

Code Pull Requests

  • The module tests end to end Ludwig visualisation commands execution and asserts proper amount of visualisation plots are saved as PDF and PNG formats.

Documentation Pull Requests

  • 22 Tests are developed that cover all available visualisations
  • Two changes are made to the ludwig/visualize.py that were causing errors for 2 of the visualisations

All tests are passing.

  • compare_classifiers_from_pred visualisation works only with prediction.csv and fails when using prediction.npy

istefano82 and others added 30 commits May 18, 2019 17:45
@istefano82
Test visualisation of learning curves using CLI command the visualisa…
2cff6bf 
…tion module call.
Test png and pdf saving of learning curves visualisations.
dd60cfd
Remove the system call and module call for visualisation tests.
6bcbe7b
Test saving of confusion matrix visualisations as pdf and png.
87947cf
Test saving of compare model perfomance visualisations as pdf and png.
d4ea650
Test skeleton saving of compare model perfomance probvisualisations …
3cfc318 
…as pdf.
@istefano82
Test generating and saving compare perfomance prob figures as pdf and…
14a928e 
… png.
@istefano82
Test generating and saving compare perfomance pred figures as pdf and…
ee6fd6d 
… png.
@istefano82
Refactor some test names to reflect they are testing classifier compa…
47a6b74 
…rision.
@istefano82
Test generating and saving compare classifier performance from subset…
11d3408 
… figures as pdf and png.
Test visualisation compare_classifiers_changing_k_output as pdf and png.
7442e28
Test visualisation compare_classifiers_multimetric_output as pdf and …
0a6e334 
…png.
Test visualisation compare_classifiers_predictions_output as pdf and …
e23654a 
…png.
@istefano82
Refactor test suite to test pdf and png output in each test case.
79f0e47
@istefano82
Refactor test to PEP8 standarts.
234f6a1
@istefano82
Refactor visualise confusion matrix to test both pdf and png outputs.
1473d27
@istefano82
Refactor visualise compare performance to test both pdf and png outputs.
c98a182
@istefano82
Refactor visualise compare classifiers from prob to test both pdf and…
3902cd0 
… png outputs.
@istefano82
Refactor visualise compare classifiers from pred to test both pdf and…
01bb7e9 
… png outputs.
@istefano82
Refactor test docstrings to describe the input parameters.
de999ec
@istefano82
Refactor visualise compare classifiers from subset to test both pdf a…
f99c66b 
…nd png outputs.
@istefano82
Refactor visualise compare classifiers changing k to test both pdf an…
6e541f3 
…d png outputs.
@istefano82
Refactor visualise compare classifiers multiclass multimetric to test…
fadec9d 
… both pdf and png outputs.
@istefano82
Refactor visualise compare classifiers predictions to test both pdf a…
bdaf103 
…nd png outputs.
@istefano82
Refactor visualise compare classifiers predictions distribution. to t…
ebd344c 
…est both pdf and png outputs.

Test fails.
@istefano82
Refactor visualise confidence treshholding to test both pdf and png o…
32546e9 
…utputs.
@istefano82
Refactor visualise confidence treshholding data vs acc to test both p…
0c6b5b9 
…df and png outputs.
@istefano82
Refactor visualise confidence treshholding data vs acc subset to test…
9f149c1 
… both pdf and png outputs.
@istefano82
Refactor visualise confidence treshholding data vs acc subset per cla…
fe01d4d 
…ss to test both pdf and png outputs.
@istefano82
Refactor visualise confidence treshholding 2 threshholds 2d test both…
43cc1f5 
… pdf and png outputs.

Test fails due to "tempalte" typo in visualise implmentation.
istefano82 added 27 commits May 28, 2019 22:15
@istefano82
Refactor visualise compare classifiers from prob to test both pdf and…
0a7dbbf 
… png outputs.
@istefano82
Refactor visualise compare classifiers from pred to test both pdf and…
acb27ce 
… png outputs.
@istefano82
Refactor test docstrings to describe the input parameters.
0933c50
@istefano82
Refactor visualise compare classifiers from subset to test both pdf a…
9750c76 
…nd png outputs.
@istefano82
Refactor visualise compare classifiers changing k to test both pdf an…
908a550 
…d png outputs.
@istefano82
Refactor visualise compare classifiers multiclass multimetric to test…
c69e98a 
… both pdf and png outputs.
@istefano82
Refactor visualise compare classifiers predictions to test both pdf a…
3bbcbb9 
…nd png outputs.
@istefano82
Refactor visualise compare classifiers predictions distribution. to t…
87da962 
…est both pdf and png outputs.

Test fails.
@istefano82
Refactor visualise confidence treshholding to test both pdf and png o…
a1dbec9 
…utputs.
@istefano82
Refactor visualise confidence treshholding data vs acc to test both p…
27fa483 
…df and png outputs.
@istefano82
Refactor visualise confidence treshholding data vs acc subset to test…
336b7be 
… both pdf and png outputs.
@istefano82
Refactor visualise confidence treshholding data vs acc subset per cla…
563e934 
…ss to test both pdf and png outputs.
@istefano82
Refactor visualise confidence treshholding 2 threshholds 2d test both…
78f5c4e 
… pdf and png outputs.

Test fails due to "tempalte" typo in visualise implmentation.
@istefano82
Refactor visualise confidence treshholding 2 threshholds 3d test both…
e0abb66 
… pdf and png outputs.
@istefano82
Split lines longer than 80 chars in the treshhold tests.
84bc0ac
@istefano82
Test visualise binary treshhold vs metric saves both pdf and png outp…
cac0d22 
…uts.
@istefano82
Test visualise roc curves saves both pdf and png outputs.
187eeff
@istefano82
Test visualise roc curves from test statistics saves both pdf and png…
f65326c 
… outputs.
@istefano82
Test visualise calibration_1_vs_all saves both pdf and png outputs.
2544874
@istefano82
Test visualise calibration_multiclass saves both pdf and png outputs.
057bd56
@istefano82
Test visualise requency_vs_f1 saves both pdf and png outputs.
052167c 
Fix frequency_vs_f1 not passing filename to utils visualisation,
Causing plots to be displayed and block test execution.
@istefano82
Fix confidence_thresholding_2thresholds_2d typo causing the test to f…
d7971cd 
…ail.
@istefano82
Fix roc_curves_from_test_statistics test commands.
e9cbad8
@istefano82
Add field name to compare_classifiers_multiclass_multimetric commands…
47c5a72 
… to fix error.
@istefano82
Refator test_png command to extend the test_pdf command list.
e6feb63
@istefano82
Add author detials to module docstring.
9fa52f2
@istefano82
Merge branch 'viz_file_output_tests' of https://github.com/istefano82…
025e579 
…/ludwig_clone into viz_file_output_tests
@istefano82 istefano82 closed this May 28, 2019
@istefano82 istefano82 deleted the viz_file_output_tests branch May 28, 2019 19:22
@istefano82 istefano82 restored the viz_file_output_tests branch May 28, 2019 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@w4nderlust w4nderlust Awaiting requested review from w4nderlust

1 more reviewer

@msaisumanth msaisumanth msaisumanth left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@w4nderlust w4nderlust

@msaisumanth msaisumanth

Labels

feature New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@istefano82 @CLAassistant @w4nderlust @msaisumanth