more expressive CUDA error macro. instead of 'cuda error = 2', now we ge... #193
Conversation
… get 'cuda error = 2 (out of memory)'
include/caffe/common.hpp
Outdated
There was a problem hiding this comment.
Probably using LOG(ERROR) or LOG(FATAL) will be better in this case
|
@sguada Good idea, Sergio. I changed the code so that it writes the error to LOG(FATAL), and I verified that it handles errors appropriately. Does this look reasonable to you? |
include/caffe/common.hpp
Outdated
There was a problem hiding this comment.
Why printf? You can just concatenate by << as the other LOG in Caffe do.
|
Hey Forrest, thanks for polishing away this rough edge! Having more informative errors should help quiet down recurring questions about memory and hardware too. Let's merge after the slight style changes I commented on. |
|
@shelhamer @kloudkl All right, I polished away that rough edge. We're now writing CUDA errors to glog with |
|
Thanks Forrest! |
There was a problem hiding this comment.
Use CHECK_EQ(res, cudaSuccess) << your_message_here instead of an if statement maybe. It will deal with log and exit automatically.
When CUDA has an error, it throws a number like "2" or "7" or something like that. Previously, Caffe was just printing out the error number, and the user needed to go look up the meaning of the error number. With this addition to the code, CUDA errors are now printed in plaintext (e.g. "out of memory.")
I verified that errors are still caught properly, and that the tests still pass. Still gives the correct ImageNet-val results.