Looks good. Thanks Evan!
Also, in order to test if every layer is implemented for gradient accumulation I made the following chagne in my branch: tnarihi@7d45526
Please consider this as well.

@jeffdonahue could you decide what to do with tnarihi/caffe@7d45526 too since you and Takuya talked about the gradient checker? Thanks.

LGTM, thanks @tnarihi i and @shelhamer. I'm also not sure of the best way to go with changing the gradient checker -- always checking accumulation is safer, but anyone who's developed a layer of their own with parameters may be confused as to why their implementation is suddenly failing the gradient checker. I guess ideally there would be a first pass that does the current gradient check and, only if that passes, also checks accumulation afterwards and fails with an error message pointing to failed accumulation if applicable. Seems like the code would get kind of ugly though...