A common debugging method is to monitor the norm of some part of a ML model's weight and activation space.

Goal:

• Create a visualization component for norms of a set of vectors and/or weights. Allow the norms to be tracked over time, like we do with loss/accuracy.
• The code and implementation should be reusable and clean.

Code pointers:

• NPM project for animated transformers: https://github.com/PAIR-code/tiny-transformers/tree/main/animated-transformer
• the current line-chart component (in case we can make this nicer, e.g. allow clicking on points to see exact values, user-changable colors, etc). https://github.com/PAIR-code/tiny-transformers/tree/main/animated-transformer/src/app/d3-line-chart
• The Angular subdirectory of that for the animated transformer tool vis: https://github.com/PAIR-code/tiny-transformers/tree/main/animated-transformer/src/app/animated-transformer
• HTML file for current plots during training: https://github.com/PAIR-code/tiny-transformers/blob/main/animated-transformer/src/app/animated-transformer/model-task-trainer/model-task-trainer.component.html#L57