This repository was archived by the owner on Apr 1, 2021. It is now read-only.

This repository was archived by the owner on Apr 1, 2021. It is now read-only.

Loss become nan with Adam and AdamW as base optimizers #37

Open

opened

on Jul 31, 2020

Loss becomes nan after training for ~20 steps - loss value stabily decreases and becomes nan with Adam or AdamW optimizers. In case of simple SGD usage it works well.

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests