v0.10.0 (21/11/2023)

New Features & Enhancements

• Support for installing mmengine without opencv by @fanqiNO1 in #1429
• Support exclude_frozen_parameters for DeepSpeedStrategy's resume by @LZHgrla in #1424

Bug Fixes

• Fix bugs in colo optimwrapper by @HIT-cwh in #1426
• Fix scale_lr in SingleDeviceStrategy by @fanqiNO1 in #1428
• Fix CI for torch2.1.0 by @fanqiNO1 in #1418

Full Changelog: v0.9.1...v0.10.0

v0.9.1 (03/11/2023)

New Features & Enhancements

• Add exclude_frozen_parameters for DeepSpeedStrategy by @LZHgrla in #1415
• Enhance inputs_to_half in DeepSpeedStrategy by @fanqiNO1 in #1400
• Support slurm distributed training for mlu devices by @POI-WX in #1396
• Add torch 2.1.0 checking in CI by @YiyaoYang1 in #1389

Bug Fixes

• Fix new config in visualizer by @HAOCHENYE in #1390
• Fix func params using without init in OneCycleLR (#1401) by @whlook in #1403
• Fix a bug when module is missing in low version of bitsandbytes by @Ben-Louis in #1388
• Fix ConcatDataset raising error when metainfo is np.array by @jonbakerfish in #1407

Docs

• Rename master to main by @zhouzaida in #1397

New Contributors

• @POI-WX made their first contribution in #1396
• @YiyaoYang1 made their first contribution in #1389
• @whlook made their first contribution in #1403
• @Ben-Louis made their first contribution in #1388
• @jonbakerfish made their first contribution in #1407

Full Changelog: v0.9.0...v0.9.1

v0.9.0 (10/10/2023)

Highlights

• Support training with ColossalAI. Refer to the Training Large Models for more detailed usages.
• Support gradient checkpointing. Refer to the Save Memory on GPU for more details.
• Supports multiple visualization backends, including NeptuneVisBackend, DVCLiveVisBackend and AimVisBackend. Refer to Visualization Backends for more details.

New Features & Enhancements

• Add a text translation example by @Desjajja in #1283
• Add NeptuneVisBackend by @wangerlie in #1311
• Add ColossalAI strategy by @HAOCHENYE in #1299
• Add collect_results support for Ascend NPU by @xuuyangg in #1309
• Unify the parameter style of DeepSpeedStrategy by @LZHgrla in #1320
• Add progressbar rich by @Dominic23331 in #1157
• Support using other file handlers by @KevinNuNu in #1188
• Refine error message by @zhouzaida in #1338
• Implement gradient checkpointing by @zeyuanyin in #1319
• Add DVCLiveVisBackend by @RangeKing in #1336
• Add AimVisBackend by @RangeKing in #1347
• Support bitsandbytes by @okotaku in #1357
• Support Adafactor Optimizer by @okotaku in #1361
• Add unit tests for autocast with Ascend device by @6Vvv in #1363
• Support metainfo of dataset can be a generic dict-like Mapping by @hiyyg in #1378
• Support for installing minimal runtime dependencies by @HAOCHENYE in #1362
• Support setting the number of iterations in Runner for each epoch by @ShuRaymond in #1292
• Support using gradient checkpointing in FSDP by @HAOCHENYE in #1382

Docs

• Add README for examples by @zhouzaida in #1295
• Add a new ecosystem in README by @zhouzaida in #1296
• Fix typo by @zhouzaida in #1298
• Add an image for Neptune by @zhouzaida in #1312
• Fix docs of ColossalAI by @HAOCHENYE in #1315
• Update QRCode by @crazysteeaam in #1328
• Add activation checkpointing usage by @zhouzaida in #1341
• Fix typo by @zhouzaida in #1348
• Update the usage of bitsandbytes in Chinese documents by @zhouzaida in #1359
• Fix doc typo our_dir in LoggerHook by @wangg12 in #1373
• Add the contributing doc in pr template by @zhouzaida in #1380
• Update the version info by @fanqiNO1 in #1383
• Fix typo by @fanqiNO1 in #1385

Bug Fixes

• Ignore examples in CI by @zhouzaida in #1297
• Fix multi-card issue in PyTorch v2.1 on Ascend by @LRJKD in #1321
• Fix get optimizer_cls by @HAOCHENYE in #1324
• Fix ndarray metainfo check in ConcatDataset by @NrealLzx in #1333
• Adapt to PyTorch v2.1 on Ascend by @LRJKD in #1332
• Fix the type check of tasks in progress bar by @zhouzaida in #1340
• The keyword mode appears nested multiple times in the log by @huaibovip in #1305
• Fix pydantic version to fix mlflow unit tests by @zhouzaida in #1351
• Fix get class attribute from a string by @HAOCHENYE in #1345
• Delete yapf verify by @okotaku in #1365
• Ensure from_cfg of Runner have the same defaults values as its init by @YinAoXiong in #1368
• Fix docs building error caused by deepspeed by @HAOCHENYE in #1379

New Contributors

• @Desjajja made their first contribution in #1283
• @wangerlie made their first contribution in #1311
• @xuuyangg made their first contribution in #1309
• @LRJKD made their first contribution in #1321
• @crazysteeaam made their first contribution in #1328
• @Dominic23331 made their first contribution in #1157
• @NrealLzx made their first contribution in #1333
• @zeyuanyin made their first contribution in #1319
• @huaibovip made their first contribution in #1305
• @6Vvv made their first contribution in #1363
• @yinaoxiong made their first contribution in #1368
• @wangg12 made their first contribution in #1373
• @hiyyg made their first contribution in #1378
• @ShuRaymond made their first contribution in #1292
• @fanqiNO1 made their first contribution in #1383

Full Changelog: v0.8.4...v0.9.0

v0.8.5 (25/09/2023)

Bug fixes

• Optional removal of the verify parameter in yapf, ensuring compatibility across different yapf versions by @okotaku in #1365

v0.8.4 (03/08/2023)

New Features & Enhancements

• Support callable collate_fn for FlexibleRunner by @LZHgrla in #1284

Bug fixes

• Skip adding vis_backends when save_dir is not set by @HAOCHENYE in #1289
• Fix dumping pure python style config in colab by @HAOCHENYE in #1290

Docs

• Find unused parameters by @zhouzaida in #1288

New Contributors

• @LZHgrla made their first contribution in #1284

Full Changelog: v0.8.3...v0.8.4

v0.8.3 (31/07/2023)

Highlights

• Support enabling efficient_conv_bn_eval for efficient convolution and batch normalization. See save memory on gpu for more details
• Add Llama2 finetune example
• Support multi-node distributed training with MLU backend

New Features & Enhancements

• Enable efficient_conv_bn_eval for memory saving convolution and batch normalization by @youkaichao in #1202, #1251 and #1259
• Add Llama2 example by @HAOCHENYE in #1264
• Compare the difference of two configs by @gachiemchiep in #1260
• Enable explicit error for deepspeed not installed by @Li-Qingyun in #1240
• Support skipping initialization in BaseModule by @HAOCHENYE in #1263
• Add parameter save_begin to control when to save checkpoints by @KerwinKai in #1271
• Support multi-node distributed training with MLU backend by @josh6688 in #1266
• Enhance error message thrown by Config, build function and ConfigDict.items by @HAOCHENYE in #1272, #1270 and #1088
• Add the loop_stage runtime information in message_hub by @zhouzaida in #1277
• Fix Visualizer that built vis_backends will not be used when save_dir is None by @Xinyu302 in #1275

Bug fixes

• Fix scalar check in RuntimeInfoHook by @i-aki-y in #1250
• Move data preprocessor to target device in FSDPStrategy by @HAOCHENYE in #1261

Docs

• Add ecosystem in README by @zhouzaida in #1247
• Add short explanation about registry scope by @mmeendez8 in #1114
• Add the data flow of Runner in README by @zhouzaida in #1257
• Introduce how to customize distributed training settings @zhouzaida in #1279

New Contributors

• @youkaichao made their first contribution in #1202
• @mmeendez8 made their first contribution in #1114
• @Xinyu302 made their first contribution in #1275

Full Changelog: v0.8.2...v0.8.3

Bug fixes

• Fix pickling the Python style config by @HAOCHENYE in #1241
• Fix the logic of setting lazy_import by @Li-Qingyun in #1239

New Contributors

• @Li-Qingyun made their first contribution in #1239

Full Changelog: v0.8.1...v0.8.2

New Features & Enhancements

• Accelerate Config.dump and support converting Lazyxxx to string in ConfigDict.to_dictby @HAOCHENYE in #1232

Bug fixes

• FSDP should call _get_ignored_modules by @HAOCHENYE in #1235

Docs

• Add a document to introduce how to train a large model by @zhouzaida in #1228

Full Changelog: v0.8.0...v0.8.1

v0.8.0 (07/03/2023)

Highlights

• Support training with FSDP and DeepSpeed. Refer to the example for more detailed usages.

• Introduce the pure Python style configuration file:

  • Support navigating to base configuration file in IDE
  • Support navigating to base variable in IDE
  • Support navigating to source code of class in IDE
  • Support inheriting two configuration files containing the same field
  • Load the configuration file without other third-party requirements

  Refer to the tutorial for more detailed usages.

  

New Features & Enhancements

• Support training with FSDP by @HAOCHENYE in #1213
• Add FlexibleRunner and Strategies, and support training with DeepSpeed by @zhouzaida in #1183
• Support pure Python style configuration file by @HAOCHENYE in #1071
• Learning rate in log can show the base learning rate of optimizer by @AkideLiu in #1019
• Refine the error message when auto_scale_lr is not set correctly by @alexander-soare in #1181
• WandbVisBackend supports updating config by @zgzhengSEU in #977

Bug fixes

• CheckpointHook should check whether file exists before removing it by @zhouzaida in #1198
• Fix undefined variable error in Runner by @HAOCHENYE in #1219

Docs

• Add a document to introduce how to debug with vscode by @zhouzaida in #1212
• Update English introduction by @evdcush in #1189
• Fix parameter typing error in document by @syo093c in #1201
• Fix gpu collection during evaluation by @edkair in #1208
• Fix a comment in runner tutorial by @joihn in #1210

New Contributors

• @alexander-soare made their first contribution in #1181
• @zgzhengSEU made their first contribution in #977
• @AkideLiu made their first contribution in #1019
• @syo093c made their first contribution in #1201
• @edkair made their first contribution in #1208
• @joihn made their first contribution in #1210

Full Changelog: v0.7.4...v0.8.0

v0.7.4 (06/03/2023)

Highlights

• Support using ClearML to record experiment data
• Add Sophia optimizers

New Features & Enhancements

• Add visualize backend for clearml by @gachiemchiep in #1091
• Support Sophia optimizers by @zhouzaida in #1170
• Refactor unittest syncbuffer by @HAOCHENYE in #813
• Allow ann_file, data_root is None for BaseDataset by @HAOCHENYE in #850
• Enable full precision training on Ascend NPU by @Ginray in #1109
• Creating a text classification example by @TankNee in #1122
• Add option to log selected config only by @KickCellarDoor in #1159
• Add an option to control whether to show progress bar in BaseInference by @W-ZN in #1135
• Support dipu device by @CokeDong in #1127
• Let unit tests not affect each other by @zhouzaida in #1169
• Add support for full wandb's define_metric arguments by @i-aki-y in #1099

Bug fixes

• Fix the incorrect device of inputs in get_model_complexity_info by @CescMessi in #1130
• Correctly saves _metadata of state_dict when saving checkpoints by @Bomsw in #1131
• Correctly record random seed in log by @Shiyang980713 in #1152
• Close MLflowVisBackend only if active by @zimonitrome in #1151
• Fix ProfileHook cannot profile ddp-training by @HAOCHENYE in #1140
• Handle the case for Multi-Instance GPUs when using cuda_visible_devices by @adrianjoshua-strutt in #1164
• Fix attribute error when parsing CUDA_VISIBLE_DEVICES in logger @Xiangxu-0103 in #1172

Docs

• Translate infer.md by @Hongru-Xiao in #1121
• Fix a missing comma in tutorials/runner.md by @gy-7 in #1146
• Fix typo in comment by @YQisme in #1154
• Translate data_element.md by @xin-li-67 in #1067
• Add the usage of clearml by @zhouzaida in #1180

New Contributors

• @CescMessi made their first contribution in #1130
• @Bomsw made their first contribution in #1131
• @Hongru-Xiao made their first contribution in #1121
• @TankNee made their first contribution in #1122
• @W-ZN made their first contribution in #1135
• @gy-7 made their first contribution in #1146
• @YQisme made their first contribution in #1154
• @Shiyang980713 made their first contribution in #1152
• @KickCellarDoor made their first contribution in #1159
• @CokeDong made their first contribution in #1127
• @zimonitrome made their first contribution in #1151
• @adrianjoshua-strutt made their first contribution in #1164
• @gachiemchiep made their first contribution in #1091
• @i-aki-y made their first contribution in #1099

Full Changelog: v0.7.3...v0.7.4