[DONT MERGE] Cold start experiments #162775

anijain2305 · 2025-09-12T00:01:31Z

Stack from ghstack (oldest at bottom):

-> [DONT MERGE] Cold start experiments #162775

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @amjames @Lucaskabela

[ghstack-poisoned]

pytorch-bot · 2025-09-12T00:01:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162775

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

❌ 68 New Failures

As of commit 39af65f with merge base d033d11 ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
inductor / inductor-cpu-test / test (cpu_inductor_torchbench, 1, 2, linux.8xlarge.amx) (gh)
lennard_jones
inductor / inductor-cpu-test / test (cpu_inductor_torchbench, 2, 2, linux.8xlarge.amx) (gh)
vision_maskrcnn
inductor / inductor-cpu-test / test (dynamic_cpu_inductor_huggingface, 1, 1, linux.8xlarge.amx) (gh)
YituTechConvBert
inductor / inductor-cpu-test / test (dynamic_cpu_inductor_timm, 1, 2, linux.8xlarge.amx) (gh)
mixer_b16_224
inductor / inductor-cpu-test / test (dynamic_cpu_inductor_timm, 2, 2, linux.8xlarge.amx) (gh)
xcit_large_24_p8_224
inductor / inductor-cpu-test / test (dynamic_cpu_inductor_torchbench, 1, 2, linux.8xlarge.amx) (gh)
lennard_jones
inductor / inductor-cpu-test / test (dynamic_cpu_inductor_torchbench, 2, 2, linux.8xlarge.amx) (gh)
vision_maskrcnn
inductor / inductor-cpu-test / test (inductor_torchbench_cpu_smoketest_perf, 1, 1, linux.24xl.spr-metal) (gh)
Process completed with exit code 255.
inductor / inductor-test / test (inductor_huggingface, 1, 1, linux.g5.4xlarge.nvidia.gpu) (gh)
AllenaiLongformerBase
inductor / inductor-test / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
mixer_b16_224
inductor / inductor-test / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
xcit_large_24_p8_224
inductor / inductor-test / test (inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
hf_T5_generate
inductor / inductor-test / test (inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
vision_maskrcnn
inductor / unit-test / inductor-cpu-test / test (inductor_amx, 1, 2, linux.8xlarge.amx) (gh)
Process completed with exit code 1.
inductor / unit-test / inductor-cpu-test / test (inductor_amx, 2, 2, linux.8xlarge.amx) (gh)
Process completed with exit code 1.
inductor / unit-test / inductor-cpu-test / test (inductor_avx2, 1, 2, linux.10xlarge.avx2) (gh)
Process completed with exit code 1.
inductor / unit-test / inductor-cpu-test / test (inductor_avx2, 2, 2, linux.10xlarge.avx2) (gh)
Process completed with exit code 1.
inductor / unit-test / inductor-halide-test / test (inductor-halide, 1, 1, linux.12xlarge) (gh)
inductor/test_halide.py::HalideCpuTests::test_large_block_sizes_cpu_halide
inductor / unit-test / inductor-test / test (inductor_cpp_wrapper, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_max_autotune.py::TestMaxAutotune::test_conv1x1_with_free_symbols
inductor / unit-test / inductor-test / test (inductor_cpp_wrapper, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_conv2d_autocast
inductor / unit-test / inductor-test / test (inductor_distributed, 1, 1, linux.g5.12xlarge.nvidia.gpu) (gh)
inductor/test_aot_inductor.py::AOTInductorTestABICompatibleGpu::test_replicate_on_devices_cuda
inductor / unit-test / inductor-test / test (inductor, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_accumulation_cuda
inductor / unit-test / inductor-test / test (inductor, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
test_ops.py::TestCommonCUDA::test_variant_consistency_eager_native_batch_norm_cuda_float32
inductor / unit-test / linux-jammy-cpu-py3.12-gcc11-inductor-triton-cpu / test (inductor-triton-cpu, 1, 1, linux.12xlarge) (gh)
inductor/test_triton_cpu_backend.py::CpuTritonTests::test_large_block_sizes_cpu
Lint / lintrunner-mypy / linux-job (gh)
>>> Lint for torch/_dynamo/output_graph.py:
Lint / lintrunner-noclang / linux-job (gh)
>>> Lint for torch/_dynamo/output_graph.py:
pull / cuda12.8-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh)
Process completed with exit code 1.
pull / linux-docs / build-docs-python-false (gh)
RuntimeError: no running event loop
pull / linux-jammy-py3.10-clang12 / test (crossref, 1, 2, lf.linux.2xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_export_numpy
pull / linux-jammy-py3.10-clang12 / test (crossref, 2, 2, lf.linux.2xlarge) (gh)
export/test_export.py::TestDynamismExpression::test_export_strict_narrow_unbacked_expr
pull / linux-jammy-py3.10-clang12 / test (default, 1, 5, lf.linux.4xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_export_numpy
pull / linux-jammy-py3.10-clang12 / test (default, 2, 5, lf.linux.4xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dont_dce_rand_dynamic_shapes
pull / linux-jammy-py3.10-clang12 / test (default, 3, 5, lf.linux.4xlarge) (gh)
inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_large_block_sizes_dynamic_shapes_cpu
pull / linux-jammy-py3.10-clang12 / test (default, 4, 5, lf.linux.4xlarge) (gh)
inductor/test_compiled_autograd.py::TestCompiledAutograd::test_accumulate_grad_polyfill_case_1_3
pull / linux-jammy-py3.10-clang12 / test (default, 5, 5, lf.linux.4xlarge) (gh)
inductor/test_compiled_autograd.py::TestCompiledAutograd::test_accumulate_grad_accuracy
pull / linux-jammy-py3.10-clang12 / test (dynamo_wrapped, 1, 3, lf.linux.2xlarge) (gh)
Process completed with exit code 1.
pull / linux-jammy-py3.10-clang12 / test (dynamo_wrapped, 2, 3, lf.linux.2xlarge) (gh)
Process completed with exit code 1.
pull / linux-jammy-py3.10-clang12 / test (dynamo_wrapped, 3, 3, lf.linux.2xlarge) (gh)
Process completed with exit code 1.
pull / linux-jammy-py3.10-clang12 / test (einops, 1, 1, lf.linux.2xlarge) (gh)
dynamo/test_einops.py::TestEinops::test_layers_version_0_6_1
pull / linux-jammy-py3.10-clang12-onnx / test (default, 2, 2, lf.linux.2xlarge) (gh)
onnx/exporter/test_small_models_e2e.py::DynamoExporterTest::test_onnx_export_control_flow
pull / linux-jammy-py3.10-clang18-asan / test (default, 1, 7, lf.linux.4xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_export_numpy
pull / linux-jammy-py3.10-clang18-asan / test (default, 2, 7, lf.linux.4xlarge) (gh)
inductor/test_torchinductor.py::SweepInputsCpuTest::test_cpu_broadcast1_broadcast3
pull / linux-jammy-py3.10-clang18-asan / test (default, 3, 7, lf.linux.4xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesCtxManagerTests::test_autocast_cpu_graph_break_inner_fn_dynamic_shapes
pull / linux-jammy-py3.10-clang18-asan / test (default, 4, 7, lf.linux.4xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dont_dce_rand_dynamic_shapes
pull / linux-jammy-py3.10-clang18-asan / test (default, 5, 7, lf.linux.4xlarge) (gh)
inductor/test_compiled_autograd.py::TestCompiledAutograd::test_accumulate_grad_accuracy
pull / linux-jammy-py3.10-clang18-asan / test (default, 6, 7, lf.linux.4xlarge) (gh)
inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_adaptive_max_pool2d3_dynamic_shapes_cpu
pull / linux-jammy-py3.10-clang18-asan / test (default, 7, 7, lf.linux.4xlarge) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_ModularIndexing_range_issue_103133
pull / linux-jammy-py3.10-gcc11 / test (default, 1, 5, lf.linux.2xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_export_numpy
pull / linux-jammy-py3.10-gcc11 / test (default, 2, 5, lf.linux.2xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dont_dce_rand_dynamic_shapes
pull / linux-jammy-py3.10-gcc11 / test (default, 3, 5, lf.linux.2xlarge) (gh)
dynamo/test_misc.py::MiscTests::test_312_binary_slice_with_graph_break1
pull / linux-jammy-py3.10-gcc11 / test (default, 4, 5, lf.linux.2xlarge) (gh)
inductor/test_compiled_autograd.py::TestCompiledAutograd::test_accumulate_grad_polyfill_case_1_3
pull / linux-jammy-py3.10-gcc11 / test (default, 5, 5, lf.linux.2xlarge) (gh)
inductor/test_compiled_autograd.py::TestCompiledAutograd::test_accumulate_grad_accuracy
pull / linux-jammy-py3.10-gcc11 / test (distributed, 1, 2, lf.linux.2xlarge) (gh)
distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_call_method_forward
pull / linux-jammy-py3.10-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge) (gh)
distributed/pipelining/test_transformer.py::TransformerTestsCPU::test_ir_cpu
pull / linux-jammy-py3.10-gcc11 / test (docs_test, 1, 1, lf.linux.2xlarge) (gh)
RuntimeError: no running event loop
pull / linux-jammy-py3.10-gcc11 / test (numpy_2_x, 1, 1, lf.linux.2xlarge) (gh)
dynamo/test_functions.py::FunctionTests::test_T
pull / linux-jammy-py3.13-clang12 / test (crossref, 1, 2, lf.linux.2xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_export_numpy
pull / linux-jammy-py3.13-clang12 / test (crossref, 2, 2, lf.linux.2xlarge) (gh)
export/test_export.py::TestDynamismExpression::test_export_strict_narrow_unbacked_expr
pull / linux-jammy-py3.13-clang12 / test (default, 1, 5, lf.linux.4xlarge) (gh)
test_fake_tensor.py::FakeTensorTest::test_export_numpy
pull / linux-jammy-py3.13-clang12 / test (default, 2, 5, lf.linux.4xlarge) (gh)
dynamo/test_dynamic_shapes.py::DynamicShapesReproTests::test_dont_dce_rand_dynamic_shapes
pull / linux-jammy-py3.13-clang12 / test (default, 3, 5, lf.linux.4xlarge) (gh)
inductor/test_torchinductor_codegen_dynamic_shapes.py::DynamicShapesCodegenCpuTests::test_large_block_sizes_dynamic_shapes_cpu
pull / linux-jammy-py3.13-clang12 / test (default, 4, 5, lf.linux.4xlarge) (gh)
inductor/test_torchinductor_dynamic_shapes.py::DynamicShapesCpuTests::test_large_block_sizes_dynamic_shapes_cpu
pull / linux-jammy-py3.13-clang12 / test (default, 5, 5, lf.linux.4xlarge) (gh)
inductor/test_compiled_autograd.py::TestCompiledAutograd::test_accumulate_grad_accuracy
pull / linux-jammy-py3.13-clang12 / test (dynamo_wrapped, 1, 3, lf.linux.2xlarge) (gh)
test_torch.py::TestTorch::test_Size
pull / linux-jammy-py3.13-clang12 / test (dynamo_wrapped, 2, 3, lf.linux.2xlarge) (gh)
test_linalg.py::TestLinalgCPU::test_svd_cpu_float32
pull / linux-jammy-py3.13-clang12 / test (dynamo_wrapped, 3, 3, lf.linux.2xlarge) (gh)
test_linalg.py::TestLinalgCPU::test_svd_cpu_float64
pull / linux-jammy-py3.13-clang12 / test (einops, 1, 1, lf.linux.2xlarge) (gh)
dynamo/test_einops.py::TestEinops::test_layers_version_0_6_1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 4de54a4 Pull Request resolved: #162775

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames Lucaskabela [ghstack-poisoned]

ghstack-source-id: d44e510 Pull Request resolved: #162775

github-actions · 2025-09-12T00:02:32Z

This PR needs a `release notes:` label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[DONT MERGE] Cold start experiments

b8fbee8

[ghstack-poisoned]

anijain2305 mentioned this pull request Sep 12, 2025

[profile trace] Fill gaps in compile time profile traces #162627

Open

pytorch-bot bot added ciflow/inductor module: dynamo labels Sep 12, 2025

anijain2305 added a commit that referenced this pull request Sep 12, 2025

[DONT MERGE] Cold start experiments

0d10fb6

ghstack-source-id: 4de54a4 Pull Request resolved: #162775

Update on "[DONT MERGE] Cold start experiments"

39af65f

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames Lucaskabela [ghstack-poisoned]

anijain2305 added a commit that referenced this pull request Sep 12, 2025

[DONT MERGE] Cold start experiments

79c7dee

ghstack-source-id: d44e510 Pull Request resolved: #162775

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[DONT MERGE] Cold start experiments #162775

[DONT MERGE] Cold start experiments #162775

anijain2305 commented Sep 12, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Sep 12, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Sep 12, 2025

Uh oh!

Uh oh!

[DONT MERGE] Cold start experiments #162775

Are you sure you want to change the base?

[DONT MERGE] Cold start experiments #162775

Conversation

anijain2305 commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162775

❌ 68 New Failures

Uh oh!

github-actions bot commented Sep 12, 2025

This PR needs a release notes: label

Uh oh!

Uh oh!

anijain2305 commented Sep 12, 2025 •

edited

Loading

pytorch-bot bot commented Sep 12, 2025 •

edited

Loading

This PR needs a `release notes:` label