[ONNX] Refactor op level debugging #97494
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/97494
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 8867c75: This comment was automatically generated by Dr. CI and updates every 15 minutes. |
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
Fixes #97728 Fixes microsoft/onnxscript#393 Provide op_level_debug in exporter which creates randomnied torch.Tensor based on FakeTensorProp real shape as inputs of both torch ops and ONNX symbolic function. The test is different from [op_correctness_test.py](https://github.com/microsoft/onnx-script/blob/main/onnxscript/tests/function_libs/torch_aten/ops_correctness_test.py) as op_level_debug generating real tensors based on the fake tensors in the model. Limitation: 1. trace_only onnxscript function is not supported due to lack of param_schema. 2. The ops with dim/indices (INT64) is not supported that they need the information(shape) from other input args. [ghstack-poisoned]
Fixes #97728 Fixes microsoft/onnxscript#393 Provide op_level_debug in exporter which creates randomnied torch.Tensor based on FakeTensorProp real shape as inputs of both torch ops and ONNX symbolic function. The test is different from [op_correctness_test.py](https://github.com/microsoft/onnx-script/blob/main/onnxscript/tests/function_libs/torch_aten/ops_correctness_test.py) as op_level_debug generating real tensors based on the fake tensors in the model. Limitation: 1. trace_only onnxscript function is not supported due to lack of param_schema. 2. The ops with dim/indices (INT64) is not supported that they need the information(shape) from other input args. [ghstack-poisoned]
Fixes #97728 Fixes microsoft/onnxscript#393 Provide op_level_debug in exporter which creates randomnied torch.Tensor based on FakeTensorProp real shape as inputs of both torch ops and ONNX symbolic function. The test is different from [op_correctness_test.py](https://github.com/microsoft/onnx-script/blob/main/onnxscript/tests/function_libs/torch_aten/ops_correctness_test.py) as op_level_debug generating real tensors based on the fake tensors in the model. Limitation: 1. trace_only onnxscript function is not supported due to lack of param_schema. 2. The ops with dim/indices (INT64) is not supported that they need the information(shape) from other input args. [ghstack-poisoned]
justinchuby
reviewed
Mar 29, 2023
|
Had CI fail on: Not sure it's on the machine or the test... |
Fixes #97728 Fixes microsoft/onnxscript#393 Provide op_level_debug in exporter which creates randomnied torch.Tensor based on FakeTensorProp real shape as inputs of both torch ops and ONNX symbolic function. The PR leverages on Transformer class to create a new fx.Graph, but shares the same Module with the original one to save memory. The test is different from [op_correctness_test.py](https://github.com/microsoft/onnx-script/blob/main/onnxscript/tests/function_libs/torch_aten/ops_correctness_test.py) as op_level_debug generating real tensors based on the fake tensors in the model. Limitation: 1. Some of the trace_only function is not supported due to lack of param_schema which leads to arg/kwargs wronly split and ndarray wrapping. (WARNINGS in SARIF) 2. The ops with dim/indices (INT64) is not supported that they need the information(shape) from other input args. (WARNINGS in SARIF) 3. sym_size and built-in ops are not supported. 4. op_level_debug only labels results in SARIF. It doesn't stop exporter. 5. Introduce ONNX owning FakeTensorProp supports int/float/bool 6. parametrized op_level_debug and dynamic_shapes into FX tests [ghstack-poisoned]
|
Most likely because of ORT. This may be useful: microsoft/onnxscript#602 |
BowenBao
reviewed
Apr 7, 2023
BowenBao
reviewed
Apr 7, 2023
| DynamicSliceExportMod(), | ||
| (x,), | ||
| additional_test_inputs=[(y,)], | ||
| ) | ||
|
|
||
| def test_mutation(self): |
There was a problem hiding this comment.
Please do me a favor and skip this one for dynamic too #98622 🤣
There was a problem hiding this comment.
Hmmm The one reported is actually not dynamic. I will skip the whole test in this PR, and put skip_ORT_version on it in the next PR.
BowenBao
approved these changes
Apr 7, 2023
Fixes #97728 Fixes #98622 Fixes microsoft/onnxscript#393 Provide op_level_debug in exporter which creates randomnied torch.Tensor based on FakeTensorProp real shape as inputs of both torch ops and ONNX symbolic function. The PR leverages on Transformer class to create a new fx.Graph, but shares the same Module with the original one to save memory. The test is different from [op_correctness_test.py](https://github.com/microsoft/onnx-script/blob/main/onnxscript/tests/function_libs/torch_aten/ops_correctness_test.py) as op_level_debug generating real tensors based on the fake tensors in the model. Limitation: 1. Some of the trace_only function is not supported due to lack of param_schema which leads to arg/kwargs wronly split and ndarray wrapping. (WARNINGS in SARIF) 2. The ops with dim/indices (INT64) is not supported that they need the information(shape) from other input args. (WARNINGS in SARIF) 3. sym_size and built-in ops are not supported. 4. op_level_debug only labels results in SARIF. It doesn't stop exporter. 5. Introduce ONNX owning FakeTensorProp supports int/float/bool 6. parametrized op_level_debug and dynamic_shapes into FX tests [ghstack-poisoned]
Fixes #97728 Fixes #98622 Fixes microsoft/onnxscript#393 Provide op_level_debug in exporter which creates randomnied torch.Tensor based on FakeTensorProp real shape as inputs of both torch ops and ONNX symbolic function. The PR leverages on Transformer class to create a new fx.Graph, but shares the same Module with the original one to save memory. The test is different from [op_correctness_test.py](https://github.com/microsoft/onnx-script/blob/main/onnxscript/tests/function_libs/torch_aten/ops_correctness_test.py) as op_level_debug generating real tensors based on the fake tensors in the model. Limitation: 1. Some of the trace_only function is not supported due to lack of param_schema which leads to arg/kwargs wronly split and ndarray wrapping. (WARNINGS in SARIF) 2. The ops with dim/indices (INT64) is not supported that they need the information(shape) from other input args. (WARNINGS in SARIF) 3. sym_size and built-in ops are not supported. 4. op_level_debug only labels results in SARIF. It doesn't stop exporter. 5. Introduce ONNX owning FakeTensorProp supports int/float/bool 6. parametrized op_level_debug and dynamic_shapes into FX tests [ghstack-poisoned]
Stack from ghstack (oldest at bottom):
Fixes #97728
Fixes #98622
Fixes microsoft/onnxscript#393
Provide op_level_debug in exporter which creates randomnied torch.Tensor based on FakeTensorProp real shape as inputs of both torch ops and ONNX symbolic function. The PR leverages on Transformer class to create a new fx.Graph, but shares the same Module with the original one to save memory.
The test is different from op_correctness_test.py as op_level_debug generating real tensors based on the fake tensors in the model.
Limitation: