[ONNX] Add new ops and enable the MNIST test #99284

justinchuby · 2023-04-17T03:40:41Z

`🤖 Generated by Copilot at d490881`

Summary

📦🚀🗺️

This pull request adds support for converting more PyTorch FX operators to ONNX using the onnxscript package. It updates the package installation script and the operator mappings in function_dispatcher.py. It also enables a test case for the max_pool2d operator.

We unleash the fury of the ONNX script
We break the chains of the skipped test
We map the functions of the ATen torch
We forge the metal of the ONNX forge

Walkthrough

Fix and update the installation of the onnxscript package (link)
Add more mappings from PyTorch ATen operators to ONNX operators in the _ATENLIB_FUNCTIONS dictionary in the function_dispatcher.py module (link, link, link, link, link)
Enable the test_max_pool2d method in the TestFxToOnnx class in the test_fx_to_onnx.py file, since the max_pool2d operator is now supported by the onnxscript package (link)

pytorch-bot · 2023-04-17T03:40:43Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/99284

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 Failures

As of commit d68419f:

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

torch/onnx/_internal/fx/function_dispatcher.py

justinchuby · 2023-04-17T17:30:58Z

Requires microsoft/onnxscript#645

test/onnx/test_fx_to_onnx.py

Previously it was passed in as an empty list and graph_builder fails when the list is empty. Discovered in pytorch/pytorch#99284 - Added maxpool_2d/3d custom OpInfo tests. Skipping 3d for now due to shape mismatch. - Pin torch-nightly version to a requirements file

torch/onnx/_internal/fx/function_dispatcher.py

BowenBao

LGTM, mergeable after consensus with @titaiwangms on test_mnist I guess?

justinchuby · 2023-04-22T00:01:18Z

@pytorchbot merge

pytorchmergebot · 2023-04-22T00:03:16Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

justinchuby · 2023-04-25T17:00:37Z

@pytorchbot merge

pytorchmergebot · 2023-04-25T17:02:30Z

Merge failed

Reason: Approval needed from one of the following:
khabinov, c-odrin, sgrigory, xunnanxu, xing-liu, ...

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

justinchuby · 2023-04-25T18:11:32Z

@pytorchbot -h

pytorch-bot · 2023-04-25T18:11:33Z

PyTorchBot Help

usage: @pytorchbot [-h] {merge,revert,rebase,label,drci} ...

In order to invoke the bot on your PR, include a line that starts with
@pytorchbot anywhere in a comment. That line will form the command; no
multi-line commands are allowed. 

Example:
    Some extra context, blah blah, wow this PR looks awesome

    @pytorchbot merge

optional arguments:
  -h, --help            Show this help message and exit.

command:
  {merge,revert,rebase,label,drci}
    merge               Merge a PR
    revert              Revert a PR
    rebase              Rebase a PR
    label               Add label to a PR
    drci                Update Dr. CI

Merge

usage: @pytorchbot merge [-f MESSAGE | -ic] [-r [{viable/strict,main}]]

Merge an accepted PR, subject to the rules in .github/merge_rules.json.
By default, this will wait for all required checks (lint, pull) to succeed before merging.

optional arguments:
  -f MESSAGE, --force MESSAGE
                        Merge without checking anything. This requires a reason for auditting purpose, for example:
                        @pytorchbot merge -f 'Minor update to fix lint. Expecting all PR tests to pass'
  -ic, --ignore-current
                        Merge while ignore the currently failing jobs.  If there are no pending checks, use -f/--force since this will fail.
  -r [{viable/strict,main}], --rebase [{viable/strict,main}]
                        Rebase the PR to re run checks before merging.  Accepts viable/strict or main as branch options and will default to viable/strict if not specified.

Revert

usage: @pytorchbot revert -m MESSAGE -c
                          {nosignal,ignoredsignal,landrace,weird,ghfirst}

Revert a merged PR. This requires that you are a Meta employee.

Example:
  @pytorchbot revert -m="This is breaking tests on trunk. hud.pytorch.org/" -c=nosignal

optional arguments:
  -m MESSAGE, --message MESSAGE
                        The reason you are reverting, will be put in the commit message. Must be longer than 3 words.
  -c {nosignal,ignoredsignal,landrace,weird,ghfirst}, --classification {nosignal,ignoredsignal,landrace,weird,ghfirst}
                        A machine-friendly classification of the revert reason.

Rebase

usage: @pytorchbot rebase [-s | -b BRANCH]

Rebase a PR. Rebasing defaults to the stable viable/strict branch of pytorch.
You must have write permissions to the repo to rebase a PR.

optional arguments:
  -s, --stable          [DEPRECATED] Rebase onto viable/strict
  -b BRANCH, --branch BRANCH
                        Branch you would like to rebase to

Label

usage: @pytorchbot label labels [labels ...]

Adds label to a PR

positional arguments:
  labels  Labels to add to given Pull Request

Dr CI

usage: @pytorchbot drci 

Update Dr. CI. Updates the Dr. CI comment on the PR in case it's gotten out of sync with actual CI results.

justinchuby · 2023-04-25T18:14:05Z

@pytorchbot merge -ic

pytorchmergebot · 2023-04-25T18:16:08Z

Merge started

Your change will be merged while ignoring the following 4 checks: linux-binary-manywheel / manywheel-py3_8-cuda11_7-with-pypi-cudnn-build / build, linux-binary-manywheel / manywheel-py3_8-cuda12_1-build / build, linux-binary-manywheel / manywheel-py3_8-cuda11_7-with-pypi-cudnn-test, linux-binary-manywheel / manywheel-py3_8-cuda12_1-test

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

justinchuby · 2023-04-25T19:41:27Z

@pytorchbot merge -f "onnx checks succeeded"

pytorchmergebot · 2023-04-25T19:45:13Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

justinchuby requested review from jeffdaily, BowenBao and abock as code owners April 17, 2023 03:40

pytorch-bot bot added the release notes: onnx torch.onnx related changes that should show up in the release notes label Apr 17, 2023

justinchuby added module: onnx Related to torch.onnx and removed release notes: onnx torch.onnx related changes that should show up in the release notes labels Apr 17, 2023

justinchuby commented Apr 17, 2023

View reviewed changes

torch/onnx/_internal/fx/function_dispatcher.py Outdated Show resolved Hide resolved

pytorchbot added the open source label Apr 17, 2023

This comment was marked as resolved.

Sign in to view

justinchuby force-pushed the justinchu/maxpool branch from da76929 to d490881 Compare April 17, 2023 16:49

justinchuby mentioned this pull request Apr 17, 2023

Fix empty list handling in maxpool | fix(atenlib) microsoft/onnxscript#645

Merged

justinchuby marked this pull request as draft April 17, 2023 17:30

pytorch-bot bot added the release notes: onnx torch.onnx related changes that should show up in the release notes label Apr 17, 2023

titaiwangms reviewed Apr 18, 2023

View reviewed changes

test/onnx/test_fx_to_onnx.py Show resolved Hide resolved

justinchuby and others added 6 commits April 20, 2023 22:35

Add ops

96bae34

Bump onnxscript

485bbb8

Update install_onnx.sh

cafdef2

Update torch/onnx/_internal/fx/function_dispatcher.py

1a9de42

Update install_onnx.sh

ad9bbcb

Update function_dispatcher.py

378b278

justinchuby force-pushed the justinchu/maxpool branch from b2d2cc3 to 378b278 Compare April 20, 2023 22:36

justinchuby marked this pull request as ready for review April 20, 2023 22:37

justinchuby commented Apr 20, 2023

View reviewed changes

torch/onnx/_internal/fx/function_dispatcher.py Outdated Show resolved Hide resolved

justinchuby commented Apr 20, 2023

View reviewed changes

torch/onnx/_internal/fx/function_dispatcher.py Outdated Show resolved Hide resolved

justinchuby added 2 commits April 20, 2023 15:42

Apply suggestions from code review

3115dd3

Update test_fx_to_onnx.py

dff55b2

BowenBao approved these changes Apr 21, 2023

View reviewed changes

justinchuby added 2 commits April 21, 2023 23:06

Merge branch 'main' into justinchu/maxpool

ef436f6

Enable test

f55d644

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Apr 22, 2023

pytorchmergebot added the merging label Apr 22, 2023

Update test

f4ba918

BowenBao mentioned this pull request Apr 22, 2023

[ONNX] Cover 'undiscoverable' ops 'torch.ops.aten' #99682

Closed

justinchuby mentioned this pull request Apr 24, 2023

[ONNX] Bump onnx-script version with imported module renaming #99926

Closed

justinchuby added 2 commits April 25, 2023 16:56

Merge branch 'main' into justinchu/maxpool

935b86a

Fix test

ed8b410

pytorchmergebot removed the merging label Apr 25, 2023

justinchuby added 2 commits April 25, 2023 17:04

Fix ops list

2d54345

Fix submodule

d68419f

pytorchmergebot added the merging label Apr 25, 2023

pytorchmergebot added Merged and removed merging labels Apr 25, 2023

pytorchmergebot closed this in 865d30a Apr 25, 2023

justinchuby deleted the justinchu/maxpool branch April 25, 2023 23:12

[ONNX] Add new ops and enable the MNIST test #99284

[ONNX] Add new ops and enable the MNIST test #99284

Uh oh!

Conversation

justinchuby commented Apr 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 Generated by Copilot at d490881

Summary

Walkthrough

Uh oh!

pytorch-bot bot commented Apr 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/99284

❌ 4 Failures

Uh oh!

Uh oh!

This comment was marked as resolved.

justinchuby commented Apr 17, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BowenBao left a comment

Choose a reason for hiding this comment

Uh oh!

justinchuby commented Apr 22, 2023

Uh oh!

pytorchmergebot commented Apr 22, 2023

Merge started

Uh oh!

justinchuby commented Apr 25, 2023

Uh oh!

pytorchmergebot commented Apr 25, 2023

Merge failed

Uh oh!

justinchuby commented Apr 25, 2023

Uh oh!

pytorch-bot bot commented Apr 25, 2023

PyTorchBot Help

Merge

Revert

Rebase

Label

Dr CI

Uh oh!

justinchuby commented Apr 25, 2023

Uh oh!

pytorchmergebot commented Apr 25, 2023

Merge started

Uh oh!

justinchuby commented Apr 25, 2023

Uh oh!

pytorchmergebot commented Apr 25, 2023

Merge started

Uh oh!

Uh oh!

justinchuby commented Apr 17, 2023 •

edited

Loading

`🤖 Generated by Copilot at d490881`

pytorch-bot bot commented Apr 17, 2023 •

edited

Loading