tpuc-opt: model_validate() in model_transform.py fails with convoluted input shape #232
Description
Hi!
So it succesfully genrates .mlir file, but it fails to validate it.
Attaching all the files needed to replicate the issue, as well as the .mlir output.
(whisper distill-small.en consists of 2 models, one of them, the "encoder", validates with no errors. it has a simple input with shape [1, 80, 30000])
This is the container I use:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8f0f8d0bd129 docker.io/sophgo/tpuc_dev:v3.4 /bin/bash 24 hours ago Up 24 hours DuoTPU
Here's the log and command line
root@8f0f8d0bd129:/workspace# model_transform.py --model_name whisper-small-en-distilled-decoder --model_def whisper-onnx/distil-small.en-decoder.onnx --test_input inputs/decoder_input.npz --input_shapes [[1,1],[4,1,448,768],[4,1,448,768],[4,1,1500,768],[4,1,1500,768],[1]] --test_result inputs/decoder_output.npz --mlir decoder/whisper-small-en-distilled-decoder.mlir
SOPHGO Toolchain v1.3.228-g19ca95e9-20230921
2025/08/07 00:36:36 - INFO :
_____________________________________________________
| preprocess: |
| (x - mean) * scale |
'-------------------------------------------------------'
config Preprocess args :
resize_dims : same to net input dims
keep_aspect_ratio : False
keep_ratio_mode : letterbox
pad_value : 0
pad_type : center
--------------------------
mean : [0.0, 0.0, 0.0]
scale : [1.0, 1.0, 1.0]
--------------------------
pixel_format : bgr
channel_format : nchw
Input_shape assigned
ConstantFolding finished
skip_fuse_bn: False
Onnxsim opt finished
ConstantFolding finished
Save mlir file: decoder/whisper-small-en-distilled-decoder_origin.mlir
[Running]: tpuc-opt decoder/whisper-small-en-distilled-decoder_origin.mlir --shape-infer --canonicalize --extra-optimize -o decoder/whisper-small-en-distilled-decoder.mlir
tpuc-opt: /home/jenkins/workspace/tpu-mlir/lib/Dialect/Top/Interfaces/Reshape.cpp:86: void tpu_mlir::top::ReshapeOp::shape_inference(): Assertion `num_input == num_output' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0. Program arguments: tpuc-opt decoder/whisper-small-en-distilled-decoder_origin.mlir --init --shape-infer --canonicalize --extra-optimize --deinit --mlir-print-debuginfo -o decoder/whisper-small-en-distilled-decoder.mlir
#0 0x00005596696bdc37 (/workspace/tpu-mlir/bin/tpuc-opt+0x5e5c37)
#1 0x00005596696bb95e (/workspace/tpu-mlir/bin/tpuc-opt+0x5e395e)
#2 0x00005596696be5ba (/workspace/tpu-mlir/bin/tpuc-opt+0x5e65ba)
#3 0x00007f325bfa2520 (/lib/x86_64-linux-gnu/libc.so.6+0x42520)
#4 0x00007f325bff69fc pthread_kill (/lib/x86_64-linux-gnu/libc.so.6+0x969fc)
#5 0x00007f325bfa2476 gsignal (/lib/x86_64-linux-gnu/libc.so.6+0x42476)
#6 0x00007f325bf887f3 abort (/lib/x86_64-linux-gnu/libc.so.6+0x287f3)
#7 0x00007f325bf8871b (/lib/x86_64-linux-gnu/libc.so.6+0x2871b)
#8 0x00007f325bf99e96 (/lib/x86_64-linux-gnu/libc.so.6+0x39e96)
#9 0x000055966a2b9121 (/workspace/tpu-mlir/bin/tpuc-opt+0x11e1121)
#10 0x000055966a209493 (/workspace/tpu-mlir/bin/tpuc-opt+0x1131493)
#11 0x000055966a315a00 (/workspace/tpu-mlir/bin/tpuc-opt+0x123da00)
#12 0x000055966980e3de (/workspace/tpu-mlir/bin/tpuc-opt+0x7363de)
#13 0x000055966a314134 (/workspace/tpu-mlir/bin/tpuc-opt+0x123c134)
#14 0x000055966a3da1b4 (/workspace/tpu-mlir/bin/tpuc-opt+0x13021b4)
#15 0x000055966a3da7e1 (/workspace/tpu-mlir/bin/tpuc-opt+0x13027e1)
#16 0x000055966a3dcc88 (/workspace/tpu-mlir/bin/tpuc-opt+0x1304c88)
#17 0x00005596696af47b (/workspace/tpu-mlir/bin/tpuc-opt+0x5d747b)
#18 0x00005596696ae844 (/workspace/tpu-mlir/bin/tpuc-opt+0x5d6844)
#19 0x000055966a5e0a08 (/workspace/tpu-mlir/bin/tpuc-opt+0x1508a08)
#20 0x00005596696a8b4a (/workspace/tpu-mlir/bin/tpuc-opt+0x5d0b4a)
#21 0x00005596696a9014 (/workspace/tpu-mlir/bin/tpuc-opt+0x5d1014)
#22 0x00005596696a7d99 (/workspace/tpu-mlir/bin/tpuc-opt+0x5cfd99)
#23 0x00007f325bf89d90 (/lib/x86_64-linux-gnu/libc.so.6+0x29d90)
#24 0x00007f325bf89e40 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e40)
#25 0x00005596696a79f5 (/workspace/tpu-mlir/bin/tpuc-opt+0x5cf9f5)
Aborted (core dumped)
Traceback (most recent call last):
File "/workspace/tpu-mlir/python/tools/model_transform.py", line 273, in <module>
tool.model_transform(args.mlir, args.add_postprocess)
File "/workspace/tpu-mlir/python/tools/model_transform.py", line 51, in model_transform
mlir_opt_for_top(mlir_origin, self.mlir_file, add_postprocess)
File "/workspace/tpu-mlir/python/utils/mlir_shell.py", line 60, in mlir_opt_for_top
_os_system(cmd)
File "/workspace/tpu-mlir/python/utils/mlir_shell.py", line 50, in _os_system
raise RuntimeError("[!Error]: {}".format(cmd_str))
RuntimeError: [!Error]: tpuc-opt decoder/whisper-small-en-distilled-decoder_origin.mlir --shape-infer --canonicalize --extra-optimize -o decoder/whisper-small-en-distilled-decoder.mlir
root@8f0f8d0bd129:/workspace#
output of tpuc-opt, compiled from commit a0a53d8, with DEBUG enabled:
root@8157fa520e21:/workspace# tpuc-opt decoder/whisper-small-en-distilled-decoder_origin.mlir --shape-infer --canonicalize --extra-optimize -o decoder/whisper-small-en-distilled-decoder.mlir
%136 = "top.Expand"(%131) {shape = [1, 1, 768]} : (tensor<1x448x768xf32>) -> tensor<1x448x768xf32> loc("/Expand_output_0_Expand")
-> %137 = "top.Reshape"(%136) {flatten_start_dim = -1 : i64, shape = [1, 1, 768]} : (tensor<1x448x768xf32>) -> tensor<1x1x768xf32> loc("/Reshape_output_0_Reshape")
%146 = "top.ScatterND"(%115, %138, %137) {reduction = 0 : i32} : (tensor<1x448x768xf32>, tensor<1x1x768x3xf32>, tensor<1x1x768xf32>) -> tensor<*xf32> loc("/ScatterND_output_0_ScatterND")
ASSERT executed at/workspace/tpu-mlir-source/lib/Dialect/Top/Interfaces/Reshape.cpp:158
ASSERT INFO:num_input == num_output
Operation:
root@8157fa520e21:/workspace#