assertion failed: !absl::exchange(g_registered, true) #30640
Description
What version of gRPC and what language are you using?
Here are the installed python packages, no idea which version of grpc runs underneath.
grpcio-1.49.0rc1
grpcio-tools-1.49.0rc1
What operating system (Linux, Windows,...) and version?
guillaume@latitude-5411:~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
What runtime / compiler are you using (e.g. python version or version of gcc)
Python 3.9.7
What did you do?
guillaume@latitude-5411:~$ cd /tmp/
guillaume@latitude-5411:/tmp$ git clone https://github.com/inmanta/terraform.git > /dev/null
guillaume@latitude-5411:/tmp$ cd terraform/
guillaume@latitude-5411:/tmp/terraform$ mktmpenv -p 3.9 > /dev/null
(tmp-730f4738cbc9bf7) guillaume@latitude-5411:/tmp/terraform$ make install > /dev/null
(tmp-730f4738cbc9bf7) guillaume@latitude-5411:/tmp/terraform$ pip list | grep grpcio
grpcio 1.49.0rc1
grpcio-tools 1.49.0rc1
(tmp-730f4738cbc9bf7) guillaume@latitude-5411:/tmp/terraform$ pytest tests/ -xvv
/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
====================================================================================================================================================================================================== test session starts ======================================================================================================================================================================================================
platform linux -- Python 3.9.13, pytest-7.1.2, pluggy-1.0.0 -- /home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/bin/python
cachedir: .pytest_cache
rootdir: /tmp/terraform, configfile: pytest.ini
plugins: asyncio-0.19.0, inmanta-2.4.0.dev20220819141146, postgresql-4.1.1, inmanta-extensions-7.0.1.dev20220819140631, inmanta-tfplugin-5.2.0, env-0.6.2
asyncio: mode=auto
collected 21 items
tests/features/test_config_tree.py::test_config_serialization PASSED [ 4%]
tests/features/test_config_tree.py::test_deprecated_config PASSED [ 9%]
tests/features/test_config_tree.py::test_block_config Fatal Python error: Aborted
Thread 0x00007f0bd67fc700 (most recent call first):
File "/usr/lib/python3.9/logging/__init__.py", line 1175 in _open
File "/usr/lib/python3.9/logging/__init__.py", line 1146 in __init__
File "/tmp/tmp5thwsb35/libs/terraform/plugins/tf/terraform_provider.py", line 97 in _io_logger
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Thread 0x00007f0bd6ffd700 (most recent call first):
File "/tmp/tmp5thwsb35/libs/terraform/plugins/tf/terraform_provider.py", line 102 in _io_logger
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Current thread 0x00007f0bd77fe700 (most recent call first):
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/grpc/_channel.py", line 1478 in __init__
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/grpc/__init__.py", line 1978 in insecure_channel
File "/tmp/tmp5thwsb35/libs/terraform/plugins/tf/terraform_provider.py", line 220 in open
File "/tmp/tmp5thwsb35/libs/terraform/plugins/terraform_resource.py", line 259 in pre
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/inmanta/agent/handler.py", line 748 in check_facts
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 58 in run
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 83 in _worker
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Thread 0x00007f0bd7fff700 (most recent call first):
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Thread 0x00007f0bec9cb700 (most recent call first):
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Thread 0x00007f0bed1cc700 (most recent call first):
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Thread 0x00007f0bedc0d700 (most recent call first):
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Thread 0x00007f0bee40e700 (most recent call first):
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Thread 0x00007f0beec0f700 (most recent call first):
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Thread 0x00007f0bef410700 (most recent call first):
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
File "/usr/lib/python3.9/threading.py", line 917 in run
File "/usr/lib/python3.9/threading.py", line 980 in _bootstrap_inner
File "/usr/lib/python3.9/threading.py", line 937 in _bootstrap
Thread 0x00007f0bf66b2740 (most recent call first):
File "/usr/lib/python3.9/selectors.py", line 469 in select
File "/usr/lib/python3.9/asyncio/base_events.py", line 1869 in _run_once
File "/usr/lib/python3.9/asyncio/base_events.py", line 601 in run_forever
File "/usr/lib/python3.9/asyncio/base_events.py", line 634 in run_until_complete
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pytest_asyncio/plugin.py", line 454 in inner
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/python.py", line 192 in pytest_pyfunc_call
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/python.py", line 1761 in runtest
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/runner.py", line 166 in pytest_runtest_call
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/runner.py", line 259 in <lambda>
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/runner.py", line 338 in from_call
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/runner.py", line 258 in call_runtest_hook
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/runner.py", line 219 in call_and_report
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/runner.py", line 130 in runtestprotocol
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/runner.py", line 111 in pytest_runtest_protocol
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/main.py", line 347 in pytest_runtestloop
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/main.py", line 322 in _main
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/main.py", line 268 in wrap_session
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/main.py", line 315 in pytest_cmdline_main
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/config/__init__.py", line 164 in main
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/lib/python3.9/site-packages/_pytest/config/__init__.py", line 187 in console_main
File "/home/guillaume/.virtualenvs/tmp-730f4738cbc9bf7/bin/pytest", line 8 in <module>
Aborted (core dumped)This is not very minimal, but I couldn't find any other way to reproduce it so far.
What did you expect to see?
The test suite should work.
What did you see instead?
E0819 04:34:59.521602551 3104795 forkable.cc:42] assertion failed: !absl::exchange(g_registered, true)
Fatal Python error: Aborted
Thread 0x00007f48ebfff700 (most recent call first):
File "/usr/lib64/python3.9/logging/__init__.py", line 1175 in _open
File "/usr/lib64/python3.9/logging/__init__.py", line 1146 in __init__
File "/tmp/tmpvh_l8_8y/libs/terraform/plugins/tf/terraform_provider.py", line 97 in _io_logger
File "/usr/lib64/python3.9/threading.py", line 910 in run
File "/usr/lib64/python3.9/threading.py", line 973 in _bootstrap_inner
File "/usr/lib64/python3.9/threading.py", line 930 in _bootstrap
Thread 0x00007f48eb7fe700 (most recent call first):
File "/tmp/tmpvh_l8_8y/libs/terraform/plugins/tf/terraform_provider.py", line 102 in _io_logger
File "/usr/lib64/python3.9/threading.py", line 910 in run
File "/usr/lib64/python3.9/threading.py", line 973 in _bootstrap_inner
File "/usr/lib64/python3.9/threading.py", line 930 in _bootstrap
Current thread 0x00007f4908f79700 (most recent call first):
File "/home/jenkins/workspace/modules_terraform_master/env/lib64/python3.9/site-packages/grpc/_channel.py", line 1478 in __init__
File "/home/jenkins/workspace/modules_terraform_master/env/lib64/python3.9/site-packages/grpc/__init__.py", line 1978 in insecure_channel
File "/tmp/tmpvh_l8_8y/libs/terraform/plugins/tf/terraform_provider.py", line 220 in open
File "/tmp/tmpvh_l8_8y/libs/terraform/plugins/terraform_resource.py", line 259 in pre
File "/home/jenkins/workspace/modules_terraform_master/env/lib64/python3.9/site-packages/****/agent/handler.py", line 748 in check_facts
File "/usr/lib64/python3.9/concurrent/futures/thread.py", line 52 in run
File "/usr/lib64/python3.9/concurrent/futures/thread.py", line 77 in _worker
File "/usr/lib64/python3.9/threading.py", line 910 in run
File "/usr/lib64/python3.9/threading.py", line 973 in _bootstrap_inner
File "/usr/lib64/python3.9/threading.py", line 930 in _bootstrap
Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).
See TROUBLESHOOTING.md for how to diagnose problems better.
Anything else we should know about your project / environment?
The testing environment uses some multi-threading and asyncio.