Update cython requirement from <3 to <4 #703

dependabot · 2023-07-17T20:55:10Z

Updates the requirements on cython to permit the latest version.

Changelog

3.0.0 unified release notes

Cython 3.0.0 has been a very large effort that cleaned up many old warts, introduced many new features, and introduces a couple of intentional behaviour changes, even though the goal remained to stay compatible as much as possible with Cython 0.29.x. For details, see the migration guide_.

.. _migration guide: https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html

As the development was spread out over several years, a lot of things have happened in the meantime. Many crucial bugfixes and some features were backported to 0.29.x and are not strictly speaking "new" in Cython 3.0.0.

Major themes in 3.0.0

Compatibility with Python

Since Cython 3.0.0 started development, CPython 3.8-3.11 were released. All these are supported in Cython, including experimental support for the in-development CPython 3.12. On the other end of the spectrum, support for Python 2.6 was dropped.

Cython interacts very closely with the C-API of Python, which is where most of the adaptation work happens. Independently, Cython strives to be able to parse newer Python constructs for use with its pure python_ mode, which has been a focus. In short, this allows to compile a wider range of Python code into optimized C code.

.. _pure python: https://cython.readthedocs.io/en/latest/src/tutorial/pure.html

Implemented PEPs ^^^^^^^^^^^^^^^^

PEP-3131_: Supporting Non-ASCII Identifiers (Github issue :issue:2601)

PEP-479_: generator_stop (enabled by default for language_level=3) (Github issue :issue:2580)

PEP-487_: Simpler customisation of class creation (Github issue :issue:2781)

PEP-563_: Postponed Evaluation of Annotations (Github issue :issue:3285)

PEP-570_: Positional-Only Parameters (Github issue :issue:2915)

PEP-572_: Assignment Expressions (a.k.a. the walrus operator :=) (Github issue :issue:2636)

PEP-590_: Vectorcall protocol (Github issue :issue:2263)

PEP-614_: Relaxing Grammar Restrictions On Decorators (Github issue :issue:4570)

Typing support in the sense of PEP-484_ (Github issues :issue:3949, :issue:4243) and PEP-560_ (Github issues :issue:2753, :issue:3537, :issue:3764) was also improved.

.. _PEP-3131: https://www.python.org/dev/peps/pep-3131 .. _PEP-479: https://www.python.org/dev/peps/pep-0479

... (truncated)

Commits

2f3a781 Prepare release of Cython 3.0.0.
ac4f0bb Update changelog.
039ef7c In Py2, the extension type might get cleaned up before the objects at system ...
8edd90a Prepare second release candidate 3.0.0rc2.
7efee97 Update changelog.
c0a3556 Test including "complex.h" in MSVC (#5527)
36a3702 Write the version to stderr as before, but only if stderr is actually capture...
8169009 Update changelog and version.
864e563 Support hex constants in enums (GH-5525)
0c6141a Document @cython.with_gil (GH-5522)
Additional commits viewable in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Note
Automatic rebases have been disabled on this pull request as it has been open for over 30 days.

dependabot · 2023-07-17T20:55:12Z

The following labels could not be found: dependencies.

codecov · 2023-07-17T20:59:16Z

Codecov Report

Patch coverage has no change and project coverage change: -1.43% ⚠️

Comparison is base (2897261) 98.44% compared to head (b9b2d01) 97.01%.
Report is 1 commits behind head on master.

❗ Current head b9b2d01 differs from pull request most recent head 304a791. Consider uploading reports for the commit 304a791 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #703      +/-   ##
==========================================
- Coverage   98.44%   97.01%   -1.43%     
==========================================
  Files          32       32              
  Lines        2245     2243       -2     
==========================================
- Hits         2210     2176      -34     
- Misses         35       67      +32

Files Changed	Coverage Δ
fastavro/_logical_readers_py.py	`100.00% <ø> (ø)`
fastavro/_logical_writers_py.py	`97.54% <ø> (-1.64%)`	⬇️
fastavro/_read_py.py	`95.41% <ø> (-2.99%)`	⬇️
fastavro/_schema_py.py	`98.87% <ø> (ø)`
fastavro/_write_py.py	`93.00% <ø> (-5.01%)`	⬇️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

scottbelden · 2023-07-28T20:03:31Z

@dependabot rebase

kloczek · 2023-09-08T15:10:07Z

This PR is not changing cython dependencies.
Here it is additional patch:

--- a/pyproject.toml
+++ b/pyproject.toml
@@ -2,7 +2,7 @@
 requires = [
     "setuptools",
     "wheel",
-    "Cython<3",
+    "Cython<4",
 ]
 build-backend = "setuptools.build_meta"

Nevertheless just released fastavro 1.8.3 is not ready for cython 3.x (I'm using cython 3.0.2)

+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
running egg_info
creating fastavro.egg-info
writing fastavro.egg-info/PKG-INFO
writing dependency_links to fastavro.egg-info/dependency_links.txt
writing entry points to fastavro.egg-info/entry_points.txt
writing requirements to fastavro.egg-info/requires.txt
writing top-level names to fastavro.egg-info/top_level.txt
writing manifest file 'fastavro.egg-info/SOURCES.txt'
reading manifest file 'fastavro.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'conda.recipe'
warning: no files found matching '*.c' under directory 'fastavro'
adding license file 'LICENSE'
adding license file 'NOTICE.txt'
writing manifest file 'fastavro.egg-info/SOURCES.txt'
* Building wheel...
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-38
creating build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/__init__.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/__main__.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_logical_readers_py.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_logical_writers_py.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_read_common.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_read_py.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_schema_common.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_schema_py.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_validate_common.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_validation_py.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_write_common.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/_write_py.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/const.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/json_read.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/json_write.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/logical_readers.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/logical_writers.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/read.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/schema.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/types.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/utils.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/validation.py -> build/lib.linux-x86_64-cpython-38/fastavro
copying fastavro/write.py -> build/lib.linux-x86_64-cpython-38/fastavro
creating build/lib.linux-x86_64-cpython-38/fastavro/io
copying fastavro/io/__init__.py -> build/lib.linux-x86_64-cpython-38/fastavro/io
copying fastavro/io/binary_decoder.py -> build/lib.linux-x86_64-cpython-38/fastavro/io
copying fastavro/io/binary_encoder.py -> build/lib.linux-x86_64-cpython-38/fastavro/io
copying fastavro/io/json_decoder.py -> build/lib.linux-x86_64-cpython-38/fastavro/io
copying fastavro/io/json_encoder.py -> build/lib.linux-x86_64-cpython-38/fastavro/io
copying fastavro/io/parser.py -> build/lib.linux-x86_64-cpython-38/fastavro/io
copying fastavro/io/symbols.py -> build/lib.linux-x86_64-cpython-38/fastavro/io
creating build/lib.linux-x86_64-cpython-38/fastavro/repository
copying fastavro/repository/__init__.py -> build/lib.linux-x86_64-cpython-38/fastavro/repository
copying fastavro/repository/base.py -> build/lib.linux-x86_64-cpython-38/fastavro/repository
copying fastavro/repository/flat_dict.py -> build/lib.linux-x86_64-cpython-38/fastavro/repository
copying fastavro/py.typed -> build/lib.linux-x86_64-cpython-38/fastavro
running build_ext

Error compiling Cython file:
------------------------------------------------------------
...
            writer_schema, named_schemas, offset, size, options,
        )


class Block:
    def __init__(
    ^
------------------------------------------------------------

fastavro/_read.pyx:966:4: Compiler crash in AnalyseDeclarationsTransform

File 'ModuleNode.py', line 203, in analyse_declarations: ModuleNode(_read.pyx:1:0,
    doc = 'Python code for reading AVRO files',
    full_module_name = 'fastavro._read')
File 'Nodes.py', line 393, in analyse_declarations: StatListNode(_read.pyx:10:0)
File 'Nodes.py', line 393, in analyse_declarations: StatListNode(_read.pyx:965:0)
File 'Nodes.py', line 5169, in analyse_declarations: PyClassDefNode(_read.pyx:965:0,
    name = 'Block')
File 'Nodes.py', line 393, in analyse_declarations: StatListNode(_read.pyx:966:4)
File 'Nodes.py', line 2712, in analyse_declarations: CFuncDefNode(_read.pyx:966:4,
    args = [...]/10,
    modifiers = [...]/0,
    outer_attrs = [...]/2,
    overridable = True,
    visibility = 'private')
File 'Nodes.py', line 2723, in declare_cpdef_wrapper: CFuncDefNode(_read.pyx:966:4,
    args = [...]/10,
    modifiers = [...]/0,
    outer_attrs = [...]/2,
    overridable = True,
    visibility = 'private')
File 'Nodes.py', line 2789, in call_self_node: CFuncDefNode(_read.pyx:966:4,
    args = [...]/10,
    modifiers = [...]/0,
    outer_attrs = [...]/2,
    overridable = True,
    visibility = 'private')

Compiler crash traceback from this point on:
  File "/usr/lib64/python3.8/site-packages/Cython/Compiler/Nodes.py", line 2789, in call_self_node
    type_entry = self.type.args[0].type.entry
AttributeError: 'PyObjectType' object has no attribute 'entry'
Compiling fastavro/_read.pyx because it changed.
[1/1] Cythonizing fastavro/_read.pyx
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
    main()
  File "/usr/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/usr/lib/python3.8/site-packages/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
    return _build_backend().build_wheel(wheel_directory, config_settings,
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 416, in build_wheel
    return self._build_with_temp_dir(['bdist_wheel'], '.whl',
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
    self.run_setup()
  File "/usr/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in run_setup
    exec(code, locals())
  File "<string>", line 37, in <module>
  File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 107, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 349, in run
    self.run_command("build")
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
    super().run_command(command)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run
    _build_ext.run(self)
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
    self.build_extensions()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
    self._build_extensions_serial()
  File "/usr/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
    self.build_extension(ext)
  File "/usr/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
    _build_ext.build_extension(self, ext)
  File "/usr/lib64/python3.8/site-packages/Cython/Distutils/build_ext.py", line 122, in build_extension
    new_ext = cythonize(
  File "/usr/lib64/python3.8/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
    cythonize_one(*args)
  File "/usr/lib64/python3.8/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: fastavro/_read.pyx

ERROR Backend subprocess exited when trying to invoke build_wheel

scottbelden · 2023-09-08T16:20:54Z

@kloczek Good point. At some point I will take a look at the cython issue, but I'm hoping if I wait long enough it will mostly get solved.

kloczek · 2023-09-08T16:24:16Z

Feel free to add that to your PR 😋

kloczek · 2023-09-08T16:25:20Z

Up I see that you already added that 👍 😄

dependabot · 2023-09-15T20:42:44Z

A newer version of cython exists, but since this PR has been edited by someone other than Dependabot I haven't updated it. You'll get a PR for the updated version as normal once this PR is merged.

Updates the requirements on [cython](https://github.com/cython/cython) to permit the latest version. - [Release notes](https://github.com/cython/cython/releases) - [Changelog](https://github.com/cython/cython/blob/master/CHANGES.rst) - [Commits](cython/cython@0.9.6.14...3.0.0) --- updated-dependencies: - dependency-name: cython dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]>

scottbelden · 2023-09-19T03:47:51Z

Notes for the future:

Initially had to pin cython to less than version 3 because it was crashing: [BUG] AttributeError: 'PyObjectType' object has no attribute 'entry' cython/cython#5704
Suggestion was made to remove auto_cpdef comments which exposed the real problem
Real problem is that _iter_avro_records in _read.pyx was not catching the StopIteration that could be raised in read_long
While fixing that, I decided to change the StopIteration to EOFError to better describe the actual error.

dependabot bot force-pushed the dependabot/pip/cython-lt-4 branch from affff64 to f0d0902 Compare July 28, 2023 20:04

scottbelden mentioned this pull request Sep 15, 2023

[BUG] AttributeError: 'PyObjectType' object has no attribute 'entry' cython/cython#5704

Open

dependabot bot and others added 4 commits September 17, 2023 18:47

change pin in pyproject

0064990

remove auto cpdef and language level

94bd868

add virtualenvs to gitignore

78c2b14

scottbelden force-pushed the dependabot/pip/cython-lt-4 branch from 917c092 to 78c2b14 Compare September 17, 2023 22:48

scottbelden added 3 commits September 17, 2023 18:57

add back in language level

b9b2d01

use EOFError and make sure to catch it in iter_avro_records

96e4fca

remove restriction on cython version

304a791

scottbelden merged commit d7f854e into master Sep 19, 2023

scottbelden deleted the dependabot/pip/cython-lt-4 branch September 19, 2023 03:48

Linchin mentioned this pull request Oct 5, 2023

bug: add exception case to support fastavro 1.8.4 googleapis/python-bigquery-storage#687

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update cython requirement from <3 to <4 #703

Update cython requirement from <3 to <4 #703

Uh oh!

dependabot bot commented on behalf of github Jul 17, 2023 •

edited

Loading

Uh oh!

dependabot bot commented on behalf of github Jul 17, 2023

Uh oh!

codecov bot commented Jul 17, 2023 •

edited

Loading

Uh oh!

scottbelden commented Jul 28, 2023

Uh oh!

kloczek commented Sep 8, 2023

Uh oh!

scottbelden commented Sep 8, 2023

Uh oh!

kloczek commented Sep 8, 2023

Uh oh!

kloczek commented Sep 8, 2023

Uh oh!

dependabot bot commented on behalf of github Sep 15, 2023

Uh oh!

scottbelden commented Sep 19, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Update cython requirement from <3 to <4 #703

Update cython requirement from <3 to <4 #703

Uh oh!

Conversation

dependabot bot commented on behalf of github Jul 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

3.0.0 unified release notes

Major themes in 3.0.0

Compatibility with Python

Uh oh!

dependabot bot commented on behalf of github Jul 17, 2023

Uh oh!

codecov bot commented Jul 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

scottbelden commented Jul 28, 2023

Uh oh!

kloczek commented Sep 8, 2023

Uh oh!

scottbelden commented Sep 8, 2023

Uh oh!

kloczek commented Sep 8, 2023

Uh oh!

kloczek commented Sep 8, 2023

Uh oh!

dependabot bot commented on behalf of github Sep 15, 2023

Uh oh!

scottbelden commented Sep 19, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dependabot bot commented on behalf of github Jul 17, 2023 •

edited

Loading

codecov bot commented Jul 17, 2023 •

edited

Loading