← 返回首页
[rasterio] Add stubs for rasterio 1.5 by thomas-maschler · Pull Request #15884 · python/typeshed · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Resetting focus

[rasterio] Add stubs for rasterio 1.5#15884

Open
thomas-maschler wants to merge 13 commits into
python:mainfrom
thomas-maschler:add-rasterio-stubs
Open

[rasterio] Add stubs for rasterio 1.5#15884
thomas-maschler wants to merge 13 commits into
python:mainfrom
thomas-maschler:add-rasterio-stubs

Conversation

Copy link
Copy Markdown

Summary

Adds type stubs for rasterio tracking the current rasterio==1.5.* release.

Follow-up to #15870 — please refer to that issue for the discussion that shaped the PR's scope (in particular the decision to ship affine references through a stubs-only indirection instead of bundling types-affine).

Also addresses rasterio/rasterio#2322 (the upstream type-hinting tracker): rasterio's maintainers opted not to add inline annotations, so external stubs are the practical alternative — this PR provides them.

What's included

  • 43 stub files covering the rasterio 1.5 public API surface, plus the Cython _-prefixed modules transitively required by public re-exports (_base, _io, _warp, _env, _err, _features, _filepath, _path, _show_versions, _transform, _typing, _version, _vsiopener, _affine_types).
  • METADATA.toml with version = "1.5.*", requires-python = ">=3.12" (rasterio 1.5 itself requires 3.12+), dependencies = ["numpy>=2", "click>=8"], and a [tool.stubtest] section pinned to rasterio==1.5.* plus libgdal-dev / gdal apt/brew packages.
  • partial-stub = true — the entire public surface is covered, but a handful of internal Cython helpers are deliberately Incomplete.
  • @tests/stubtest_allowlist.txt for the remaining @disjoint_base markers / Cython-introspected parameter-name drift in a small set of private modules.
  • @tests/test_cases/check_{crs,dataset,open,transform_geom,windows}.py regression tests.

affine.Affine strategy

Per the discussion on #15870, rasterio's stubs reference affine.Affine through a single stubs-only indirection module (rasterio._affine_types) that aliases Affine to Any:

# rasterio/_affine_types.pyi from typing import Any, TypeAlias Affine: TypeAlias = Any

Affine v3 is currently in the 3.0rc3 pre-release and already ships complete inline type hints plus a py.typed marker (affine.py was moved to affine/__init__.py in rc3 specifically to enable that distribution). When v3 is final, the swap is one line in one file:

# rasterio/_affine_types.pyi from affine import Affine as Affine

This avoids any need to ship types-affine in typeshed.

PEP 702 @deprecated markers

Cover every documented RasterioDeprecationWarning site, including the overload-based parameter deprecations:

  • warp.transform_geom(antimeridian_cutting=, antimeridian_offset=)
  • windows.from_bounds(height=, width=, precision=)
  • merge.merge(precision=)
  • features.geometry_window(north_up=, rotated=, pixel_precision=)
  • transform.TransformerBase.rowcol(precision=)
  • windows.WindowMethodsMixin.window(precision=)

…and whole-symbol cases: CRS.is_valid, Window.round_shape, DatasetReaderBase.statistics, DatasetBase.is_tiled, env.hascreds, env.ensure_env_credentialled, io.FilePath, path.parse_path, path.vsi_path.

Two deprecations cannot be expressed via overload because **rpc_options swallows the deprecated kwarg in the modern overload (transform.rowcol(precision=) and transform.TransformMethodsMixin.index(precision=)); these carry inline notes in the stub.

Local validation

Run against rasterio==1.5.0 in a fresh venv:

Check Result
check_typeshed_structure pass
mypy_test (3.10‑3.15) pass
regr_test rasterio pass
stubtest_third_party rasterio pass
pyright (basic settings) pass
ruff check + ruff format --check pass
flake8 --select=Y pass

Notes

  • WktVersion.WKT2_2019 = "WKT2_2018" — the stub mirrors this upstream quirk (likely a typo, but stubtest would flag any "correction").
  • plot.contrast_strech is misspelled upstream; the stub matches with an inline comment.
  • rasterio.rio.* (the CLI) is deliberately not stubbed.

thomas-maschler and others added 2 commits June 8, 2026 12:31
Covers the public rasterio 1.5.x API plus the private Cython modules that are transitively required by public re-exports. `partial-stub = true` (a handful of internal Cython helpers are `Incomplete`). `affine.Affine` is referenced through a single stubs-only indirection module (`rasterio._affine_types`) that aliases `Affine` to `Any` until affine ships its own `py.typed` (planned for v3), per discussion in issue python#15870. When affine v3 lands, replacing the body of `_affine_types.pyi` with `from affine import Affine as Affine` will restore precision package-wide in a single edit. PEP 702 `@deprecated` markers cover every documented `RasterioDeprecationWarning` site, including the overload-based parameter deprecations (warp.transform_geom antimeridian_*, windows.from_bounds height/width/precision, merge precision, features.geometry_window north_up/rotated/pixel_precision, transform.TransformerBase.rowcol precision, windows.WindowMethodsMixin .window precision) and whole-symbol cases (CRS.is_valid, Window .round_shape, DatasetReaderBase.statistics, DatasetBase.is_tiled, env.hascreds, env.ensure_env_credentialled, io.FilePath, rasterio.path.{parse_path,vsi_path}). Closes python#15870 (in part; see issue for the affine-stubs decision).

This comment has been minimized.

thomas-maschler and others added 3 commits June 8, 2026 12:55
rasterio ships self-contained binary wheels for every platform/Python combination typeshed CI targets (manylinux x86_64/aarch64, macOS x86_64/arm64, Windows), so no system GDAL is needed for stubtest. Removing libgdal-dev also unblocks the aggregate apt-install step: libgdal-dev on Ubuntu noble hard-depends on default-libmysqlclient-dev, whose libmysqlclient-dev provider is currently broken in the noble archive. Also strips justifying / explanatory comments throughout the stubs.
* check_open.py: rename `_opaque_mode` -> `check_opaque_mode` so the helper is no longer flagged as `reportUnusedFunction`. * check_transform_geom.py: mark the two intentional deprecated-overload calls with `# pyright: ignore[reportDeprecated]` (they exist to exercise the `@deprecated` overload routing).

This comment has been minimized.

thomas-maschler and others added 2 commits June 8, 2026 13:15
`pyrightconfig.stricter.json` (which checks rasterio because the package isn't on the exclude list) flagged `_base._parse_path(path: str): ...` for an unknown return type (`reportUnknownParameterType`) on every Darwin/Windows matrix combo. The runtime forwards to `rasterio._path._parse_path`; mirror its return annotation here. Also apply ruff format to drop the blank lines between adjacent class members (typeshed style; was getting re-applied by pre-commit-ci).

This comment has been minimized.

srittau left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

A few general notes:

  • For various reasons, we use tests only sparingly in typeshed for complicated situations. For example, assert_type(reader.bounds, BoundingBox) basically only repeats the type hint and tests that the type checkers works correctly.

    Without looking at the tests in detail, we should probably just remove them wholesale for now.

  • Using Any needs an explanation, which types are allowed or why Any is necessary, unless it's obvious for someone not familar with the library. dict[str, Any] is probably the most common case that's considered obvious, specializations like NDArray[Any] is another case. Example where it's not obvious:

    def __init__(self, *args: Any, **kwargs: Any) -> None: ...

    What happens with args and kwargs here? Why can't it be typed? Are they just forwarded? In that case we could type them with the arguments of the function they are forwarded to. Or are they forwarded to a function where the arguments are unknown? Then that should be a comment. Just leaving it unannotated for now is also an option.

    Another example:

    def clone_file_obj(fobj: Any) -> Any: ...

    What's the meaning of Any here?

Comment on lines +1 to +11
rasterio\._typing
rasterio\._affine_types
rasterio\.features\.Geometry
rasterio\.merge\.MethodFunction

rasterio\._base.*
rasterio\._io.*
rasterio\._env.*
rasterio\._err.*
rasterio\._features.*
rasterio\._transform.*

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

There should be comments explaining why these entries are on the allowlist.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

I added comments

Comment thread stubs/rasterio/METADATA.toml Outdated
Comment on lines +5 to +8
partial-stub = true

[tool.stubtest]
ignore-missing-stub = true

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

Are these two options really necessary? Are really modules (that are not listed in the allowlist) missing?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

i removed those flags added missing stubs

thomas-maschler and others added 4 commits June 11, 2026 09:55
Per maintainer feedback (PR python#15884): typeshed reserves test_cases for non-trivial overload-routing scenarios; the original tests largely restated the stub signatures (`assert_type(reader.bounds, BoundingBox)`) without adding coverage.
Per maintainer feedback on METADATA.toml and the allowlist (PR python#15884). * Drop `partial-stub = true` from METADATA.toml and the matching `ignore-missing-stub = true` from `[tool.stubtest]`. Stubtest now enforces full coverage of the public API. * Rewrite `@tests/stubtest_allowlist.txt` with per-entry header comments grouping the remaining allowlist patterns by purpose: - stubs-only helper modules (`_typing`, `_affine_types`) - stubs-only TypeAliases referenced in signatures only - Cython implementation dunders (__pyx_capi__, __test__, etc.) - attrs-generated introspection + comparator dunders - private Cython modules with param-name drift - vendored third-party packages (rasterio._vendor.*) - the click-based CLI subpackage (rasterio.rio.*) * Add genuine public API surfaced by the audit that follows from dropping `ignore-missing-stub`: - `_vsiopener.FileContainer.{isdir, isfile, ls, mtime, rm, size}` abstract methods - `env.Env.aws_creds_from_context_options` - `transform.TransformerBase.{__enter__, __exit__}` - `_filepath.FilePathBase.{close, exists}` - module-level `log` Logger on `features`, `io`, `session` - `dtypes.{f16i, f32i, f64i}` numpy.finfo constants
Per maintainer feedback (PR python#15884): "Using `Any` needs an explanation, which types are allowed or why `Any` is necessary". * Add five named TypeAliases to `rasterio/_typing.pyi` (with header comments documenting accepted types and runtime semantics): - `_GDALOption`: scalar values for every GDAL CSL-style option list — config (`set_gdal_config` / `Env`), per-call warp options (`reproject`, `WarpedVRT`), RPC/transformer options, and metadata tag values. `str | int | float | bool | None`. - `_OpenOption`: open/creation options — `_GDALOption` plus `Enum` members (encoded as `.name.upper()`) and tuples of scalars (joined with commas). Used by `rasterio.open`, `MemoryFile.open`, `ZipMemoryFile.open`, `shutil.copy`, `Dataset*Base.__init__`. - `_OGRGeometry`: opaque OGR geometry handle (Cython internal). - `_NestedScalar`: recursive `float | list[_NestedScalar]` — preserves nesting for `_warp.recursive_round`. - `_Opener`: fsspec-style opener callable for `rasterio.open(opener=)`. * Use `TypeVar("_FileT")` in `_filepath.clone_file_obj(fobj) -> fobj` to express that the returned file-like has the same concrete type as the input. * Use `object` (rather than `Any`) on Cython catch-all signatures where any value is accepted at the boundary but callers can't introspect: `FilePathBase.__init__(*args: object, **kwargs: object)` and `_features.GeomBuilder.build(geom: object)`. * Replace dtype-typed `Any` with `numpy.typing.DTypeLike` on `_io._gdal_typename`, `_get_gdal_dtype`, `_is_complex_int`. * Add brief inline comments naming the accepted foreign-library types where they're irreducibly `Any` (matplotlib axes/pyplot, boto3.Session, swiftclient.Connection). * Document `**kwargs` for the deprecated swallow pattern on `Window.round_lengths` / `round_offsets` / `round_shape` (accept legacy `op` and `pixel_precision` kwargs). * `dtypes.is_ndarray`, `crs.CRS.from_user_input`: brief comment explaining the runtime dispatch.

This comment has been minimized.

Copy link
Copy Markdown
Author

Thanks for the review. This is ready for another look. I addressed your comments in separate commits.

This comment has been minimized.

Copy link
Copy Markdown
Author

@srittau please let me know if there are any other changes I should do. Thanks

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

thomas-maschler requested a review from srittau June 29, 2026 17:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Footer

© 2026 GitHub, Inc.