- Updated OpenAPI schema to reflect changes in API endpoints.
- Renamed parameters in API calls for consistency, changing `column_name` to `columns` in lineage-related endpoints.
- Adjusted the handling of optional properties in the `ModelSQLMeshModel` class to ensure default values are set.
- Changed route parameter types from `str` to `path` for better URL handling in several endpoints.
These changes improve the clarity and functionality of the API.
Description
Added support for Amazon S3 Tables (managed Iceberg tables) within the Athena Engine Adapter.
Specific changes include:
Catalog Support: Updated AthenaEngineAdapter.catalog_support to CatalogSupport.FULL_SUPPORT to allow cross-catalog queries without issuing SET CATALOG commands. This prevents SQLMesh from stripping out custom catalog names (e.g., s3tablescatalog/...) from model names.
S3 Table Partitioning: For S3 tables, partition properties are now correctly generated as partitioning=ARRAY['...'] instead of the standard partitioned_by property format.
S3 Table Properties: S3 Table CTAS queries in Athena implicitly default to table_type='ICEBERG' and do not allow explicit specification of most table properties (except format='PARQUET'). The adapter now filters out unsupported TBLPROPERTIES and table_type when generating CREATE TABLE ... AS statements for S3 tables to prevent pyathena.error.OperationalError.
Dictionary Mutation Bug: Fixed a subtle bug in _build_table_properties_exp where table_properties was being mutated during AST parsing, causing properties like s3_table to drop out during intermediate schema table creation.
Test Plan
Tested creating S3 Iceberg tables via sqlmesh plan utilizing custom MODEL properties (s3_table true / catalog defined).
Verified partitioning is properly formulated as an unquoted ARRAY structure for S3 tables.
Verified that temporary intermediate schema generation properly inherits S3 properties without failure.
Ran make fast-test to ensure no regressions with standard Hive/Iceberg tables in the Athena adapter
Checklist
I have run make style and fixed any issues
I have added tests for my changes (if applicable)
All existing tests pass (make fast-test)
My commits are signed off (git commit -s) per the DCO