apache-airflow-providers-common-ai

Changelog

0.11.0

Release Date: 2026-10-08

Note

On Airflow >= 3.3, AgentOperator’s usage_limits counts usage across every attempt of the task instance combined – initial run, every retry, and every HITL regeneration add to one running total (including the implicit request_limit=50 default), instead of each attempt starting fresh. Scale each limit by retries + 1, or set usage_limits=None, to keep the old per-attempt headroom. On every Airflow version, a regeneration shares its count with the run before it only when usage_limits is set. A step replayed with durable=True no longer counts toward usage_limits or the usage XCom, on any Airflow version. The usage XCom is now also pushed on a failed attempt, reporting that attempt’s own usage (not the cross-attempt total). On Airflow < 3.3, or when usage_limits is None, each attempt is checked and counted on its own, unchanged. See the cross-attempt usage budget.

Note

get_schema on SQLToolset and DataFusionToolset now returns a JSON object {"columns": [{"name", "type"}, ...], "column_count": N} instead of a bare JSON array of columns. The tool also accepts an optional name_contains substring filter, and on a table with more columns than max_columns (default 100), or one whose serialized columns exceed max_result_bytes, it returns a bounded summary (column_count, a type_histogram and a sample_columns preview, with truncated, truncated_by and a hint) in place of the full list. Update any system prompt or direct call_tool("get_schema", ...) caller that read the old top-level array: check truncated first, then read result["columns"] on a full result or result["sample_columns"] on a summary. A summary carries no columns key, so result["columns"] raises KeyError on any table wide enough to be summarized.

Note

AgentOperator no longer accepts code_mode. Pass the pydantic-ai-harness capability instead: replace code_mode=True with capabilities=[CodeMode()], using from pydantic_ai_harness import CodeMode. The capability takes its own arguments, such as max_tool_calls. If the task also passes agent_params={"capabilities": [...]}, move those into capabilities= too, since the two cannot be combined. The import runs when the Dag file is parsed, so the code-mode extra is now needed by the Dag processor as well as the workers. The extra’s floor is now pydantic-ai-harness>=0.24.0, the first release with max_tool_calls. See Code mode.

Features

  • Add 'max_retries' to 'AgentSkillsToolset' (#74381)

  • Add 'include_traceback' to model-backed retry policies (#74308)

  • Let the Modal sandbox backend read credentials from a modal connection (#74302)

  • Include prompt cache token counts in AgentOperator's usage XCom (#74277)

  • Bound get_schema results for very wide tables in the SQL toolsets (#74017)

  • Support Airflow 2.11 in the Common AI provider (#73991)

  • Cache repeated agent prompts by default in AgentOperator (#73994)

  • Add a vendor-neutral managed-agent hook contract to Common AI (#73532)

  • Export files an agent built in a sandbox to object storage (#73990)

  • Add first-class 'capabilities' to Common AI 'AgentOperator' (#73984)

  • Let Common AI toolsets use the agent's tool retry budget (#73957)

  • Add embedding kwargs to common AI hooks and operators (#72002)

  • Tell the model how many rows a Parquet or Avro file holds (#73925)

  • Count and trace tool calls made by native agent frameworks (#73901)

  • Let HookToolset pin arguments the model must not choose (#73900)

  • Add 'ObjectStorageToolset' for reading files on object storage (#73899)

  • Rework common.ai Strands support into a plugin and add Google ADK (#73898)

  • Name the tools a durable agent retry will run again (#73873)

  • Return an approved agent answer as 'output_type' after human review (#73904)

  • Let the sandbox toolset attach to a sandbox another task provisioned (#73559)

  • Add Strands Agents support for 'common.ai' toolsets (#73587)

  • Add Modal provider with 'ModalHook' and connection type (#73418)

  • Add OpenSandbox backend for sandbox tools (#71676)

  • Let a person approve an agent's tool calls before they run (#73586)

Bug Fixes

  • Tell the model a pinned hook argument is fixed for every allowed method (#74380)

  • Keep retried task attempts and their data under the attempt UUID (#74222)

  • Fix Common AI durable retries not replaying tools from a capability without an id (#74314)

  • Fix Common AI durable retries and tool-approval resumes on pydantic-ai 2.50+ (#74313)

  • Mask secrets in tool results before they reach the model (#73897)

  • Account for AgentOperator spend on failed runs and across retries (#73706)

  • Stop 'DataFusionToolset' from materializing full results for a capped query (#73384)

  • Fix mypy error in common.ai decision helper with pydantic-ai 2.46 (#73652)

Misc

  • Drop support for Python 3.10 (#74157)

  • Remove Python 3.10 compatibility shims (#74153)

  • Remove 'code_mode' from 'AgentOperator' in favor of the 'CodeMode' capability (#74312)

  • Make 'execute_tool' the public method 'AirflowToolset' subclasses implement (#73938)

  • Mark experimental 'common.ai' features ahead of 1.0 (#73896)

  • Require 'common.sql' 2.2.0 for common.ai's SQL extras (#73867)

Doc-only

  • Explain unknown platform labels in managed-agent metrics (#72786)

  • Add toolset overview and MCP tool filtering example to 'common.ai' docs (#74378)

  • Link registry modules to the guide section that documents them (#71477)

  • Add restricted-agent examples to common.ai toolset guides (#74379)

  • Support PAT and key-pair auth in the Snowflake Cortex Agent hook (#73932)

  • Update provider READMEs for the Python 3.11 baseline (#74158)

  • Lead the 'common.ai' sandbox docs with when to use it and where each piece runs (#74297)

  • Correct usage_limits=None docstring on AgentOperator and LLMOperator (#74307)

  • Document decision models served over the System One API in common.ai (#74266)

  • Add Azure Blob Storage support to the DataFusion object storage layer (#73374)

  • Render the HITL review workflow as a Mermaid diagram (#74024)

  • Use "Airflow versions" in user-facing docs, not "cores" (#74045)

  • Document running your own Pydantic AI agent, and sandboxes in other frameworks (#73902)

  • Surface the retry policy decision on task instances page (#73030)

  • Update changelog to clarify new LLMBranchOperator "branches" feature (#73850)

  • Add connections and agent-security entries to the common.ai sidebar (#73837)

  • Persist 'retry_reason' not just for retries but even when a task fails (#73027)

0.10.0

Release Date: 2026-09-29

Note

LLMBranchOperator and LLMOperator now push a decision XCom on every run, next to return_value, with the model’s pick, the action taken, the confidence and probabilities when the model reports them, and the decision_policy that applied. LLMBranchOperator also offers the downstream task IDs to the model in sorted order (it was set order, which differed between workers), gained branches as a template field, and builds its option type from pydantic-ai’s Choices on 2.46+ or an equivalent enum whose member names are generated; the option values are still the task IDs, so do_branch receives the same strings as before.

Note

execute_complete on LLMOperator, LLMBranchOperator, LLMSQLQueryOperator and LLMSchemaCompareOperator gained a keyword argument, decision. LLMOperator and LLMBranchOperator pass it on resume, so a subclass of either that overrides execute_complete with the old three-argument signature raises TypeError when the reviewed task resumes; add decision=None to the override. The other two accept the keyword but do not pass it yet. A review that was already pending when you upgraded resumes without it and is unaffected.

Note

Configuring fallback_conn_ids on a connection (or the matching operator/decorator argument) changes what exception a task raises once every connection in the chain fails: it is pydantic_ai.exceptions.FallbackExceptionGroup, not the last provider’s own exception. An existing RetryRule(exception=ModelHTTPError, ...) – in LLMRetryPolicy.fallback_rules or a plain ExceptionRetryPolicy – stops matching as soon as the connection gains a fallback chain, with no change to the Dag needed to trigger it. Match pydantic_ai.exceptions.FallbackExceptionGroup explicitly as well, or inspect its .exceptions attribute for the original per-model errors. See Retry policies, “When the connection also carries a fallback chain”.

Note

SQLToolset now rejects allowed_tables=None and allowed_tables=[] with ValueError. Up to 0.9.0 both were accepted and exposed every table in the schema, so a Dag that builds the list dynamically (a Variable.get with a None default, a config file, a filtered comprehension) silently handed the agent the whole schema whenever the lookup came back empty. Such a Dag now fails at import instead. Exposing every table is still the default, but only by omitting the argument: no value you can pass requests it, so a runtime lookup can never widen the allow-list by accident. Dags that passed allowed_tables=None explicitly should drop the argument.

Note

The PydanticAI vendor connection types are renamed from pydanticai-azure, pydanticai-bedrock and pydanticai-vertex to pydanticai_azure, pydanticai_bedrock and pydanticai_vertex. The hyphenated names could never be expressed as a connection URI scheme, so a connection stored in a secrets backend or in AIRFLOW_CONN_* never resolved. An existing connection created with a hyphenated conn_type no longer matches its hook and the task fails to find it: re-create it with the underscored type (in the UI, pick the same vendor again and save).

Breaking changes

  • Reject allowed_tables=None and allowed_tables=[] in SQLToolset so only omitting it allows every table (#73381, #73452)

  • Add a decision policy so an LLM operator asks a person when the model is unsure (#73368)

  • Fix PydanticAI vendor connections not resolving from secrets backends (#72853)

Features

  • Allow templated connection IDs in agent toolsets (#73578)

  • Cancel the agent run when a common.ai LLM or agent task is killed (#73495)

  • Add JSON Lines support for DocumentLoaderOperator (#72246)

  • Add an address-layer egress allowlist for hosted sandboxes (#73534)

  • Add ClassifierRetryPolicy for classifier models and keep LLMRetryPolicy as the text-model policy (#73450, #73501)

  • Add a deferrable batch execution mode to common.ai (#72938)

  • Add a Modal backend for the sandbox toolset (#72910)

  • Add support for TypeSafe Jev classifier models (#73363)

  • Add branches to LLMBranchOperator so the model reads what each branch means (#73367, #73368)

  • Add connection-driven provider failover for common.ai LLM calls (#72156)

  • Support assigned reviewers in LLM approval reviews (#72157)

  • Stamp Airflow run identity onto agent runs and traces (#73275)

  • Support notifiers in LLM approval reviews (#72159)

  • Support timeout defaults in LLM approval reviews (#72155)

  • Support per-run cost limits in common.ai LLM and Agent operators (#71403)

Bug Fixes

  • Reject non-string prompts in LLMFileAnalysisOperator before reading files (#71734)

  • Reject durable replay and human-in-the-loop review when an agent holds a SandboxToolset (#73529)

  • Fix LLMBranchOperator sending branch options in a different order on each worker (#73366)

  • Fail LLMOperator approval at parse time on Airflow cores older than 3.1 (#73261)

  • Fix 'DocumentLoaderOperator' validation errors naming the wrong argument (#73053)

  • Report the Airflow version error first when human-in-the-loop review needs Airflow 3.1 or newer (#73052)

  • Restore Dag-parse-time validation for common.ai operator arguments (#70628)

Misc

  • Require pydantic-ai-slim 2.33.0 or newer so Anthropic models work with the anthropic 1.x SDK (#73511)

  • Replace the retired gemini-2.0-flash example model in the Vertex connection placeholders and docs (#73516)

  • Update the Azure OpenAI connection placeholders and document when api_version must be omitted (#73024)

  • Add TypedDict type hints for AirflowPlugin list fields (#69761)

Doc-only

  • Document sandbox enforcement, teardown and cost-limit scope for agents (#73589)

  • Add a guide to developing and testing Common AI tasks locally (#73602)

  • Fix stale and duplicated content in the 'common.ai' provider docs (#73567)

  • Lead the common.ai docs with a runnable quick start and a verifiable result (#73556)

  • Nest the 'common.ai' docs sidebar and lead with features and providers (#73548)

  • Add use-case pages to the common.ai provider docs (#73542)

  • Rename the common.ai retry policies page to cover both policies (#73526)

  • Reorganize 'common.ai' provider docs into topic-based navigation (#73523)

  • Add a decision guide for choosing between common.ai toolsets (#72936)

  • Document GCS data sources in the common.ai SQL toolset guides (#73370)

  • Update the LLM schema-compare guide and example for plain database tables (#73273)

  • Document how retry policies and durable execution work together (#73160)

  • Document usage_limits on all common.ai LLM operators (#73283)

  • Make LLM retry policy failure model documentation more accurate (#73158)

  • Document what the LLM can and cannot do in retry policies (#72947)

0.9.0

Release Date: 2026-09-14

Note

A rejected LLMBranchOperator review now skips the direct downstream tasks – teardown tasks excepted – instead of failing the task. Set fail_on_reject=True to keep failing the task, or ignore_downstream_trigger_rules=True to skip every downstream task rather than only the direct ones.

Features

  • Support bzip2 and xz compressed inputs in LLM file analysis (#70302)

  • Add .md file support in LLMFileAnalysisOperator (#71611)

  • Add test_connection support to LangChainHook and LlamaIndexHook (#71841)

  • Add BaseManagedAgentToolset for vendor-managed AI agents (#71946)

Bug Fixes

  • Add require_approval preflight check to @task.llm_schema_compare (#71688)

  • Skip downstream tasks instead of failing when an LLM branch review is rejected (#71073, #72183)

  • Fix Vertex AI hook silently discarding credentials when vertexai flag is set (#72012)

Misc

  • Import TaskInstanceState from airflow.sdk (#72446)

Doc-only

  • Fix LlamaIndexHook docs to stop claiming Ollama/vLLM support (#72013)

  • Document the missing resource category in common.ai retry policy docs (#72189)

  • Document LLMFileAnalysisOperator's inherited LLM and HITL parameters (#71856)

  • Fix reversed credential precedence in Bedrock hook docstring (#71826)

  • Correct the common-ai toolset list and document the shields extra (#71819)

0.8.0

Release Date: 2026-08-23

Note

The query tool of SQLToolset and DataFusionToolset returns a different shape. Rows were a dict per row alongside a count of every matching row: {"rows": [{"id": 1, "name": "a"}], "count": 900}. They are now columnar, and row_count counts the rows actually returned: {"columns": ["id", "name"], "rows": [[1, "a"]], "row_count": 1}. A truncated result also carries truncated_by – max_rows or the new max_result_bytes – and total_rows appears only when the driver reports a trustworthy query total. The tool’s own description states the new shape, so agents adapt without changes; update any system prompt that describes the old shape, and any code calling toolset.call_tool("query", ...) directly.

Features

  • Add SandboxToolset for sandboxed agent shell and file access (#68847)

  • Support require_approval in LLMSchemaCompareOperator (#71051)

  • Bound SQL toolset query results by size, not just row count (#71317)

  • Support custom redaction and message length cap in LLMRetryPolicy (#70830)

Bug Fixes

  • Let the agent retry on a rejected DataFusion query instead of failing the task (#71445)

Misc

  • Show which LLM providers each Common AI connection type reaches (#70497)

Doc-only

  • Document the dedicated pydantic-ai vendor connection types (#71774)

  • Document the common-ai MCPHook (#71817)

  • Document the common-ai LangChain and LlamaIndex connection types (#71818)

  • Fix and expand the common-ai provider's When to use guidance (#71820)

  • Add LLMSchemaCompareOperator to the common-ai operator index table (#71738)

0.7.0

Release Date: 2026-08-08

Note

The skills extra now requires pydantic-ai-skills>=1.2.0 (previously >=0.11.0); the file-exclusion support added in #69924 relies on the 1.x API. Every 0.x release is excluded, so environments that pin pydantic-ai-skills below 1.2.0 will fail to resolve apache-airflow-providers-common-ai[skills]. To migrate, upgrade pydantic-ai-skills to 1.2.0 or newer; if you cannot, stay on common.ai 0.6.0. Installations that do not use the skills extra are unaffected.

Note

SQLToolset(allowed_tables=...) now fails closed: COPY in any form, and any function sqlglot cannot type, are refused before the query runs. This closes a bypass where pg_read_file(), query_to_xml() and COPY ... FROM PROGRAM reached data and files outside allowed_tables. Project UDFs and a few common builtins such as json_build_object are also not recognized; pass them in the new allowed_functions argument to permit them. The database role remains the real security boundary.

Features

  • Add pydantic-ai capability matrix references doc to Common AI (#69887)

  • Support excluding files from 'common.ai' Agent Skills discovery (#69924)

  • Use task state store for 'common.ai' durable execution on Airflow 3.3+ (#68926)

  • Support '.txt' files in 'LLMFileAnalysisOperator' (#70431)

  • Support cloud URI globs in DocumentLoaderOperator (#70299)

  • Support require_approval in LLMBranchOperator (#70651)

  • Render multi-branch review choices as a multi-select (#71046)

Bug Fixes

  • Allow Common AI SQL imports without DataFusion (#69990)

  • Fix HITL review showing altered agent output to reviewers (#70070)

  • Fix 'LLMSQLQueryOperator' not stripping single-line markdown code fences (#70137)

  • Harden common.ai SQLToolset allowed_tables against function/COPY bypass (#70134)

  • Preserve output_type through human approval in LLM operators (#70075)

  • Fix DocumentLoaderOperator ignoring unknown parser on byte input (#70071)

  • Reject unsupported require_approval in LLM branch and schema compare operators (#70069)

  • Fix '@task.llm_branch' import failure on Task SDK-only workers (#70068)

  • Fix common.ai durable execution skipping Toolset-capability tools (#69881)

  • Redact secrets from LLMRetryPolicy classification prompts (#70229)

  • Validate common AI tool-call arguments to enable pydantic-ai retries (#70096)

  • Validate template fields after rendering in common.ai operators (#70338)

Misc

  • Add dataclasses-json floor to common.ai llamaindex extra (#69755)

  • Add a Retry Policies category to the provider registry (#70499)

  • Mark asserts under 'TYPE_CHECKING' in 'DocumentLoaderOperator' (#70378)

  • Mark KubernetesPodOperator and AgentOperator as durable capable (#70289)

Doc-only

  • Add self-hosted model guide for the common.ai provider (#69867)

  • Update retired model ids and fill doc gaps in common.ai provider docs (#69711)

  • Add quick start guide to common.ai provider docs (#69552)

  • Document the dynamic 'system_prompt' pattern for common-ai agents (#69636)

  • Add feature-comparison table and toolset links to common.ai provider docs (#69649)

  • Add an Examples entry point to the common.ai provider docs (#69650)

  • Document when to use common.ai vs vendor-specific AI providers (#69551)

  • Document the sql extra required for LLMSQLQueryOperator (#70564)

0.6.0

Release Date: 2026-07-10

Features

  • Add 'env_provider' and 'Extra.env' support to MCP stdio transport (#69225)

Bug Fixes

  • Replace deprecated 'pydantic-ai' MCP classes with 'MCPToolset' in 'common.ai' (#69006)

Misc

  • Migrate common.ai provider to pydantic-ai 2.x and remove the <2 cap (#69358)

0.5.0

Release Date: 2026-06-22

Note

SQLToolset(allowed_tables=[...]) now enforces the allow-list on the query and check_query tools, not only on metadata discovery (#68487). The SQL an agent submits is parsed with sqlglot and rejected before execution if it reaches any table outside the list (including through subqueries, CTEs, JOINs, set operations and DML). While a list is active, constructs a schema.table list cannot describe are also rejected: quoted identifiers, inline comments, cross-database references, SHOW, table-valued functions and dynamic SQL. Agents querying a restricted connection must send unquoted, comment-free SQL. This is an application-level guardrail and not a substitute for least-privilege database permissions.

Breaking changes

  • Enforce SQLToolset allowed_tables on queries, not just discovery (#68487)

Features

  • common.ai: Park approval reviews in awaiting_input on Airflow 3.3+ (#68489)

  • Add spec_file support to PydanticAIHook.create_agent (#67788)

  • Return common.ai SQLToolset errors to the agent so it self-corrects (#68117)

  • Allow DESCRIBE/SHOW in common.ai SQLToolset read-only queries (#68102)

  • Support multi-schema introspection in common.ai SQLToolset (#68103)

  • Add token_provider for short-lived MCP auth in common.ai (#68104)

  • Add code mode (Monty sandbox) to common.ai AgentOperator (#68407)

  • Add 'message_history' to 'AgentOperator' for multi-turn agent sessions (#68648)

Bug Fixes

  • Fix 'LlamaIndexEmbeddingOperator' returning 'vector=None' for every chunk (#68491)

  • Verify durable cached agent steps match the request before replay (#68372)

Misc

  • Access AgentRunResult.usage as a property in common.ai logging (#68405)

  • Bump pydantic-ai-slim>=1.99.0 (#68105)

Doc-only

  • Explain the agent tool boundary in common.ai security docs (#68404)

Breaking change: operators with output_type=<BaseModel subclass> (LLMOperator, LLMAgentOperator, LLMFileAnalysisOperator, and their @task.llm / @task.agent / @task.llm_file_analysis decorators) now return the Pydantic model instance through XCom instead of dumping it to a dict, on Airflow versions whose worker registers operator-declared output classes for deserialization. Downstream tasks should type-hint the model class (def downstream(result: MyModel)) and use attribute access (result.field) instead of subscript access. The output class must be defined at module scope and bound to an attribute matching its __name__; classes that are nested, dynamically built, or otherwise non-importable by qualname cannot be re-imported and will fail to deserialize at the consumer.

The worker walks the loaded DAG and registers each declared class before any task runs, so same-DAG downstream tasks (including mapped .expand(...) producers) deserialize the model without any configuration change. The UI XCom viewer renders the value via the stringify path and works without configuration (it shows module.MyModel@version=1(field=value,...) rather than a pretty form). Cross-DAG xcom_pull consumers still need the class qualified name added to [core] allowed_deserialization_classes – the consumer DAG’s worker only loads its own DAG. On Airflow versions whose worker does not register declared classes, the operators dump to dict instead.

0.4.0

Release Date: 2026-06-07

Note

This release changes the return type of LLMOperator, LLMAgentOperator and LLMFileAnalysisOperator: structured output is now returned through XCom as Pydantic model instances instead of plain dict objects. Downstream tasks that consume these XCom values must be updated accordingly. As this provider is still pre-1.0, the breaking change ships in a minor release.

Breaking changes

  • Return Pydantic model instances through XCom for structured output (#67644)

Features

  • Add 'OpenTelemetry' tracing for 'common.ai' Pydantic AI agents (#67792)

  • Add a bridge to expose 'common.ai' toolsets as LangChain tools (#67791)

  • Add Agent Skills support to the Common AI provider (#67786)

  • Accept Sequence[UserContent] in common.ai TaskFlow decorators (#67389)

  • Add LlamaIndex operators to common.ai provider (#67121)

  • Add 'DocumentLoaderOperator' to 'common.ai' provider (#67120)

  • Add 'Langchain' hook to 'common-ai' provider (#67192)

Bug Fixes

  • Register operator-declared XCom classes from a worker-side DAG walk (#67875)

  • common-ai: Honour serialize_output=True on LLMFileAnalysisOperator (#67858)

Misc

  • Bump common.ai floor to pydantic-ai-slim>=1.71.0 and document capabilities passthrough (#67444)

  • Remove further findings from positional session check (#67712)

  • Add prek hook to enforce HTTPException is imported from fastapi (#67367)

  • Add prek hook enforcing the "example" tag on example DAGs (#67354)

0.3.0

Release Date: 2026-05-23

Features

  • Add 'LLMRetryPolicy' to common-ai provider (#65451)

Bug Fixes

  • Update dependencies to fix dependabot alarms in providers.common.ai (#66628)

0.2.0

Release Date: 2026-05-11

Features

  • Add UsageLimits support to common.ai operators (#66248)

Doc-only

  • Add Configuration Reference docs page to Common AI provider (#66024)

0.1.1

Release Date: 2026-04-26

Misc

  • Update dependencies to address Dependabot security alarms in providers.common.ai (#65048)

  • Bump vite (#64799)

0.1.0

Release Date: 2026-04-13

Note

This release of provider is only available for Airflow 3.0+ as explained in the Apache Airflow providers support policy.

Was this entry helpful?