Ferruzzi/multi team/bundle team cache#68727
Closed
ferruzzi wants to merge 33 commits into
Closed
Conversation
Make explicit on both the authoring and review sides that direct AirflowException raises are being reduced, not added. New direct usages are disallowed (already enforced by the check-no-new-airflow-exceptions prek hook), and the reviewer guidance clarifies that the only acceptable movement of an AirflowException line is relocating an already-existing one verbatim during a refactor, which is not a new usage. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…pache#68653) The skill classified provider changes by spawning one Explore sub-agent per PR. On a normal release wave (dozens of providers, hundreds of commits) that is hundreds of sub-agents — the main reason the run is so long and token-heavy. Replace the per-PR fan-out with a volume-aware strategy: classify inline when few commits remain (≲15), otherwise spawn one sub-agent per provider that classifies that provider's whole remaining commit list in a single pass (chunking only providers with >25 commits). Multi-provider PRs are already classified independently per provider, so the provider is the natural batch unit and the breaking-change checklist is amortized across its commits instead of paid per commit. This takes the sub-agent count from O(commits) down to roughly O(providers) — often zero for incremental updates.
* Fix insert_rows executemany=True: require clickhouse-connect>=1.3.0 * Regenerate lock and provider deps for clickhouse-connect 1.3.0 # Conflicts: # generated/provider_dependencies.json.sha256sum * Remove not nedeed override for exclude-newer-package for clickhouse-connect * update uv.lock * delete redundant tests --------- Co-authored-by: Jose Muñoz <jmunozkr@gmail.com> Co-authored-by: Jose Muñoz <jose.munoz@clickhouse.com>
…d connections (apache#68645) A bulk 'create' with action_on_existence=overwrite dumped the whole request body and set every field on the existing record, so fields the request omitted were reset to their defaults. For multi-team setups this silently nulled an existing pool's or connection's team_name ownership when an overwrite changed only e.g. slots; description and include_deferred were affected too. Only write the fields the request actually provided (model_dump(exclude_unset=True)), so omitted fields keep their current value while explicitly-set fields (even None) are still applied. Adds regression tests for both pools and connections.
The scheduler's _executable_task_instances_to_queued bypassed pool team ownership checks, allowing tasks from one team to use pools owned by another team. The existing PoolSlotsAvailableDep had the correct logic but was never invoked in the scheduler's critical section. Add an inline team check that batch-resolves pool-to-team mappings and blocks cross-team pool access before tasks are queued for execution.
* fix: add missing queued state color to CalendarToolTip (May 15) * fix: calendar cell coloring to prioritize severity in Total Runs view * refactor: clarify calendar cell color keys and extract priority resolution logic * test: add missing test cases for calendar scale total runs * fix: expand calendar legend to clarify running and failed states
Bumps the fab-ui-package-updates group with 1 update in the /providers/fab/src/airflow/providers/fab/www directory: [eslint](https://github.com/eslint/eslint). Updates `eslint` from 10.4.1 to 10.5.0 - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](eslint/eslint@v10.4.1...v10.5.0) --- updated-dependencies: - dependency-name: eslint dependency-version: 10.5.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: fab-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ge classification (apache#68215)
* feat: Add code mode (Monty sandbox) to common.ai AgentOperator * docs: Clarify when to use common.ai code mode * Report the real import error when code mode's harness import fails
apache#68372) durable=True cached model responses and tool results under purely positional keys, so a retry replayed cached steps even when the agent changed between attempts (prompt tweak, model upgrade, toolset change, or a deploy landing between retries). The retry silently continued a different conversation with no warning above DEBUG. Each cache entry now stores a fingerprint of the request that produced it (model identity, message history minus per-attempt fields, settings, and the full ModelRequestParameters; tool name, args, and tool_call_id for tool steps). On a hit the fingerprint is compared first: a mismatch logs a warning and re-runs the step live. A divergence invalidates downstream tool steps too, because a fresh model response mints new tool_call_ids. Entries written by older provider versions have no fingerprint and re-run instead of replaying.
* Fix the gap of Taiwanese Mandarin translation * fixup! Fix the gap of Taiwanese Mandarin translation
* Fix Databricks operators with templated json payloads * Address review on templated Databricks json payloads Do not overwrite the json template field during execute() in the Create/Submit/RunNow operators, so retries and deferral resumes re-render templated payloads instead of reusing a stale rendered dict. RunNow keeps the merged payload on a transient _merged_json and execute_complete() rebuilds it from the template fields, which also recovers a job_parameters passed as the named argument across a resume. SubmitRun works on a deep copy so per-task params injection no longer mutates the named template fields in place. Validate payload types before any Databricks API call in RunNow and SubmitRun so an invalid payload fails fast with no remote side-effects. Document the dict-literal json fallback and the execution-time validation timing in the changelog, docstrings, and operator params. --------- Co-authored-by: deepinsight coder <nalamvamsi13@gmail.com>
…l in KiotaRequestAdapterHook (apache#68375) * refactor: Use async versions of CertificateCredential and ClientSecretCredential to avoid blocking the event loop, especially when used concurrently * refactor: Fix stale cached request adapter causing "HTTP transport has already been closed" errors * refactor: Invalidate cached request adapters which have closed session in KiotaRequestAdapterHook
…r SmtpHook (apache#62409) * Fixed the bug, added a test * Fixed the other 3 retry loops and updated tests * Made the documentation consistent * Made retry_limit consistent across all use of SMTP * Fixed line length * Updated logging, made loops better
…) fails (apache#68542) It was being logged, but no error was returned to the caller.
Fix periodic 500 due to idle_in_transaction timeout in FAB auth manager
TaskInstance.register_asset_changes_in_db runs on every task-success state transition. For a task that declares no outlet assets and emits no outlet events -- the common case -- the method still executed its body and issued an AssetModel SELECT with empty IN () clauses before doing nothing useful. Return early when both task_outlets and outlet_events are empty, avoiding that round-trip on the hot path that gates scheduling the next task. Signed-off-by: Daniel Standish <daniel.standish@astronomer.io> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ferruzzi
requested review from
XD-DENG,
bbovenzi,
guan404ming,
pierrejeambrun,
rawwar,
ryanahamilton and
shubhamraj-git
as code owners
June 18, 2026 19:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to request by @o-nikolas in #68599 to add caching like we did in the scheduler.
Adding a
_bundle_name_to_team_namemap/cache and a_multi_teamflag (read once at construction, like the scheduler) onDagFileProcessorManageralso allowed me to move theif multi-teamcheck into the_get_team_names()and_get_team_name()helpers, simplifying the calling code.The cache/map is cleared inside the
_refresh_dag_bundles()method, which is the only time we can pick up new mappings anyway.Was generative AI tooling used to co-author this PR?
Generated-by: [Claude Opus 4.8] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.