Skip to content

Fix Amazon system tests creating connections on Airflow 2 - #73571

Open
vincbeck wants to merge 2 commits into
apache:mainfrom
aws-mwaa:vincbeck/af2_system_tests_connections
Open

vincbeck wants to merge 2 commits into
apache:mainfrom
aws-mwaa:vincbeck/af2_system_tests_connections

Conversation

@vincbeck

Copy link
Copy Markdown
Contributor

These Dags created their Airflow connection through the REST API, which only exists in Airflow 3, so on Airflow 2 the setup task kept retrying against a non-existent API server until it failed. The shared test helper already knows how to create a connection on both versions.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

These Dags created their Airflow connection through the REST API, which only
exists in Airflow 3, so on Airflow 2 the setup task kept retrying against a
non-existent API server until it failed. The shared test helper already knows
how to create a connection on both versions.
@vincbeck
vincbeck requested a review from o-nikolas as a code owner September 22, 2026 19:45
@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels Sep 22, 2026
@seanghaeli

Copy link
Copy Markdown
Contributor

Looks good, just noticed this pattern is also present in example_google_api_youtube_to_s3:

make_authenticated_rest_api_request(
path="/api/v2/connections",
method="POST",
body={
"connection_id": conn_id_name,
"conn_type": "google_cloud_platform",
"extra": json.dumps(conn_extra),
},
)

I know we don't run this test on our dashboard but it might be good to get rid of the pattern entirely from the codebase since it's outdated

The Dag created its Airflow connection through the REST API, which only exists
in Airflow 3, so on Airflow 2 the setup task kept retrying against a
non-existent API server until it failed. This was the last remaining occurrence
of that pattern in the codebase.
@vincbeck

Copy link
Copy Markdown
Contributor Author

Looks good, just noticed this pattern is also present in example_google_api_youtube_to_s3:

make_authenticated_rest_api_request(
path="/api/v2/connections",
method="POST",
body={
"connection_id": conn_id_name,
"conn_type": "google_cloud_platform",
"extra": json.dumps(conn_extra),
},
)

I know we don't run this test on our dashboard but it might be good to get rid of the pattern entirely from the codebase since it's outdated

I agree, it does not hurt

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants