Skip to content

Source: Aws - #603

Closed
ekrichbaum wants to merge 28 commits into
bb-Ricardo:mainfrom
ekrichbaum:AWS
Closed

ekrichbaum wants to merge 28 commits into
bb-Ricardo:mainfrom
ekrichbaum:AWS

Conversation

@ekrichbaum

Copy link
Copy Markdown

Initial working Amazon AWS source integration. Run against large install base across multiple subs in the org. Next phase of this will include EKS and other elements but for now virtual machines are fully functional.

ekrichbaum and others added 13 commits October 24, 2026 10:09
Include optional SDK integration, cross-account role assumption, regression tests, and anonymized setup examples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Added account_name configuration for friendly labels in cluster names and updated documentation on instance naming and account handling.
Refactor error handling for VRF creation and add mapping check.
Clarify VRF creation and mapping for VPCs in NetBox integration.
Added resolve_shared_networks method to handle VPC and subnet ownership conflicts. Updated addresses method to include IPv6 networks and modified VPC ownership retrieval logic.
Comment out optional AWS service sync settings in the configuration file.
Updated AWS source description to include S3, Lambda, and EKS.
Refactor AWS source tests to improve tag handling and error management. Update test cases to ensure proper handling of tags and permissions.
Added validation for AWS services and enhanced error handling for S3, Lambda, and EKS resources. Updated methods for syncing AWS resources with NetBox, including improved handling of tags and resource identification.
Added context manager to handle AWS access denial gracefully and updated discovery methods to record denied scopes.
Updated AWS source documentation to reflect optional service discovery for S3, Lambda, and EKS. Clarified permissions and added sections on authorization-denied discovery and troubleshooting.
Expand documentation on authorization failures and resource conflicts in AWS source.
Added AWSAccessCheck dataclass to track access checks and modified AWSClient to record access checks during AWS service calls.
Refactor AWS service tests to improve clarity and functionality. Added new tests for handling conflicts and access tracking.
Added access_scope and reconcile_access_tags methods to manage AWS resource access and tagging. Updated retain_inaccessible_objects to call retain_source_objects.
@semx

semx commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

Superseded by #608, which has been merged into development.

The branch here had module/sources/aws/client.py left as a near-duplicate of connection.py (both defining AWSHandler), with a circular self-import and no actual AWSClient (boto3 discovery layer) ever committed, so this couldn't run. #608 adds a complete, tested AWSClient implementation plus a few related fixes in connection.py (reconcile_access_tags/access_scope were missing, name-collision handling aborted the whole sync instead of skipping, an EKS placement fallback bug, and a stale access_denied/access_checks accumulation bug across repeated syncs).

Closing in favor of #608 — thanks for getting this started!

@semx semx closed this Oct 29, 2026
@ekrichbaum

Copy link
Copy Markdown
Author

I had fixed the bad paste already into that separated branch, but at least it got pulled. almost impossible to maintain this many individual branches that didn't get pulled in for a considerable amount of time.

@ekrichbaum
ekrichbaum deleted the AWS branch October 31, 2026 15:52
semx added a commit that referenced this pull request Oct 31, 2026
A lost connection, a timeout, missing credentials or an unknown profile
raised a bare botocore exception out of discover(), discover_organization()
or the session setup, although the module promises AWSDiscoveryError for
every failure that is not an authorization denial. Wrap them the way the
client in #603 did and add tests for all three.

Co-authored-by: Eric Krichbaum <11653069+ekrichbaum@users.noreply.github.com>
semx added a commit that referenced this pull request Oct 31, 2026
aws: retries, timeouts and SDK errors, taken over from the #603 client
ekrichbaum added a commit to ekrichbaum/netbox-sync that referenced this pull request Oct 31, 2026
The clients used the botocore defaults. Use the standard retry mode with
5 attempts and a connect timeout of 10 seconds, the way the client in bb-Ricardo#603
did, and add a test that fails when a client is created without it.

Co-authored-by: Eric Krichbaum <11653069+ekrichbaum@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants