Skip to content

aws: add EC2/EBS/ENI discovery source with optional S3, Lambda, EKS, and Organizations support - #608

Merged
semx merged 1 commit into
developmentfrom
feature/aws-source
Oct 29, 2026
Merged

semx merged 1 commit into
developmentfrom
feature/aws-source

Conversation

@semx

@semx semx commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

Adds a read-only AWS source, supersedes #603 (stalled on a fork branch with a broken circular self-import and a never-committed AWSClient).

What this adds

  • EC2 instance / EBS volume / ENI discovery, synced into VM/Cluster/VirtualDisk objects with VPC-scoped VRF and IP import.
  • Explicit region list or regions = auto discovery.
  • Single account or AWS Organizations scanning (per-account role assumption, allowlist, external ID, management-account opt-in).
  • Optional S3, Lambda, and EKS inventory (off by default, opt-in per service).
  • Authorization-denial handling: a denied call for one account/region/service/resource is tolerated, logged, and reconciled into a Sync:AccessDenied:<Operation> NetBox tag, instead of aborting the whole run or pruning objects the source just couldn't see this time.
  • Name-collision handling: a conflicting NetBox object is skipped with a log line and its existing children are retained, rather than aborting the run.
  • docs/source_aws.md with required read-only IAM permissions, Organizations setup, and the authorization-denial contract.

Testing

  • tests/test_aws_source.py + tests/test_aws_services.py, ~190 cases against botocore.stub.Stubber (pagination, access-denied semantics, Organizations, S3/Lambda/EKS, name collisions) — no AWS credentials needed.
  • Full suite green on top of current development.
  • Verified read-only against a real AWS account (identity + EC2/S3/Lambda/EKS discovery, zero write calls, no credential/secret leakage in logs or NetBox data).

Closes #603.

…and Organizations support

Read-only boto3 discovery of EC2 instances, EBS volumes, and network
interfaces, with VPC-scoped VRF/IP import. Supports explicit or
auto-discovered regions, single-account or AWS Organizations scanning
via per-account role assumption, and optional S3, Lambda, and EKS
inventory. Authorization denials are tolerated per account/region/
service/resource and reconciled into NetBox tags so partial access
doesn't abort a run or prune objects the source couldn't see this time.
@semx
semx requested a review from bb-Ricardo as a code owner October 29, 2026 07:43
@semx
semx merged commit 22040b2 into development Oct 29, 2026
1 check passed
@semx semx mentioned this pull request Oct 29, 2026
@ekrichbaum

Copy link
Copy Markdown

This was a materially different set of client.py and connection.py items. there are some helpers and other items here but a lot of my protections and adjustments were removed from client.py. connection.py seems more limited with a catch around a ValueError, but again changed the collision logging and other elements. It will take some time to evaluate and compare items for the real world. It would have been preferable to note on the first PR those adjustments or needs prior to this point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants