Skip to content

Add Openstack Source - #600

Open
ekrichbaum wants to merge 14 commits into
bb-Ricardo:mainfrom
ekrichbaum:Openstack
Open

ekrichbaum wants to merge 14 commits into
bb-Ricardo:mainfrom
ekrichbaum:Openstack

Conversation

@ekrichbaum

Copy link
Copy Markdown

Previous pull wasn't added before changes and Hetzner source. Rebased and separated Ovirt and Openstack into separate pulls although common elements exist.

Added optional OpenStack SDK installation instructions and configuration details.
This document provides comprehensive details on the OpenStack source integration, including installation, authentication, configuration, object mapping, and validation steps for syncing with NetBox.
Added link to OpenStack source documentation for clarity.

@semx semx left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested against a DevStack all-in-one (stable/2026.2, openstacksdk 4.22.0) syncing into NetBox 4.7.3. Test servers covered: same name in two projects, two networks, floating IP, attached volume, boot-from-volume, and SHUTOFF.

1. No VMs are created when the compute host also runs nova-conductor/scheduler

GET /os-availability-zone/detail lists the host in two zones:

  • internal: nova-scheduler, nova-conductor
  • nova: nova-compute

add_cluster records every host of every zone (L837-842). get_cluster_for_host then returns the first match (L474-478), which is internal. As a result, the hypervisor device lands in cluster internal, while the VMs go to nova with device set to that hypervisor. NetBox rejects every VM:

POST /api/virtualization/virtual-machines/ Bad Request
{'device': ['The selected device (nbtest-devstack) is not assigned to this cluster (nova).']}

The interface and virtual-disk POSTs fail afterwards ('virtual_machine': ['This field is required.']).

  • Result: 0 VMs in NetBox, the IPs are created unassigned, and the process still exits 0.
  • With cluster_exclude_filter = internal, all VMs sync.
  • I did not test a multi-node cloud, where compute nodes are not members of internal.

2. Servers with the same name in different projects

With all_projects = True, the second server is skipped:

WARNING: Virtual machine 'nbA-dup' for cluster 'nova' already parsed. Make sure to use unique VM names. Skipping

The duplicate check only uses AZ + name (L1057-1066). After stopping the server that won, the next run rewrote the same NetBox VM with the other server's data:

Virtual machine 'nbA-dup' attribute 'comments' changed from '96804584-…' to 'b93ceea0-…'
Virtual machine interface 'vNIC1 (private) (nbA-dup)' attribute 'primary_mac_address' changed from 'FA:16:3E:CA:51:3D' to 'FA:16:3E:D7:0C:C3'

3. Interface names depend on position

Names are vNIC{count} ({network}) with one counter across all networks (L1157-1175). After removing the first network from a server, the next run renamed the remaining interface:

Updating NetBox 'virtual machine interface' object 'vNIC1 (nbA-net2) (nbA-multi)' with data: {'name': 'vNIC1 (nbA-net2)', ...}

Its previous name was vNIC2 (nbA-net2).

4. The hypervisor interface is synthetic and overwrites the existing type

The hypervisor always gets eth0 with type virtual (L971-977). I changed that interface to 1000base-t in NetBox, and the next run reset it:

Interface 'eth0 (nbtest-devstack)' attribute 'type' changed from '1000base-t' to 'virtual'

5. Primary IPv4 is the last fixed IP

Every fixed address replaces the previous candidate (L1192-1197). The server on private (10.0.0.12) and nbA-net2 (10.77.0.154) got primary_ip4 = 10.77.0.154/32.

6. docs/source_openstack.md vs code

  • L21-24 documents --build-arg INSTALL_OPENSTACK=true, but the Dockerfile in this PR has no such argument.
  • L115 and L181 refer to an "AWS implementation", but there is no AWS source in the repo.
  • L165-166 says disk sizes are GiB before 4.1 and MiB from 4.1. The code multiplies GB by 1000 (L1210): a 2 GB volume is stored as 2000.

7. Removed-settings block copied from vmware

sync_tags = True produces:

Setting 'sync_tags' has been removed ... You need to switch to 'host_tag_source', 'vm_tag_source' or 'cluster_tag_source'

None of those options exist in this source.

8. Smaller points

  • Hypervisor platform: it is created as QEMU 8002002, i.e. the raw hypervisor_version.
  • Base branch: the PR targets main; other changes go to development.
  • Conflicts: it conflicts with #599 in README.md, file_output.py, sources/__init__.py, pyproject.toml, settings-example.ini and uv.lock.
  • Duplicated method: it carries its own object_synced_by_other_source, while #599 moves the same method into SourceBase.
  • Version range: openstacksdk>=4.0,<5 is a range, while requirements.txt pins exact versions.
  • Commit message: commit fe0fe8f is titled "Change greeting from 'Hello World' to 'Goodbye World'" but only changes uv.lock.

What worked

  • Lockfile: every uv.lock entry added by the PR matches the PyPI sha256.
  • Test suite: pytest merged into development: 392 passed, 8 skipped (vcsim tests included).
  • Errors: a wrong password or unreachable Keystone exits 1 with pruning disabled.
  • Idempotency: a rerun without changes writes nothing.
  • Disks and floating IPs: root/volume/boot-from-volume disks are correct, and floating IPs land on the right interface with the fixed IP as primary.
  • Deleted servers: a deleted server is orphaned as expected.

@ekrichbaum

Copy link
Copy Markdown
Author

Also used continuously in production with a large amount of vms.

image

I also detect internal but beyond having the cluster, the vms are detected in the correct one. 7 is probably due to trying to juggle all of these as separate projects while keeping a copy of all of them together for my actual use. I'll add these notes and see what I come up with.

@ekrichbaum

Copy link
Copy Markdown
Author

I'll revamp and test tomorrow. I see really 5 adjustments that happen.

  • AZ host mapping needs to prefer nova-compute membership although I don't have that sort of a setup. I'll build an instance for that edge case.
  • Dupe VM names to retain the server UUID when power state changes. and a config option.
  • Existing interface names retained when a NIC is matched by MAC (this one is interesting, see VMWare discussion post and PR from another user).
  • Primary IP selection prefer first fixed address instead of last. This seems just as arbitrary as there isn't a way to truly guage intent, but it's as simple as the other.
  • Update the docs and sync_tags guidance. Clean out the AWS branch references.

Once I test out these changes tomorrow I can update the 4 files again. Then I need to get back to the AWS and Digital Ocean sources. I've gotten an internal request for GCP as well but not sure I will go there. It's less than 1% of our 10k+ vms so may not be worth the effort unless there is more demand.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants