Metadata-Version: 2.4
Name: netbox-device-importer
Version: 0.2.0
Summary: NetBox plugin: SSH-scrape network gear and import devices/interfaces/VLANs/IPs into NetBox.
Author: Dries Velle
License-Expression: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: paramiko>=3.0
Provides-Extra: branching
Requires-Dist: netboxlabs-netbox-branching<2,>=1.1; extra == "branching"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-django>=4.8; extra == "dev"
Dynamic: license-file

# NetBox Device Importer — User Guide

Import network devices into NetBox by SSH-scraping them (switches / firewalls) or
from a CSV of manual devices. Runs from the **Device Import & Sync** menu and writes
Devices, Interfaces, VLANs, IPs and Prefixes for you.

> Deploying, upgrading or maintaining the plugin? See `DEPLOYMENT.md` in the source
> distribution — it covers install, upgrade, and the environment requirements,
> assumptions and limitations. This page is for using it.
>
> **What changed in this version?** The changelog travels with the plugin — it is
> appended below this guide on the package page, and in
> `pip show netbox-device-importer`.

---

## Before you import — set these up in NetBox first

The importer **matches on existing NetBox objects** and will **not** create the
following. Create them before importing, and match names **exactly** (case-sensitive):

- **Tenant**, **Site**, **Device Role**, **Device Type** must already exist.
- **Device Type "Model" must equal the model the device reports.** For FortiGate the
  reported model is normalised (e.g. `FortiGate-100F` → `FG100F`); the Device Type
  model in NetBox must match the normalised value.

**Device Type interfaces no longer need to be pre-created.** The importer reads the
real port list off the device and makes NetBox match it, per device:

- Ports the device has but NetBox does not are **created**, named exactly as the device
  names them.
- A port recorded under a different spelling of the same name (`Gi1/0/1` where the
  device says `GigabitEthernet1/0/1`) is **renamed**, not duplicated.
- An interface NetBox has that the device does not report is **left alone and flagged**,
  never deleted — deleting one would take its cables, IP assignments and VLAN
  memberships with it. The one exception is a device the importer created during the
  same run, where nothing can depend on an interface yet.

Every one of those changes is logged as a warning and counted on the device's card in
**Logs**, so a committed import can be reviewed in its branch before you merge it.

Interface **type** is set from the device where it says so, and left as `other` where it
does not. Notably, HCoS copper ports and all FortiGate ports report no usable media
type, and an empty SFP cage reports none either. A type that is already set correctly in
NetBox is never overwritten with a guess.

The importer **does** create automatically: **Locations**, **Racks** (default height
42U), **VLANs** (named `AutoCreatedFromImportScript` unless the device supplies a
name), **Prefixes**, **IP addresses**, and **device interfaces** (above). It never
creates or modifies a **Device Type** or its interface templates.

> Keep whatever interface templates you like on a Device Type for adding devices by
> hand — the importer reconciles past them and leaves them alone. This matters for
> **modular switches**, where two units of the same model can have different slots
> populated: the port list belongs to the device, not the model.

---

## Supported platforms

| Platform value | Devices |
|---|---|
| `hios` | Hirschmann HiOS switches |
| `hcos` | Hirschmann HiSecOS (HCoS) switches |
| `ios` | Cisco IOS switches |
| `smbos` | Cisco SMB OS switches |
| `fortios` | Fortinet FortiOS firewalls |

Stacked Cisco switches (`ios`, `smbos`) are detected and **skipped** (not supported).

---

## Using it

Open **Device Import & Sync** in the left nav. Two entry points:

### Single Device (SSH)

Import one SSH-reachable device. You fill in:

- **Device IP, SSH port, username, password**
- **Platform** (dropdown)
- **Placement** via NetBox pickers — **Tenant, Site, Location, Rack, Device Role**.
  These chain: pick a Site and the Location list narrows to it, pick a Location and
  the Rack list narrows again. You can only pick things that already exist, so
  placement can't be mistyped.
- Optional: rack height (only if the rack must be created), position, asset tag.

Name, serial, model, interfaces and VLANs are read from the device — you do **not**
enter them. (The scraped model must already exist as a Device Type, per above.)

### Multi Import (CSV)

Pick the CSV format, upload the file, submit. Two formats:

**SSH devices — `Sources.csv`** (the importer logs in and scrapes each device):

```
DeviceIP,sshPort,username,password,tenant,site,location,rack,rackHeightInU,positionInRack,deviceRole,platform,AssetTag-RDSPP
```

`platform` must be one of `hios`, `hcos`, `ios`, `fortios`, `smbos`.

**Static devices — `Devices.csv`** (no SSH; you provide everything):

```
Device-IP,netmask,VLAN-IDs,MAC-address,Device-Name,Device-Type,Serialnumber,Firmware,tenant,site,location,rack,rackHeightInU,positionInRack,deviceRole,AssetTag-RDSPP
```

- **`VLAN-IDs`** is a `-`-delimited **list** (not a range), e.g. `0-99-105`. The
  **first** entry is the **untagged** VLAN (`0` = none); the rest are **tagged**.
- Interface for a static device is the single Device-Type interface (`1/1`, or
  `External-NIC` for the `5PX 3000i RT3U G2` UPS).

**CSV rules (both formats):**

- A **header row is required** and must match the columns above.
- **No trailing blank lines.**
- All names are **case-sensitive** and must match existing NetBox objects.

---

## Every committed import goes into its own branch

You do **not** need to snapshot NetBox before an import. Each committed run creates its
own branch — named `import-<your user>-<date>-<time>-job<number>` — and imports into that.
Main is untouched until you say so.

The workflow:

1. Run the import. The job page and the **Logs** page both name the branch it used.
2. Open the branch in NetBox and check the devices, interfaces, VLANs and IPs.
3. **Happy with it?** Merge the branch. The changes land in main.
4. **Not happy?** Delete the branch. Nothing to undo — main never saw it.

Two things to keep in mind:

- **Merge or delete before the next import.** A new branch is taken from main, so it will
  not contain a previous, unmerged import. If you start a second import while one is still
  open, you will be warned and asked to confirm — merging both afterwards can conflict.
- **If branching is unavailable** (plugin disabled, branch limit reached), the import is
  stopped and you are told why. You can cancel, or confirm that you have taken a VM
  snapshot and import directly into main — that is the old procedure, and the only case
  where a snapshot still matters.

Dry runs never create a branch; they write nothing in the first place.

---

## Dry run — always test first

Every form has a **Commit to NetBox** checkbox. **Leave it unchecked** for a dry run:
the device is scraped, parsed, and every intended change is written to the job log,
then **rolled back** — nothing is saved. Check it only when the dry-run log looks
correct.

---

## Watching the result

Submitting queues a background job and takes you to its page. There you see live
progress and a log line for each step (prerequisites OK, VLAN OK, Mgmt IP OK,
interfaces completed, and any warnings/errors). Re-open the job any time from
**Jobs** in NetBox.

---

## Credentials

SSH credentials you provide (in the CSV or the single-device form) are used **only for
that run** and are **never stored** in NetBox — they do not appear on the job record.

---

## Common messages

| Message | Meaning / fix |
|---|---|
| `Site does not exist: X` | Create the Site in NetBox (exact, case-sensitive name), or fix the CSV value. |
| `Tenant does not exist` / `Device-Role does not exist` | Same — create it first or correct the name. |
| `Device-Type does not exist: '<model>'` | Create a Device Type whose Model field is exactly that model (FortiGate uses the normalised name). Its interfaces do not need creating. |
| `Could not determine the device model` | The scrape or parser failed to read a model, so no Device Type was looked up. Check the SSH capture and the device's `platform` value. |
| `Interface created on X` / `Interface renamed on X` | Normal: NetBox is being brought in line with the device. Review them in the import branch before merging. |
| `Interface on X not found on the device, left in place` | NetBox has an interface the device did not report. Nothing was deleted — check whether it is stale and remove it by hand if so. |
| `Interface deleted from X` | Only happens on a device this run created, where the interface came from a Device Type template and nothing could depend on it yet. |
| `Ambiguous interface names on X` | Two NetBox interfaces differ only in spelling (e.g. `Gi1/0/1` and `GigabitEthernet1/0/1`). Neither was renamed — resolve the duplicate by hand. |
| `Interface missing after reconciliation` | A bug, not a setup problem: please report it with the debug log. |
| `SSH connection failed` | Check reachability/credentials/port; the NetBox worker must be able to reach the device. |
| `Stacked switch - not processed` | Stacked Cisco switches are intentionally skipped. |
| `Firewall interface config incomplete` | The FortiGate scrape returned too few interfaces (transient SSH/parse issue) — retry. |

---


# Changelog

All notable changes to **netbox-device-importer** are documented here.

Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versioning
follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html) as defined in the
project deployment guide:

| Bump | When |
|---|---|
| **PATCH** (0.1.0 → 0.1.1) | Bug fix, parser tweak, doc change. No behaviour change to a correct import. |
| **MINOR** (0.1.0 → 0.2.0) | New capability, backwards-compatible: a new platform, a new form field, a new setting. |
| **MAJOR** (0.x → 1.0.0) | Breaking change: removing a platform, changing CSV columns, or a NetBox-version jump that changes behaviour. |

NetBox compatibility is **not** encoded in the version string — it is advertised via
`min_version` / `max_version` in `PluginConfig`. Each release below states the range it
was verified against.

This file is concatenated onto the user guide in the wheel's package metadata, so the
changelog for the exact installed version travels with the plugin
(`pip show netbox-device-importer`, or the project page on the index).

---

## [Unreleased]

---

## [0.2.0] — 2026-10-12

Verified against **NetBox 4.6.3** and **4.7.2**; declared compatible with
**4.2.0 – 4.7.99**.

### Changed

- **NetBox 4.7 support.** `max_version` raised from 4.6.99 to 4.7.99; declared
  compatible with **4.2.0 – 4.7.99**, verified against **4.6.3** and **4.7.2** (Django
  6.1). No code changes were needed. Before this, NetBox 4.7 skipped the plugin at
  startup and the nav menu was missing. With branching on NetBox 4.7, use
  **netboxlabs-netbox-branching 1.2.x** (verified 1.2.2); 1.1.x supports NetBox 4.6 at most.

- **BREAKING for existing Cisco IOS devices: interface names are no longer abbreviated.**
  The plugin now uses the name from the running config verbatim, so `ios` interfaces are
  `GigabitEthernet1/0/1` instead of `Gi1/0/1`. Existing IOS devices are **renamed in
  place** on their next import (the interface row is kept, so cables, IPs and VLAN
  memberships survive). Review the rename list in the import branch before merging.
- **Device Type interfaces are no longer a prerequisite.** Creating a Device Type is
  still required; pre-creating its interfaces is not. The plugin reads the real port list
  off the device and reconciles the *device's* interfaces against it. It never creates,
  renames or deletes a DeviceType or its interface templates.
- Interfaces NetBox is missing are **created**, and ones recorded under a different
  spelling of the same name are **renamed** rather than duplicated. Previously an
  unmatched interface was skipped with a warning, silently discarding that port's VLAN
  and IP data.
- A surplus interface is **logged and kept**, never deleted — deleting one cascades to
  its cables, IP assignments and VLAN memberships. The single exception is a device the
  plugin created in the same run, where nothing can depend on an interface yet.
- **A trunk with no allowed-VLAN list is now written as `tagged-all`.** On IOS,
  `switchport mode trunk` without `switchport trunk allowed vlan` permits every VLAN;
  previously such a port was stored as `access` (or `tagged` with an empty list), both of
  which claim the trunk carries only its native VLAN. A native VLAN is kept as the
  interface's untagged VLAN alongside `tagged-all`, which is how NetBox models it.
- Interface **type** is now set from the device where it reports one, and left as `other`
  where it does not. A type already set in NetBox is never overwritten with a guess, so
  an empty SFP cage cannot downgrade a correct entry.
- `Device-Type does not exist` now quotes the exact model and says which field to match.
  A device whose model could not be parsed reports a scrape failure instead, rather than
  sending the user off to create a Device Type called `Unknown`.
- A device that fails on a missing prerequisite now shows the reason on its summary card
  instead of `Imported successfully` next to a red icon.
- Per-device interface counts (created / renamed / deleted / unexpected) now appear on
  the device card in **Logs**.

- **Event rules, webhooks and notifications now fire for imported objects.** They are a
  side effect of the change-logging context above: imports are now ordinary, auditable
  changes as far as NetBox is concerned. Instances with webhooks or event rules bound to
  Device / Interface / VLAN / IP-address will start receiving them for import runs.
- **Dry runs remain completely silent** — `event_tracking` is skipped when "Commit to
  NetBox" is unchecked (mirroring NetBox's own `ScriptJob`), so a rolled-back run writes
  no changelog entries and fires no webhooks for changes that never landed.
- A run with no user attached to the job is left as before: no changelog entries, no
  events.

### Added

- **Physical port enumeration per platform**, so unconfigured ports are imported too:
  `show interfaces status` (`ios`, `smbos`), `show port` (`hios`), `show port all`
  (`hcos`), `get system interface physical` (`fortios`).
- `ios` now imports **FastEthernet and TenGigabitEthernet** ports. Only
  `interface GigabitEthernet` was matched before, so on a 2960C twelve of fourteen
  physical ports were invisible, and on a 9200L the four 10G uplinks were.

- **Every committed import now runs inside its own branch** (requires the
  [netbox-branching](https://netboxlabs.com/docs/branching/) plugin). The branch is named
  `import-<user>-<YYYY-MM-DD>-<HHMM>-job<pk>`, owned by the user who started the run, and
  named in the job log and on the Logs page. Nothing reaches main until the user reviews
  the branch and merges it; a bad run is deleted instead of unwound, which replaces the
  pre-import VM snapshot procedure. Dry runs create no branch — they already write nothing.
  All netbox-branching use is confined to `netbox_device_importer/branching.py`, imported
  lazily, so the plugin still installs and runs where branching is absent.
- **Submit-time branch check with explicit consent.** If a branch cannot be created —
  plugin missing or disabled, API changed, branch limit reached — the import is **not**
  enqueued. The user is shown the reason and must confirm they have taken a VM snapshot
  before an unbranched import into main proceeds; cancelling changes nothing. Who
  confirmed, and why a branch was unavailable, is recorded in the job log. If an earlier
  import branch is still unmerged, the user is warned that the new branch will not contain
  it and that merging both may conflict, and must confirm before continuing.
- **The plugin version is shown on every plugin page** — a small muted line under the
  card ("Device Import & Sync v*X.Y.Z*") on Single Device, Multi Import, Logs and the
  log detail page. Deployments pin an exact version, so this tells you which build an
  instance is running without shell access.

### Fixed

- **Paginated output was truncated after the first page.** `send_large_command` cleared
  `--More--` exactly once, so anything past the first page break was silently lost —
  `show port all` on a 20-port MACH already exceeds one page. It now pages until the
  device stops asking, with a cap that logs if it is hit.
- **Cisco IOS transceiver detection never matched.** `GetSpf()` looked ports up by their
  unabbreviated suffix while the port dict was keyed by the abbreviated name, so
  `show inventory` data was discarded. `show inventory` also names ports short on some
  IOS releases and long on others; both are now matched.
- **VLANs from a logical interface block could be attributed to a physical port.** In the
  `ios` parser a `interface Port-channel14` block appearing before the physical
  interfaces had its `switchport` lines flushed into the first physical port parsed.
- **The management netmask could be taken from the wrong interface.** The management IP
  was matched with `startswith()` and no boundary, so on a device that also configures a
  longer address sharing the same leading characters (`10.0.0.6` vs `10.0.0.69`) the
  first-listed one won. On `ios` that produced an unusable address string and failed the
  device; on `hios`/`hcos` the stray leftover digit was accepted as an already-CIDR value
  and **silently wrote `/9` instead of `/24`**. All five match sites now require the
  address to end where it should, and ignore a trailing `secondary` keyword.
- **The `ios` management VLAN was misread whenever the SVI carried a description.**
  `GetMgmtVlan()` looked only at the line before the management IP, so
  `interface Vlan89` / ` description Mgmt` / ` ip address …` yielded
  `" description Mgmt"` as the VLAN id and created an interface literally named
  `VLAN  description Mgmt`. It now tracks the SVI block it is inside, reports
  `untagged` when the management IP sits on a routed port, and no longer matches a
  longer address sharing the same prefix (`10.0.0.6` vs `10.0.0.69`).

- **Imports are now recorded in the NetBox changelog.** Previously the job ran in the RQ
  worker with no request context, so NetBox's change-logging signals bailed out and
  plugin writes produced no `ObjectChange` rows at all — nothing on any object's
  Changelog tab, nothing in Operations → Changelog. Every object the import creates or
  updates is now change-logged and attributed to the user who started the run
  (`netbox_device_importer/change_logging.py`, applied around the device loop in
  `jobs.py`).

- **Transactions now target the database alias writes actually go to.** Per-device
  `transaction.atomic()` was bound to `default`. With a branch active, netbox-branching
  routes writes to a per-branch connection alias, so that transaction would have wrapped
  nothing — dry-run rollback and per-device failure isolation would both have failed
  silently inside a branch. The alias is now resolved at runtime via the router, with a
  nested transaction on `default` when the two differ.
- **Coexistence with other plugins that register a request processor.** The synthetic
  request the import runs under now carries the full attribute set NetBox's own
  `copy_safe_request` produces (`path`, `path_info`, `method`, `GET`, `POST`, `COOKIES`,
  `FILES`, `META`, `headers`), not just `user`/`id`. Every registered request processor
  receives this object, so a minimal one made other plugins raise `AttributeError` and
  fail the entire import — reproduced with **netbox-branching**, whose
  `ActiveBranchContextManager` reads `request.path`.

---

## [0.1.1] — 2026-07-23

Verified against **NetBox 4.6.3**; declared compatible with **4.2.0 – 4.6.99**.

First packaged release of the plugin. It replaces the standalone `pynetbox`-over-REST
script: the per-vendor parsers were carried over unchanged, and every NetBox write was
rewritten against the Django ORM.

### Added

- **Device Import & Sync** nav menu with two entry points:
  - **Single Device (SSH)** — one device, with chained Tenant → Site → Location → Rack
    pickers so placement cannot be mistyped.
  - **Multi Import (CSV)** — `Sources.csv` (SSH devices) and `Devices.csv` (static
    devices, no SSH).
- SSH scraping and parsing for five platforms: `hios` (Hirschmann HiOS), `hcos`
  (Hirschmann HiSecOS), `ios` (Cisco IOS), `smbos` (Cisco SMB OS), `fortios`
  (Fortinet FortiOS), plus CSV-only static devices (e.g. UPS units).
- All NetBox writes go through the Django ORM via `NetBoxWriter`, with `full_clean()`
  before every `save()`. Creates Locations, Racks, VLANs, Prefixes and IP addresses;
  matches (and never creates) Tenant, Site, Device Role and Device Type.
- **Dry run** — the "Commit to NetBox" checkbox is off by default; every device is
  processed inside its own transaction and rolled back, so the full intended change set
  is visible in the log without touching data.
- Per-device transaction isolation: one device failing rolls back only that device and
  the run continues.
- **Three-way logging** — live progress on the core Job page, a human-readable
  `import-job-<pk>.user.log`, and a full `import-job-<pk>.debug.log`, plus a structured
  `summary.json` driving a **Logs** page with collapsible per-device status cards
  (ok / warning / failed). Log sets rotate, newest `log_max_sets` kept.
- Credential safety: SSH credentials are stashed in the Django cache under a random
  short-TTL key, the job pops and deletes the key immediately, and the credentials never
  enter persisted job arguments. A redaction filter scrubs them from every log
  destination and fails closed.
- Plugin settings: `prefix_with_site`, `cred_cache_ttl`, `default_rack_height`,
  `log_dir`, `log_max_sets`.
- Example CSV templates shipped in `netbox_device_importer/examples/`.
- User guide (`README.md`, embedded in the wheel metadata) and a maintainer deployment
  guide.

### Known limitations

- **Plugin writes produce no NetBox changelog entries.** The import runs in the RQ
  worker with no request context, so NetBox's change-logging signals bail out and no
  `ObjectChange` rows are created; attribution exists only on the Job record and in the
  plugin's own logs. Webhooks and event rules likewise do not fire for these writes.
- Stacked Cisco switches (`ios`, `smbos`) are detected and skipped.
- FortiOS devices returning fewer than 20 parsed interfaces are skipped as a truncated
  scrape.
- Device Type interfaces must be pre-created with exact names (`1/1`, `1/2`, …; static
  single-interface devices use `1/1`, except the `5PX 3000i RT3U G2` UPS which uses
  `External-NIC`). Unmatched interfaces are logged and skipped, not created.
