Skip to content

mikrotik-crs328-initial-setup, Handoff

URL: https://mkdocs.justinsforge.com/memory/handoffs/mikrotik-crs328-initial-setup-2026-05-14/

Date: 2026-05-14 Owner: next Claude Code session (or Justin direct) when he sits down to rack the switch Parent session: Console Opus 4.7, 2026-05-14 23:15 CDT

The Goal

Bring a new MikroTik CRS328-24P-4S+RM online as the rack aggregation switch downstream of the existing Google Wifi edge, on RouterOS, with VLAN-aware bridge, PoE, and a Finn LACP 2x10G bond. Day-1 scope is L2 + PoE working on a single VLAN matching today's 192.168.86.0/24; VLAN segmentation and edge-router promotion are explicit phase-2 work.

Current State

  • Switch is on hand: CRS328-24P-4S+RM (24x gig PoE, 4x 10G SFP+, 1U, 500W PSU, dual-boot RouterOS/SwOS).
  • Decision made: RouterOS (not SwOS, not bridge-only mode). Justin wants the same OS he'll grow into for routing later.
  • Decision made: slot in as rack aggregation switch downstream of Google Wifi. Not replacing edge router on day 1.
  • Current LAN is flat 192.168.86.0/24, Google Wifi gateway, no VLANs. See forge/system-map/fleet.md for the full Finn + LXC/VM map (homeassistant .180, plex .73, media-server .74, console .50, adguard .75, n8n .82, immich .83, frigate .84, minecraft .23, Finn .67, Sol .33).
  • Nothing physically cabled or powered yet. No config exists on the switch.

Files You'll Work With

File Purpose
forge/system-map/fleet.md Current LAN map, IPs, must stay coherent after VLAN work
forge/infra/network/crs328-baseline.rsc (to create) Full RouterOS config to paste in one shot
forge/infra/network/crs328-finn-bond-proxmox.conf (to create) Proxmox-side LACP bond snippet for Finn's /etc/network/interfaces
forge/memory/general/reference_finn_bmc_lan_configured.md Finn IPMI lives at .68, do not disrupt during cable swap
forge/memory/handoffs/mikrotik-crs328-initial-setup-2026-05-14.md This file

Likely Approach

Phase 1, bench setup (before racking): 1. Console-port serial, 115200 8N1, log in admin / blank. 2. /system reset-configuration no-defaults=yes skip-backup=yes, reboot to bare config. 3. Upgrade to latest stable 7.x: package update then /system routerboard upgrade then reboot twice. 4. Set identity, admin password, NTP, syslog to Console. 5. Export baseline: /export file=baseline-2026-05-14 + /system backup save name=baseline-2026-05-14. Pull both off via scp before doing anything else.

Phase 2, L2 backbone (single VLAN, drop-in compatible with today's LAN): 1. Create br0 bridge with vlan-filtering=yes. 2. Add ether1-24 + sfp-sfpplus1-4 as bridge ports, all pvid=10, frame-types=admit-only-untagged-and-priority-tagged for now. 3. Allocate one copper port as uplink to Google Wifi. 4. Cable Finn 10G #1 to sfp-sfpplus1; verify link + L2 reachability to .67.

Phase 3, PoE: 1. Per-port /interface ethernet poe set etherN poe-out=auto-on. 2. Higher poe-priority on cameras and APs (they get shed last on brownout). 3. /interface ethernet poe monitor [find] to verify draw.

Phase 4, Finn LACP bond (2x 10G): 1. Switch: /interface bonding add name=bond-finn slaves=sfp-sfpplus1,sfp-sfpplus2 mode=802.3ad lacp-rate=1sec transmit-hash-policy=layer-3-and-4. 2. Add bond-finn to br0 with pvid=10. 3. Proxmox: match with bond-mode 802.3ad, miimon 100, bond-xmit-hash-policy layer3+4 in /etc/network/interfaces. 4. Test: iperf3 Console → Finn should clear 9.4 Gbps single-stream, ~18 Gbps multi-stream.

Phase 5 (deferred, separate handoff): VLAN segmentation (MGMT/Servers/IoT/Cameras/Guest/Lab), edge-router promotion (Google Wifi demoted to APs or replaced), firewall.

10G SFP+ Allocation

Port Use Cable
sfp-sfpplus1 Finn 10G #1 DAC, MikroTik S+DA0001 or S+DA0003
sfp-sfpplus2 Finn 10G #2 (LACP bond) DAC
sfp-sfpplus3 Reserved: Console/Vector 10G or NAS empty
sfp-sfpplus4 Reserved: future edge-router uplink or second switch empty

VLAN Plan (Phase 5, not day 1)

VLAN Subnet Purpose
10 192.168.86.0/24 MGMT/main, matches current LAN during transition
20 10.20.0.0/24 Servers (Finn, future bare-metal)
30 10.30.0.0/24 IoT (HA-controlled, smart plugs, TVs)
40 10.40.0.0/24 Cameras (Frigate, doorbell), no-internet default
50 10.50.0.0/24 Guest
99 10.99.0.0/24 Lab/spawn (Vector, ephemeral workers)

PoE Budget

500W PSU, ~370W usable after switch overhead. Way more than current load needs (handful of Frigate cams + doorbell + 2-3 APs ~= 80-120W).

SFP+ Cable Picks

  • <3m in-rack: passive DAC, MikroTik S+DA0001/0003 (~$15-25 each), or FS.com MikroTik-coded equivalents.
  • 5-7m: active DAC or AOC.

  • 10m or between rooms: MM fiber LC-LC + S+85DLC03D transceivers.

  • Avoid 10GBASE-T SFP+ copper modules unless required (2.5W heat per cage in a 4-cage cluster is asking for trouble).

Don't Do

  • Don't replace Google Wifi as edge router on day 1, that's a separate, bigger project.
  • Don't enable VLAN segmentation before the bridge is verified working untagged on VLAN 10. You will lock yourself out.
  • Don't reuse Finn's currently-cabled 1G port without first verifying the new 10G path; keep the old cable plugged in until iperf passes.
  • Don't disrupt Finn IPMI at 192.168.86.68 during cable swap; that's a separate physical port.
  • Don't use 10GBASE-T SFP+ copper modules in adjacent SFP+ cages, thermal will bite.
  • Don't skip the pre-change /export and /system backup save.

Deliverables

  1. CRS328 racked, powered, on latest stable RouterOS 7.x, identity + admin password set.
  2. forge/infra/network/crs328-baseline.rsc checked in, full config including bridge, ports, PoE, bond-finn.
  3. Finn cabled to switch via 2x SFP+ DAC, LACP bond-finn up on both sides, iperf3 Console → Finn passing 9 Gbps+ single-stream.
  4. All existing LXC/VM IPs still reachable, Plex/Immich/Frigate still streaming.
  5. Baseline backup pulled off switch to forge/infra/network/backups/crs328-baseline-2026-05-14.{rsc,backup}.
  6. forge/system-map/fleet.md updated with switch entry + 10G topology note.

Done When

  • Switch reachable at chosen MGMT IP on VLAN 10
  • Latest stable RouterOS 7.x running, routerboard firmware matched
  • Baseline .rsc + .backup archived off-switch
  • Finn bond-finn shows both slaves up, LACP partner state 0x3d
  • iperf3 -c finn -P 4 ≥ 18 Gbps aggregate from Console
  • All current 1G clients still reach internet + each other
  • PoE devices (cams, doorbell, APs) powered through new switch with poe monitor showing healthy draw
  • forge/system-map/fleet.md reflects new switch + Finn 10G bond
  • Open questions resolved (see below) and recorded in handoff

Open Questions to Resolve Before Phase 5

  1. Keep Google Wifi as edge router long-term, or plan promotion of CRS328 to edge?
  2. Switch physical location, same rack as Finn? Cable distance to Finn?
  3. Does Finn already have both onboard SFP+ ports free? (MS-01 has 2x SFP+ onboard, probably yes.)
  4. Want the full RouterOS config drafted as a single .rsc ahead of time, or build it interactively during racking?

Status Update, 2026-05-20

Scanned 192.168.86.0/24 from Console: 51 live hosts, none answering on RouterOS-typical ports (22/23/80/443/161/2000/8291/8728/8729), no Mikrotik OUI in ARP. Switch is still not on the home LAN, consistent with the original "nothing physically cabled or powered yet" state.

If/when racked, expected sequence: 1. Console-port serial bench config first (per Phase 1 above), do not DHCP from Google Wifi during bootstrap. 2. After identity + admin password + SSH key set, then cable an uplink and confirm reachable from Console as ssh mikrotik (alias to be added in ~/.ssh/config + reference_ssh_aliases.md). 3. Add a row to system-map/fleet.md "Network edge" section with the assigned IP (currently shows TBD).

[Claude Code], 2026-05-20.