RomanNum3ral
/
OSX-PROXMOX
mirror of https://github.com/luchina-gabriel/OSX-PROXMOX.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OSX-PROXMOX/docs/reference/ORION_HYBRID_ARCHITECTURE.md

18 KiB
Raw Blame History

ORION Hybrid Architecture Documentation

Version: 2.0.0-hybrid Last Updated: 2025-01-20 System: Dell PowerEdge R730 (CQ5QBM2)

Table of Contents

  1. Overview
  2. Architecture Design
  3. Key Features
  4. Hardware Specifications
  5. Network Architecture
  6. Virtual Machines
  7. Deployment Process
  8. Monitoring & Management
  9. Troubleshooting

Overview

The ORION Hybrid Architecture combines the best features from multiple deployment strategies to create a robust, flexible, and intelligent network infrastructure:

  • Proxmox VE as the virtualization foundation (flexibility)
  • NixOS + VyOS Router for high-performance routing (performance)
  • AI Autonomous Agent for intelligent monitoring (intelligence)
  • macOS Support via OSX-PROXMOX (development)
  • iDRAC Automation for remote management (automation)

Design Philosophy

Best of Both Worlds:

  • ✅ Virtualization flexibility from Proxmox
  • ✅ Bare-metal routing performance from VyOS
  • ✅ Declarative configuration from NixOS
  • ✅ AI-powered automation and monitoring
  • ✅ macOS development environment
  • ✅ Full remote management via iDRAC

Architecture Design

┌─────────────────────────────────────────────────────────────────┐
│                    Dell PowerEdge R730 ORION                    │
│                      (CQ5QBM2 - 384GB RAM)                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌───────────────────────────────────────────────────────────┐ │
│  │              iDRAC Enterprise (192.168.1.2)               │ │
│  │          Redfish API - Full Remote Management             │ │
│  └───────────────────────────────────────────────────────────┘ │
│                                                                 │
│  ┌───────────────────────────────────────────────────────────┐ │
│  │            Proxmox VE 8.x Hypervisor Layer                │ │
│  │              Management: 192.168.100.10:8006              │ │
│  └───────────────────────────────────────────────────────────┘ │
│                            │                                    │
│         ┌──────────────────┼──────────────────┐                │
│         │                  │                  │                │
│  ┌──────▼──────┐   ┌──────▼──────┐   ┌──────▼──────┐          │
│  │   VM 200    │   │   VM 300    │   │   VM 100    │          │
│  │   Router    │   │  AI Agent   │   │    macOS    │          │
│  │             │   │             │   │   Sequoia   │          │
│  │ NixOS+VyOS  │   │   NixOS     │   │             │          │
│  │  8 cores    │   │  4 cores    │   │  12 cores   │          │
│  │   32GB      │   │   16GB      │   │    64GB     │          │
│  │             │   │             │   │             │          │
│  │ Services:   │   │ Services:   │   │ Purpose:    │          │
│  │ • BGP       │   │ • AI Agent  │   │ • Dev Env   │          │
│  │ • Firewall  │   │ • Prometh.  │   │ • Testing   │          │
│  │ • DHCP/DNS  │   │ • Grafana   │   │ • Build     │          │
│  │ • NAT       │   │ • Alerts    │   │             │          │
│  └─────────────┘   └─────────────┘   └─────────────┘          │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
         │              │              │
    ┌────▼────┐    ┌───▼───┐     ┌───▼───┐
    │   WAN   │    │  LAN  │     │ Guest │
    │  Telus  │    │ .100  │     │ .200  │
    │ 10GbE   │    │10GbE  │     │ 10GbE │
    └─────────┘    └───────┘     └───────┘

Key Features

1. Hybrid Virtualization Model

  • Proxmox VE 8.x: Enterprise-grade hypervisor

    • Web-based management UI
    • Live migration support
    • Snapshot and backup capabilities
    • LXC container support

  • NixOS VMs: Declarative, reproducible configurations

    • Atomic updates and rollbacks
    • Immutable infrastructure
    • Easy version control

2. High-Performance Routing

  • VyOS Router (VM 200):
    • Dedicated routing VM with minimal overhead
    • BIRD2 for BGP routing
    • nftables for high-performance firewalling
    • Hardware-accelerated networking (virtio)

3. AI-Powered Monitoring

  • Autonomous Agent (VM 300):
    • Real-time network health monitoring
    • Automatic issue detection
    • Self-healing capabilities
    • Predictive analysis

4. macOS Development

  • macOS Sequoia (VM 100):
    • Full macOS 15 support via OSX-PROXMOX
    • 12 cores / 64GB RAM
    • Metal GPU acceleration
    • OpenCore bootloader

5. Full Automation

  • iDRAC Redfish API:
    • Remote power management
    • Boot configuration
    • Hardware monitoring
    • Virtual media mounting

Hardware Specifications

Dell PowerEdge R730 (CQ5QBM2)

Component Specification
CPUs 2x Intel Xeon E5-2690 v4 (14 cores, 2.6GHz)
Total Cores 28 physical / 56 threads
RAM 384GB DDR4-2400 (12x 32GB Samsung)
Storage Controller PERC H730 Mini (RAID 10)
Network 8x NICs (4x 10GbE + 4x 1GbE)
Power Dual 750W redundant PSUs
Management iDRAC 8 Enterprise

Network Interface Mapping

Interface MAC Speed Purpose
eno1 D0:94:66:24:96:7C 1GbE Proxmox Management
eno2 D0:94:66:24:96:7D 1GbE Reserved
eno3 D0:94:66:24:96:7E 10GbE WAN (Telus Fiber) → vmbr0
eno4 D0:94:66:24:96:80 10GbE LAN (Internal) → vmbr1
eno5 - 10GbE macOS Network → vmbr2
eno6 - 10GbE Storage Network → vmbr3
enp3s0f0 - 10GbE Available (Slot 3)
enp3s0f1 - 10GbE Available (Slot 3)

Network Architecture

IP Addressing Scheme

WAN (Telus Fiber)

  • Interface: vmbr0 (eno3)
  • IPv4: DHCP from Telus
  • IPv6: 2602:F674::/48 (prefix delegation)
  • BGP AS: 394955
  • Peers:
    • 206.75.1.127 (Primary - AS 6939)
    • 206.75.1.47 (Secondary - AS 6939)
    • 206.75.1.48 (Tertiary - AS 6939)

LAN (Internal Network)

  • Interface: vmbr1 (eno4)
  • IPv4: 192.168.100.0/24
  • Gateway: 192.168.100.1 (Router VM)
  • DHCP Range: 192.168.100.100 - 192.168.100.200
  • DNS: 192.168.100.1 (Unbound)
  • IPv6: 2602:F674:1000::/64

Guest Network

  • Interface: vmbr2
  • IPv4: 192.168.200.0/24
  • Gateway: 192.168.200.1 (Router VM)
  • Isolation: Restricted to WAN only

Management Network

  • IPv4: 192.168.1.0/24
  • Proxmox: 192.168.1.10 (eno1)
  • iDRAC: 192.168.1.2
  • Router: 192.168.1.1 (eth3)

Network Bridges (Proxmox)

vmbr0: WAN Bridge
  - Physical: eno3 (10GbE)
  - Purpose: Router VM WAN interface
  - VLAN: Aware (for future VLANs)

vmbr1: LAN Bridge
  - Physical: eno4 (10GbE)
  - Purpose: Internal network for VMs
  - IP: 192.168.100.1/24

vmbr2: macOS Bridge
  - Physical: eno5 (10GbE)
  - Purpose: macOS VM network

vmbr3: Storage Bridge
  - Physical: eno6 (10GbE)
  - Purpose: NFS/iSCSI storage network

Virtual Machines

VM 200: ORION-Router

Operating System: NixOS 24.11 + VyOS

Resources:

  • CPUs: 8 cores (host passthrough)
  • RAM: 32GB
  • Disk: 50GB
  • NICs: 4x virtio (WAN, LAN, Guest, Mgmt)

Services:

  • BIRD2: BGP routing (AS 394955)
  • VyOS: Advanced routing and firewall
  • Unbound: DNS resolver (192.168.100.1)
  • Kea DHCP: DHCP server
  • nftables: High-performance firewall
  • Prometheus Node Exporter: Metrics

Network Interfaces:

  • eth0: WAN (vmbr0) - DHCP from Telus
  • eth1: LAN (vmbr1) - 192.168.100.1/24
  • eth2: Guest (vmbr2) - 192.168.200.1/24
  • eth3: Mgmt (vmbr1) - 192.168.1.1/24

Configuration: vm-configs/router-vm/configuration.nix

Features:

  • Stateful firewall with nftables
  • NAT for LAN and Guest networks
  • DHCPv6 prefix delegation
  • BGP route announcements
  • DNS over TLS forwarding
  • Automatic failover between BGP peers

VM 300: ORION-AI-Agent

Operating System: NixOS 24.11

Resources:

  • CPUs: 4 cores (host passthrough)
  • RAM: 16GB
  • Disk: 50GB
  • NICs: 1x virtio (LAN)

Services:

  • Autonomous Agent: Python-based monitoring
  • Prometheus: Metrics collection (port 9090)
  • Grafana: Visualization (port 3000)
  • Alert Manager: Alert routing
  • Node Exporter: System metrics

Network:

  • IP: 192.168.100.20/24
  • Gateway: 192.168.100.1

Configuration: vm-configs/ai-agent-vm/configuration.nix

AI Agent Capabilities:

  • Real-time network monitoring
  • BGP session health checks
  • Bandwidth analysis
  • Anomaly detection
  • Automatic remediation:
    • Restart BGP if all sessions down
    • Alert on high CPU/memory
    • Detect routing loops
  • Hourly status reports

Monitoring Targets:

  • Router VM (192.168.100.1:9100)
  • AI Agent itself (localhost:9100)
  • Proxmox host (192.168.100.10:9100)

Dashboards: http://192.168.100.20:3000

  • Default credentials: admin / orion2025 (change immediately!)

VM 100: HACK-Sequoia-01

Operating System: macOS Sequoia 15

Resources:

  • CPUs: 12 cores (Haswell-noTSX)
  • RAM: 64GB
  • Disk: 256GB
  • NICs: 1x virtio (macOS network)

Configuration: OpenCore 1.0.4

  • SMBIOS: iMacPro1,1
  • SIP: Enabled
  • Secure Boot: Default

Purpose: macOS development environment

Setup: Refer to existing deploy-orion.sh for detailed macOS VM creation

Deployment Process

Prerequisites

  1. Hardware:

    • Dell R730 powered on and accessible
    • iDRAC configured (IP: 192.168.1.2)
    • Network cables connected

  2. Software:

    • Python 3.x with requests library
    • Proxmox VE ISO downloaded
    • NixOS minimal ISO downloaded
    • SSH access configured

  3. Network:

    • Management network (192.168.1.0/24) configured
    • Internet access for downloads

Deployment Steps

Step 1: Run Automated Deployment

# Clone repository
git clone <repo-url>
cd luci-macOSX-PROXMOX

# Install Python dependencies
pip3 install requests

# Run hybrid deployment
python3 deploy-orion-hybrid.py

The deployment wizard will guide you through:

  1. ✅ Prerequisites check
  2. ✅ iDRAC configuration
  3. ✅ Proxmox installation
  4. ✅ Network bridge setup
  5. ✅ Router VM creation
  6. ✅ macOS VM creation
  7. ✅ AI Agent VM creation
  8. ✅ Monitoring setup
  9. ✅ Verification

Step 2: Install Proxmox VE

  1. Mount Proxmox ISO via iDRAC virtual media
  2. Boot system from CD
  3. Follow installer:
    • Hostname: orion-pve.local
    • IP: 192.168.100.10/24
    • Gateway: 192.168.100.1
    • DNS: 1.1.1.1
  4. Access web UI: https://192.168.100.10:8006

Step 3: Configure Network Bridges

In Proxmox web UI:

  1. Navigate to: Datacenter → Node → System → Network
  2. Create bridges:
    • vmbr0: eno3 (WAN)
    • vmbr1: eno4 (LAN)
    • vmbr2: eno5 (macOS)
    • vmbr3: eno6 (Storage)
  3. Apply configuration and reboot

Step 4: Create Router VM

# Create VM
qm create 200 \
  --name ORION-Router \
  --cores 8 \
  --memory 32768 \
  --net0 virtio,bridge=vmbr0 \
  --net1 virtio,bridge=vmbr1 \
  --net2 virtio,bridge=vmbr2 \
  --net3 virtio,bridge=vmbr1 \
  --scsi0 local-lvm:50

# Download NixOS ISO
wget -O /var/lib/vz/template/iso/nixos-minimal.iso \
  https://channels.nixos.org/nixos-24.11/latest-nixos-minimal-x86_64-linux.iso

# Mount ISO and boot
qm set 200 --ide2 local:iso/nixos-minimal.iso,media=cdrom
qm start 200

# Open console and install NixOS
# Copy configuration from: vm-configs/router-vm/configuration.nix

Step 5: Create AI Agent VM

# Create VM
qm create 300 \
  --name ORION-AI-Agent \
  --cores 4 \
  --memory 16384 \
  --net0 virtio,bridge=vmbr1 \
  --scsi0 local-lvm:50

# Mount NixOS ISO and install
# Copy configuration from: vm-configs/ai-agent-vm/configuration.nix

Step 6: Create macOS VM

Refer to deploy-orion.sh for detailed macOS VM setup using OSX-PROXMOX.

Step 7: Verification

# Check VM status
qm list

# Test router connectivity
ping -c 3 192.168.100.1

# Test BGP sessions
ssh admin@192.168.100.1 "birdc show protocols"

# Access Grafana
firefox http://192.168.100.20:3000

# Test internet from LAN
ping -c 3 8.8.8.8

Monitoring & Management

Prometheus Metrics

Endpoint: http://192.168.100.20:9090

Targets:

  • Router: 192.168.100.1:9100
  • AI Agent: 192.168.100.20:9100
  • Proxmox: 192.168.100.10:9100

Sample Queries:

# WAN bandwidth (Mbps)
rate(node_network_receive_bytes_total{device="eth0",instance="192.168.100.1:9100"}[5m]) * 8 / 1000000

# CPU usage
100 - (avg by (instance) (irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

# Memory usage
(1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100

Grafana Dashboards

Access: http://192.168.100.20:3000 Login: admin / orion2025

Pre-configured Dashboards:

  • ORION Network Overview
  • Router Performance
  • BGP Session Status
  • Bandwidth Analysis
  • System Resources

AI Agent Status

# Check agent status
ssh admin@192.168.100.20 "systemctl status orion-agent"

# View logs
ssh admin@192.168.100.20 "journalctl -u orion-agent -f"

# View alerts
ssh admin@192.168.100.20 "tail -f /var/log/orion-agent.log"

iDRAC Management

Access: https://192.168.1.2 Login: root / calvin

Python CLI:

# Power on
python3 deploy-orion-hybrid.py power-on

# Power off
python3 deploy-orion-hybrid.py power-off

# Reboot
python3 deploy-orion-hybrid.py reboot

# Status
python3 deploy-orion-hybrid.py status

Troubleshooting

Router VM Issues

BGP Sessions Not Establishing

# Check BGP status
ssh admin@192.168.100.1 "birdc show protocols all"

# Check firewall
ssh admin@192.168.100.1 "nft list ruleset | grep 179"

# Test connectivity to BGP peers
ssh admin@192.168.100.1 "ping -c 3 206.75.1.127"

# Restart BIRD
ssh admin@192.168.100.1 "sudo systemctl restart bird2"

DHCP Not Working

# Check Kea DHCP status
ssh admin@192.168.100.1 "systemctl status kea-dhcp4"

# View DHCP leases
ssh admin@192.168.100.1 "cat /var/lib/kea/dhcp4.leases"

# Restart DHCP
ssh admin@192.168.100.1 "sudo systemctl restart kea-dhcp4"

DNS Not Resolving

# Check Unbound status
ssh admin@192.168.100.1 "systemctl status unbound"

# Test DNS resolution
ssh admin@192.168.100.1 "dig @127.0.0.1 google.com"

# View Unbound logs
ssh admin@192.168.100.1 "journalctl -u unbound -f"

AI Agent Issues

Agent Not Running

# Check service status
ssh admin@192.168.100.20 "systemctl status orion-agent"

# View recent logs
ssh admin@192.168.100.20 "journalctl -u orion-agent --since '10 minutes ago'"

# Restart agent
ssh admin@192.168.100.20 "sudo systemctl restart orion-agent"

Prometheus Not Collecting Metrics

# Check Prometheus targets
curl http://192.168.100.20:9090/api/v1/targets

# Check Prometheus config
ssh admin@192.168.100.20 "systemctl status prometheus"

# Restart Prometheus
ssh admin@192.168.100.20 "sudo systemctl restart prometheus"

macOS VM Issues

Refer to OSX-PROXMOX documentation and existing troubleshooting guides.

Network Performance Issues

# Check interface status on router
ssh admin@192.168.100.1 "ip link show"

# Monitor bandwidth
ssh admin@192.168.100.1 "iftop -i eth0"

# Check for errors
ssh admin@192.168.100.1 "ip -s link show eth0"

# Test throughput
iperf3 -s # on router
iperf3 -c 192.168.100.1 # from client

Maintenance

Regular Tasks

Daily:

  • Check Grafana dashboards for anomalies
  • Review AI agent alerts

Weekly:

  • Review BGP session uptime
  • Check system resource usage
  • Review firewall logs

Monthly:

  • Update NixOS VMs: nixos-rebuild switch --upgrade
  • Update Proxmox: apt update && apt upgrade
  • Review and rotate logs
  • Test backup restore

Backup Strategy

Proxmox VZ Backup:

# Backup all VMs
vzdump --all --mode snapshot --compress zstd

# Backup specific VM
vzdump 200 --mode snapshot --compress zstd

NixOS Configuration Backup:

# Configurations are in Git - commit regularly
git add vm-configs/
git commit -m "Update VM configurations"
git push

Support & Documentation

  • Main Documentation: DELL_R730_ORION_PROXMOX_INTEGRATION.md
  • Quickstart Guide: ORION_QUICKSTART.md
  • Configuration: orion-config.json
  • Deployment Script: deploy-orion-hybrid.py
  • Legacy Script: deploy-orion.sh

Version History

  • 2.0.0-hybrid (2025-01-20): Hybrid architecture with NixOS router and AI agent
  • 1.0.0 (2025-01-19): Initial Proxmox + pfSense deployment

End of Documentation