Add comprehensive deployment resources for ORION infrastructure
Created complete deployment toolkit to guide users through getting
ORION up and running with automated validation and troubleshooting.
New Files:
----------
1. scripts/pre-deployment-check.sh (Validation Script)
- Checks all prerequisites before deployment
- Validates Python installation and dependencies
- Tests iDRAC connectivity
- Verifies repository files are present
- Checks for required ISOs
- Validates disk space and network configuration
- Exit codes: 0 = ready, 1 = issues found
2. scripts/post-deployment-check.sh (Verification Script)
- Comprehensive 15-point validation
- Tests all deployed services:
* iDRAC accessibility
* Proxmox web UI
* Router VM (ping, SSH, DNS, BGP)
* AI Agent VM (Prometheus, Grafana)
* Internet connectivity
* IPv6 connectivity
* Monitoring targets
- Provides clear pass/fail/warning status
- Suggests fixes for common issues
3. DEPLOYMENT_CHECKLIST.md (Complete Guide)
- Detailed 18-step deployment process
- Time estimates for each phase (3-4 hours total)
- Pre-deployment phase (30 min)
- Deployment phase (2-3 hours)
- Post-deployment hardening (30 min)
- Each step with checkboxes for tracking
- Inline commands and verification steps
- Troubleshooting guide for common issues
- Success criteria checklist
- Timeline summary
Features:
---------
Pre-Deployment Check:
✓ Python version validation (3.8+)
✓ Dependencies check (requests library)
✓ iDRAC reachability test
✓ Repository integrity verification
✓ ISO availability check
✓ Disk space validation
✓ Git status check
✓ SSH key detection
✓ Network route verification
Post-Deployment Check:
✓ iDRAC health
✓ Proxmox web UI
✓ Router VM connectivity
✓ Router SSH, DNS, DHCP services
✓ BGP session status (counts established peers)
✓ AI Agent VM reachability
✓ Prometheus metrics collection
✓ Grafana dashboard
✓ Prometheus targets monitoring
✓ Internet connectivity (IPv4 and IPv6)
✓ AI Agent service status
✓ Node exporter metrics
Deployment Checklist:
✓ Pre-deployment validation (6 steps)
✓ Proxmox installation guide (step 7)
✓ Network bridge configuration (step 8)
✓ Router VM creation and setup (steps 9-11)
✓ AI Agent VM deployment (step 12)
✓ Monitoring configuration (step 13)
✓ Connectivity testing (step 14)
✓ Post-deployment validation (step 15)
✓ Security hardening (step 16)
✓ Backup configuration (step 17)
✓ Documentation tasks (step 18)
✓ Optional enhancements (macOS, VPN, etc.)
✓ Troubleshooting guide
✓ Timeline summary
Benefits:
---------
1. Automation: Pre-flight checks catch issues before deployment
2. Validation: Post-deployment ensures everything works
3. Guidance: Step-by-step checklist prevents missing steps
4. Time Management: Clear estimates for each phase
5. Error Prevention: Inline verification commands
6. Troubleshooting: Common issues and solutions documented
7. Professional: Color-coded output with clear pass/fail status
Usage:
------
Before deployment:
./scripts/pre-deployment-check.sh
During deployment:
Follow DEPLOYMENT_CHECKLIST.md step-by-step
After deployment:
./scripts/post-deployment-check.sh
Both scripts provide:
- Color-coded output (green=pass, yellow=warn, red=fail)
- Clear summary statistics
- Actionable next steps
- Exit codes for automation
Files Made Executable:
----------------------
- scripts/pre-deployment-check.sh (755)
- scripts/post-deployment-check.sh (755)
This completes the "Deploy Now" path, giving users everything
they need to successfully deploy ORION infrastructure with
confidence and proper validation.
This commit is contained in:
Claude
parent
6b9f29171d
commit
c591731c83
|
|
@ -0,0 +1,693 @@
|
|||
# ORION Deployment Checklist
|
||||
|
||||
**Complete deployment guide for Dell R730 ORION Hybrid Infrastructure**
|
||||
|
||||
---
|
||||
|
||||
## Pre-Deployment Phase (30 minutes)
|
||||
|
||||
### ☐ 1. Verify Hardware
|
||||
|
||||
- [ ] Dell R730 powered on and accessible
|
||||
- [ ] iDRAC configured at 192.168.1.2
|
||||
- [ ] All 8 network cables connected
|
||||
- [ ] RAID controller configured (RAID 10 recommended)
|
||||
- [ ] Server shows healthy status in iDRAC
|
||||
|
||||
**Verification**:
|
||||
```bash
|
||||
# Access iDRAC
|
||||
open https://192.168.1.2
|
||||
# Login: root / calvin
|
||||
```
|
||||
|
||||
### ☐ 2. Prepare Workstation
|
||||
|
||||
- [ ] Python 3.8+ installed
|
||||
- [ ] Git installed
|
||||
- [ ] Network access to 192.168.1.0/24
|
||||
|
||||
**Verification**:
|
||||
```bash
|
||||
python3 --version # Should show 3.8+
|
||||
git --version
|
||||
ping -c 3 192.168.1.2
|
||||
```
|
||||
|
||||
### ☐ 3. Clone Repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/luci-digital/luci-macOSX-PROXMOX.git
|
||||
cd luci-macOSX-PROXMOX
|
||||
```
|
||||
|
||||
- [ ] Repository cloned successfully
|
||||
- [ ] All files present
|
||||
|
||||
### ☐ 4. Install Dependencies
|
||||
|
||||
```bash
|
||||
pip3 install requests
|
||||
```
|
||||
|
||||
- [ ] Python requests library installed
|
||||
|
||||
### ☐ 5. Download Required ISOs
|
||||
|
||||
**Proxmox VE**:
|
||||
```bash
|
||||
# Download from: https://www.proxmox.com/en/downloads
|
||||
# Save to: ~/Downloads/proxmox-ve_8.x.iso
|
||||
```
|
||||
|
||||
**NixOS** (optional - can download later):
|
||||
```bash
|
||||
# Download from: https://nixos.org/download.html
|
||||
# Direct link: https://channels.nixos.org/nixos-24.11/latest-nixos-minimal-x86_64-linux.iso
|
||||
wget -O /tmp/nixos-minimal.iso \
|
||||
https://channels.nixos.org/nixos-24.11/latest-nixos-minimal-x86_64-linux.iso
|
||||
```
|
||||
|
||||
- [ ] Proxmox ISO downloaded (~1GB)
|
||||
- [ ] NixOS ISO downloaded (~900MB) - optional
|
||||
|
||||
### ☐ 6. Run Pre-Deployment Check
|
||||
|
||||
```bash
|
||||
chmod +x scripts/pre-deployment-check.sh
|
||||
./scripts/pre-deployment-check.sh
|
||||
```
|
||||
|
||||
- [ ] All checks passed (or only warnings)
|
||||
- [ ] Ready to proceed
|
||||
|
||||
**Expected output**: "✓ Ready for deployment!"
|
||||
|
||||
---
|
||||
|
||||
## Deployment Phase (2-3 hours)
|
||||
|
||||
### ☐ 7. Install Proxmox VE
|
||||
|
||||
**Time: 30 minutes**
|
||||
|
||||
1. **Mount ISO via iDRAC**:
|
||||
```
|
||||
- Open https://192.168.1.2
|
||||
- Go to: Virtual Console → Launch Virtual Console
|
||||
- Virtual Media → Connect Virtual Media
|
||||
- Map CD/DVD → Select proxmox-ve_8.x.iso
|
||||
```
|
||||
|
||||
2. **Configure boot**:
|
||||
```bash
|
||||
python3 deploy-orion-hybrid.py
|
||||
# Follow wizard to configure boot to CD
|
||||
```
|
||||
|
||||
3. **Install Proxmox**:
|
||||
- Boot system (will boot from virtual CD)
|
||||
- Follow Proxmox installer:
|
||||
* Target disk: /dev/sda (or appropriate disk)
|
||||
* Country: Your location
|
||||
* Timezone: Your timezone
|
||||
* Password: Set strong password (save it!)
|
||||
* Email: your-email@domain.com
|
||||
* Hostname: `orion-pve.local`
|
||||
* IP: `192.168.100.10/24`
|
||||
* Gateway: `192.168.100.1`
|
||||
* DNS: `1.1.1.1`
|
||||
- Click Install
|
||||
- Wait for installation (10-15 minutes)
|
||||
- Reboot when prompted
|
||||
|
||||
4. **Verify Proxmox**:
|
||||
```bash
|
||||
# Wait 2-3 minutes after reboot
|
||||
curl -k https://192.168.100.10:8006
|
||||
# Should return HTML
|
||||
```
|
||||
|
||||
- [ ] Proxmox installed successfully
|
||||
- [ ] Can access web UI: https://192.168.100.10:8006
|
||||
- [ ] Can login as root
|
||||
|
||||
**Checkpoint**: Proxmox web UI accessible ✓
|
||||
|
||||
---
|
||||
|
||||
### ☐ 8. Configure Network Bridges
|
||||
|
||||
**Time: 10 minutes**
|
||||
|
||||
1. **Login to Proxmox web UI**:
|
||||
- URL: https://192.168.100.10:8006
|
||||
- User: root
|
||||
- Password: (from step 7)
|
||||
|
||||
2. **Create bridges**:
|
||||
- Navigate to: Datacenter → pve → System → Network
|
||||
- Click "Create" → "Linux Bridge"
|
||||
|
||||
**vmbr0 (WAN)**:
|
||||
```
|
||||
Name: vmbr0
|
||||
Bridge ports: eno3
|
||||
Comment: WAN - Telus Fiber
|
||||
VLAN aware: ✓
|
||||
```
|
||||
|
||||
**vmbr1 (LAN)**:
|
||||
```
|
||||
Name: vmbr1
|
||||
Bridge ports: eno4
|
||||
IPv4/CIDR: 192.168.100.1/24
|
||||
Comment: LAN - Internal Network
|
||||
VLAN aware: ✓
|
||||
```
|
||||
|
||||
**vmbr2 (Guest)**:
|
||||
```
|
||||
Name: vmbr2
|
||||
Bridge ports: eno5
|
||||
Comment: Guest Network
|
||||
```
|
||||
|
||||
**vmbr3 (Storage)**:
|
||||
```
|
||||
Name: vmbr3
|
||||
Bridge ports: eno6
|
||||
Comment: Storage Network
|
||||
```
|
||||
|
||||
3. **Apply configuration**:
|
||||
- Click "Apply Configuration"
|
||||
- May need to reboot Proxmox
|
||||
|
||||
- [ ] All 4 bridges created
|
||||
- [ ] Configuration applied
|
||||
- [ ] Proxmox still accessible after changes
|
||||
|
||||
**Checkpoint**: Network bridges configured ✓
|
||||
|
||||
---
|
||||
|
||||
### ☐ 9. Create Router VM (VM 200)
|
||||
|
||||
**Time: 45 minutes**
|
||||
|
||||
1. **Create VM in Proxmox**:
|
||||
```
|
||||
- Click "Create VM"
|
||||
- General:
|
||||
* Node: pve
|
||||
* VM ID: 200
|
||||
* Name: ORION-Router
|
||||
- OS:
|
||||
* ISO: Upload NixOS ISO first, then select it
|
||||
* Type: Linux
|
||||
* Version: 6.x - 2.6 Kernel
|
||||
- System:
|
||||
* BIOS: OVMF (UEFI)
|
||||
* Add EFI Disk: ✓
|
||||
* Machine: q35
|
||||
- Disks:
|
||||
* Bus/Device: SCSI
|
||||
* Storage: local-lvm
|
||||
* Size: 50 GB
|
||||
- CPU:
|
||||
* Cores: 8
|
||||
* Type: host
|
||||
- Memory:
|
||||
* Memory: 32768 MB (32 GB)
|
||||
- Network:
|
||||
* net0: vmbr0 (WAN)
|
||||
* Add 3 more NICs:
|
||||
- net1: vmbr1 (LAN)
|
||||
- net2: vmbr2 (Guest)
|
||||
- net3: vmbr1 (Management)
|
||||
```
|
||||
|
||||
2. **Start VM and install NixOS**:
|
||||
```bash
|
||||
# In Proxmox web UI
|
||||
- Select VM 200
|
||||
- Click "Start"
|
||||
- Click "Console"
|
||||
```
|
||||
|
||||
3. **In NixOS installer console**:
|
||||
```bash
|
||||
# Partition disk
|
||||
parted /dev/sda -- mklabel gpt
|
||||
parted /dev/sda -- mkpart ESP fat32 1MiB 512MiB
|
||||
parted /dev/sda -- set 1 esp on
|
||||
parted /dev/sda -- mkpart primary 512MiB 100%
|
||||
|
||||
# Format
|
||||
mkfs.fat -F 32 -n boot /dev/sda1
|
||||
mkfs.ext4 -L nixos /dev/sda2
|
||||
|
||||
# Mount
|
||||
mount /dev/disk/by-label/nixos /mnt
|
||||
mkdir -p /mnt/boot
|
||||
mount /dev/disk/by-label/boot /mnt/boot
|
||||
|
||||
# Generate config
|
||||
nixos-generate-config --root /mnt
|
||||
```
|
||||
|
||||
4. **Transfer configuration**:
|
||||
```bash
|
||||
# On your workstation
|
||||
scp vm-configs/router-vm/configuration.nix nixos@<ROUTER_IP>:/tmp/
|
||||
|
||||
# In NixOS installer console
|
||||
mv /tmp/configuration.nix /mnt/etc/nixos/
|
||||
|
||||
# Install
|
||||
nixos-install
|
||||
# Set root password when prompted
|
||||
|
||||
# Reboot
|
||||
reboot
|
||||
```
|
||||
|
||||
5. **Verify router**:
|
||||
```bash
|
||||
# After reboot (wait 2-3 minutes)
|
||||
ping -c 3 192.168.100.1
|
||||
ssh admin@192.168.100.1 # May need to setup SSH key first
|
||||
```
|
||||
|
||||
- [ ] VM 200 created
|
||||
- [ ] NixOS installed
|
||||
- [ ] Router configuration applied
|
||||
- [ ] Router accessible at 192.168.100.1
|
||||
- [ ] Can ping router
|
||||
|
||||
**Checkpoint**: Router VM running and accessible ✓
|
||||
|
||||
---
|
||||
|
||||
### ☐ 10. Configure Router SSH Access
|
||||
|
||||
**Time: 5 minutes**
|
||||
|
||||
```bash
|
||||
# Generate SSH key if you don't have one
|
||||
ssh-keygen -t ed25519 -C "your-email@domain.com"
|
||||
|
||||
# Copy public key to router
|
||||
# You'll need to do this via console first time
|
||||
|
||||
# In router VM console:
|
||||
mkdir -p /home/admin/.ssh
|
||||
nano /home/admin/.ssh/authorized_keys
|
||||
# Paste your public key (from ~/.ssh/id_ed25519.pub)
|
||||
|
||||
# Set permissions
|
||||
chmod 700 /home/admin/.ssh
|
||||
chmod 600 /home/admin/.ssh/authorized_keys
|
||||
chown -R admin:admin /home/admin/.ssh
|
||||
|
||||
# Test SSH
|
||||
ssh admin@192.168.100.1
|
||||
```
|
||||
|
||||
- [ ] SSH key generated
|
||||
- [ ] Public key added to router
|
||||
- [ ] Can SSH without password
|
||||
|
||||
---
|
||||
|
||||
### ☐ 11. Verify Router Services
|
||||
|
||||
**Time: 10 minutes**
|
||||
|
||||
```bash
|
||||
# SSH to router
|
||||
ssh admin@192.168.100.1
|
||||
|
||||
# Check BIRD BGP
|
||||
sudo birdc show protocols
|
||||
# Should show 3 BGP peers (may be down until WAN is configured)
|
||||
|
||||
# Check firewall
|
||||
sudo nft list ruleset
|
||||
# Should show nftables rules
|
||||
|
||||
# Check DNS
|
||||
dig @127.0.0.1 google.com
|
||||
# Should resolve
|
||||
|
||||
# Check DHCP
|
||||
systemctl status kea-dhcp4
|
||||
# Should be active
|
||||
|
||||
# Check node exporter
|
||||
curl http://localhost:9100/metrics
|
||||
# Should return metrics
|
||||
```
|
||||
|
||||
- [ ] BGP service running (BIRD2)
|
||||
- [ ] Firewall active (nftables)
|
||||
- [ ] DNS resolving (Unbound)
|
||||
- [ ] DHCP server running (Kea)
|
||||
- [ ] Node exporter running
|
||||
|
||||
**Checkpoint**: Router services operational ✓
|
||||
|
||||
---
|
||||
|
||||
### ☐ 12. Create AI Agent VM (VM 300)
|
||||
|
||||
**Time: 30 minutes**
|
||||
|
||||
1. **Create VM**:
|
||||
```
|
||||
- Click "Create VM"
|
||||
- VM ID: 300
|
||||
- Name: ORION-AI-Agent
|
||||
- OS: NixOS ISO
|
||||
- Cores: 4
|
||||
- Memory: 16384 MB (16 GB)
|
||||
- Disk: 50 GB
|
||||
- Network: net0 → vmbr1 (LAN)
|
||||
```
|
||||
|
||||
2. **Install NixOS** (same process as router):
|
||||
```bash
|
||||
# Partition, format, mount (same as router)
|
||||
# Transfer configuration
|
||||
scp vm-configs/ai-agent-vm/configuration.nix nixos@<IP>:/tmp/
|
||||
scp vm-configs/ai-agent-vm/autonomous_agent.py nixos@<IP>:/tmp/
|
||||
|
||||
# Install
|
||||
nixos-install
|
||||
reboot
|
||||
```
|
||||
|
||||
3. **Verify AI Agent**:
|
||||
```bash
|
||||
ping -c 3 192.168.100.20
|
||||
ssh admin@192.168.100.20
|
||||
|
||||
# Check services
|
||||
systemctl status prometheus
|
||||
systemctl status grafana
|
||||
systemctl status orion-agent
|
||||
```
|
||||
|
||||
- [ ] VM 300 created
|
||||
- [ ] NixOS installed
|
||||
- [ ] AI Agent configuration applied
|
||||
- [ ] Prometheus running (port 9090)
|
||||
- [ ] Grafana running (port 3000)
|
||||
- [ ] AI agent service running
|
||||
|
||||
**Checkpoint**: AI Agent VM operational ✓
|
||||
|
||||
---
|
||||
|
||||
### ☐ 13. Configure Monitoring
|
||||
|
||||
**Time: 10 minutes**
|
||||
|
||||
1. **Access Grafana**:
|
||||
```
|
||||
URL: http://192.168.100.20:3000
|
||||
Username: admin
|
||||
Password: orion2025
|
||||
```
|
||||
|
||||
2. **Change password**:
|
||||
- Profile → Change Password
|
||||
- Set new strong password
|
||||
|
||||
3. **Verify Prometheus data source**:
|
||||
- Configuration → Data Sources
|
||||
- Should see "Prometheus" configured
|
||||
|
||||
4. **Check targets**:
|
||||
```bash
|
||||
# Visit Prometheus
|
||||
open http://192.168.100.20:9090/targets
|
||||
# Should show:
|
||||
# - orion-router (192.168.100.1:9100) - UP
|
||||
# - orion-ai-agent (localhost:9100) - UP
|
||||
# - proxmox (192.168.100.10:9100) - UP/DOWN (if exporter installed)
|
||||
```
|
||||
|
||||
- [ ] Can access Grafana
|
||||
- [ ] Password changed
|
||||
- [ ] Prometheus collecting metrics
|
||||
- [ ] At least 2 targets UP
|
||||
|
||||
**Checkpoint**: Monitoring operational ✓
|
||||
|
||||
---
|
||||
|
||||
### ☐ 14. Test Internet Connectivity
|
||||
|
||||
**Time: 5 minutes**
|
||||
|
||||
```bash
|
||||
# From your workstation
|
||||
ping -c 3 192.168.100.1
|
||||
|
||||
# Set your workstation to use router as gateway
|
||||
# (or connect a device to LAN)
|
||||
|
||||
# Test DNS
|
||||
dig @192.168.100.1 google.com
|
||||
|
||||
# Test internet
|
||||
ping -c 3 8.8.8.8
|
||||
ping -c 3 google.com
|
||||
|
||||
# Test BGP (on router)
|
||||
ssh admin@192.168.100.1 "birdc show protocols"
|
||||
```
|
||||
|
||||
- [ ] Can reach router
|
||||
- [ ] DNS resolving through router
|
||||
- [ ] Internet connectivity working
|
||||
- [ ] BGP sessions established (if WAN configured)
|
||||
|
||||
**Checkpoint**: Network routing functional ✓
|
||||
|
||||
---
|
||||
|
||||
### ☐ 15. Run Post-Deployment Validation
|
||||
|
||||
**Time: 5 minutes**
|
||||
|
||||
```bash
|
||||
chmod +x scripts/post-deployment-check.sh
|
||||
./scripts/post-deployment-check.sh
|
||||
```
|
||||
|
||||
- [ ] All critical checks passed
|
||||
- [ ] No failed checks
|
||||
- [ ] System ready for use
|
||||
|
||||
**Expected output**: "✓ Deployment successful!"
|
||||
|
||||
---
|
||||
|
||||
## Post-Deployment Phase (30 minutes)
|
||||
|
||||
### ☐ 16. Security Hardening
|
||||
|
||||
- [ ] Change default passwords:
|
||||
- [ ] Proxmox root password
|
||||
- [ ] Router admin password
|
||||
- [ ] AI Agent admin password
|
||||
- [ ] Grafana admin password
|
||||
|
||||
- [ ] Configure SSH keys (disable password auth):
|
||||
```bash
|
||||
# On each VM, edit /etc/nixos/configuration.nix
|
||||
services.openssh.settings.PasswordAuthentication = false;
|
||||
|
||||
# Rebuild
|
||||
sudo nixos-rebuild switch
|
||||
```
|
||||
|
||||
- [ ] Review firewall rules:
|
||||
```bash
|
||||
ssh admin@192.168.100.1 "sudo nft list ruleset"
|
||||
```
|
||||
|
||||
### ☐ 17. Configure Backups
|
||||
|
||||
```bash
|
||||
# Setup Proxmox backup schedule
|
||||
# In Proxmox web UI:
|
||||
# Datacenter → Backup → Add
|
||||
|
||||
# Backup VMs weekly:
|
||||
# - Schedule: Weekly, Sunday 2:00 AM
|
||||
# - VMs: 200, 300
|
||||
# - Retention: Keep 4 backups
|
||||
```
|
||||
|
||||
- [ ] Proxmox backup schedule configured
|
||||
- [ ] VM configs backed up to Git
|
||||
- [ ] Backup destination configured
|
||||
|
||||
### ☐ 18. Documentation
|
||||
|
||||
- [ ] Network diagram updated
|
||||
- [ ] Passwords stored in password manager
|
||||
- [ ] IP addresses documented
|
||||
- [ ] SSH keys backed up
|
||||
- [ ] Configuration files committed to Git
|
||||
|
||||
---
|
||||
|
||||
## Optional Enhancements
|
||||
|
||||
### ☐ macOS VM (VM 100)
|
||||
|
||||
Follow existing `deploy-orion.sh` guide for macOS setup.
|
||||
|
||||
**Time: 2-3 hours**
|
||||
|
||||
- [ ] macOS VM created
|
||||
- [ ] macOS installed
|
||||
- [ ] OSX-PROXMOX configured
|
||||
- [ ] OrbStack installed (for future glass UI)
|
||||
|
||||
### ☐ Additional Monitoring
|
||||
|
||||
- [ ] Custom Grafana dashboards created
|
||||
- [ ] Email alerts configured
|
||||
- [ ] Slack notifications setup
|
||||
- [ ] Mobile access configured
|
||||
|
||||
### ☐ VPN Access
|
||||
|
||||
- [ ] WireGuard VPN configured on router
|
||||
- [ ] Remote access tested
|
||||
- [ ] Mobile clients configured
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting Guide
|
||||
|
||||
### Router VM won't boot
|
||||
|
||||
```bash
|
||||
# Check VM settings in Proxmox
|
||||
# Ensure:
|
||||
# - Boot order: scsi0 first
|
||||
# - EFI disk enabled
|
||||
# - Correct network bridges
|
||||
|
||||
# View console for errors
|
||||
# In Proxmox: VM 200 → Console
|
||||
```
|
||||
|
||||
### BGP sessions not establishing
|
||||
|
||||
```bash
|
||||
ssh admin@192.168.100.1
|
||||
|
||||
# Check WAN interface has IP
|
||||
ip addr show eth0
|
||||
|
||||
# Test connectivity to BGP peers
|
||||
ping -c 3 206.75.1.127
|
||||
|
||||
# Check BIRD logs
|
||||
journalctl -u bird2 -f
|
||||
|
||||
# Restart BIRD
|
||||
sudo systemctl restart bird2
|
||||
```
|
||||
|
||||
### Grafana not accessible
|
||||
|
||||
```bash
|
||||
ssh admin@192.168.100.20
|
||||
|
||||
# Check service
|
||||
systemctl status grafana
|
||||
|
||||
# Check firewall
|
||||
sudo nft list ruleset | grep 3000
|
||||
|
||||
# Check logs
|
||||
journalctl -u grafana -f
|
||||
|
||||
# Restart
|
||||
sudo systemctl restart grafana
|
||||
```
|
||||
|
||||
### No internet from LAN
|
||||
|
||||
```bash
|
||||
# On router
|
||||
ssh admin@192.168.100.1
|
||||
|
||||
# Check NAT
|
||||
sudo nft list table ip nat
|
||||
|
||||
# Check routing
|
||||
ip route show
|
||||
|
||||
# Check WAN interface
|
||||
ip addr show eth0
|
||||
|
||||
# Test from router itself
|
||||
ping -c 3 8.8.8.8
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
✅ **Deployment is successful when**:
|
||||
|
||||
1. Proxmox web UI accessible
|
||||
2. Router VM routing traffic
|
||||
3. BGP sessions established (3/3)
|
||||
4. AI Agent collecting metrics
|
||||
5. Grafana showing dashboards
|
||||
6. Internet connectivity working
|
||||
7. DNS resolution working
|
||||
8. All VMs auto-start on boot
|
||||
9. Monitoring shows all services healthy
|
||||
10. Post-deployment check passes
|
||||
|
||||
---
|
||||
|
||||
## Timeline Summary
|
||||
|
||||
| Phase | Time | Tasks |
|
||||
|-------|------|-------|
|
||||
| Pre-Deployment | 30 min | Setup, downloads, validation |
|
||||
| Proxmox Install | 30 min | Install and configure Proxmox |
|
||||
| Network Config | 10 min | Create bridges |
|
||||
| Router VM | 45 min | Install and configure |
|
||||
| AI Agent VM | 30 min | Install and configure |
|
||||
| Testing | 15 min | Validate deployment |
|
||||
| Security | 30 min | Harden security |
|
||||
| **Total** | **3-4 hours** | Complete deployment |
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues:
|
||||
|
||||
1. Check troubleshooting section above
|
||||
2. Review logs: `journalctl -xe`
|
||||
3. Check VM console output in Proxmox
|
||||
4. Consult `ORION_HYBRID_ARCHITECTURE.md`
|
||||
5. Verify network configuration
|
||||
6. Re-run validation scripts
|
||||
|
||||
**Happy Deploying! 🚀**
|
||||
|
|
@ -0,0 +1,217 @@
|
|||
#!/bin/bash
|
||||
# ORION Post-Deployment Validation Script
|
||||
# Verifies all services are running correctly
|
||||
|
||||
set -e
|
||||
|
||||
# Colors
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
# Configuration
|
||||
PROXMOX_IP="192.168.100.10"
|
||||
ROUTER_IP="192.168.100.1"
|
||||
AI_AGENT_IP="192.168.100.20"
|
||||
IDRAC_IP="192.168.1.2"
|
||||
|
||||
echo "=========================================="
|
||||
echo "ORION Post-Deployment Validation"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
|
||||
CHECKS_PASSED=0
|
||||
CHECKS_FAILED=0
|
||||
CHECKS_WARNED=0
|
||||
|
||||
check_pass() {
|
||||
echo -e "${GREEN}✓${NC} $1"
|
||||
((CHECKS_PASSED++))
|
||||
}
|
||||
|
||||
check_fail() {
|
||||
echo -e "${RED}✗${NC} $1"
|
||||
((CHECKS_FAILED++))
|
||||
}
|
||||
|
||||
check_warn() {
|
||||
echo -e "${YELLOW}⚠${NC} $1"
|
||||
((CHECKS_WARNED++))
|
||||
}
|
||||
|
||||
# Check 1: iDRAC accessibility
|
||||
echo "[1/15] Checking iDRAC..."
|
||||
if curl -k -s --connect-timeout 5 https://$IDRAC_IP &> /dev/null; then
|
||||
check_pass "iDRAC at $IDRAC_IP is accessible"
|
||||
else
|
||||
check_fail "iDRAC not accessible"
|
||||
fi
|
||||
|
||||
# Check 2: Proxmox web UI
|
||||
echo "[2/15] Checking Proxmox..."
|
||||
if curl -k -s --connect-timeout 5 https://$PROXMOX_IP:8006 &> /dev/null; then
|
||||
check_pass "Proxmox web UI at $PROXMOX_IP:8006 is accessible"
|
||||
else
|
||||
check_fail "Proxmox web UI not accessible"
|
||||
fi
|
||||
|
||||
# Check 3: Router reachability
|
||||
echo "[3/15] Checking Router VM..."
|
||||
if ping -c 3 -W 2 $ROUTER_IP &> /dev/null; then
|
||||
check_pass "Router VM at $ROUTER_IP is reachable"
|
||||
else
|
||||
check_fail "Router VM not reachable"
|
||||
fi
|
||||
|
||||
# Check 4: Router SSH
|
||||
echo "[4/15] Checking Router SSH..."
|
||||
if nc -z -w 2 $ROUTER_IP 22 2>/dev/null; then
|
||||
check_pass "Router SSH port is open"
|
||||
else
|
||||
check_warn "Router SSH not accessible (may need SSH key setup)"
|
||||
fi
|
||||
|
||||
# Check 5: Router DNS
|
||||
echo "[5/15] Checking Router DNS..."
|
||||
if nc -z -u -w 2 $ROUTER_IP 53 2>/dev/null; then
|
||||
check_pass "Router DNS service is running"
|
||||
else
|
||||
check_warn "Router DNS not responding"
|
||||
fi
|
||||
|
||||
# Check 6: DNS resolution
|
||||
echo "[6/15] Testing DNS resolution..."
|
||||
if dig @$ROUTER_IP google.com +short +timeout=5 &> /dev/null; then
|
||||
check_pass "DNS resolution working"
|
||||
else
|
||||
check_warn "DNS resolution not working"
|
||||
fi
|
||||
|
||||
# Check 7: BGP sessions (if router is accessible)
|
||||
echo "[7/15] Checking BGP sessions..."
|
||||
if command -v ssh &> /dev/null && ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no admin@$ROUTER_IP "birdc show protocols" 2>/dev/null | grep -q "Established"; then
|
||||
BGP_COUNT=$(ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no admin@$ROUTER_IP "birdc show protocols" 2>/dev/null | grep -c "Established" || echo "0")
|
||||
check_pass "BGP sessions established ($BGP_COUNT peers)"
|
||||
else
|
||||
check_warn "Cannot check BGP sessions (SSH required)"
|
||||
fi
|
||||
|
||||
# Check 8: AI Agent reachability
|
||||
echo "[8/15] Checking AI Agent VM..."
|
||||
if ping -c 3 -W 2 $AI_AGENT_IP &> /dev/null; then
|
||||
check_pass "AI Agent VM at $AI_AGENT_IP is reachable"
|
||||
else
|
||||
check_fail "AI Agent VM not reachable"
|
||||
fi
|
||||
|
||||
# Check 9: Prometheus
|
||||
echo "[9/15] Checking Prometheus..."
|
||||
if curl -s --connect-timeout 5 http://$AI_AGENT_IP:9090/-/ready &> /dev/null; then
|
||||
check_pass "Prometheus is running on $AI_AGENT_IP:9090"
|
||||
else
|
||||
check_warn "Prometheus not accessible"
|
||||
fi
|
||||
|
||||
# Check 10: Grafana
|
||||
echo "[10/15] Checking Grafana..."
|
||||
if curl -s --connect-timeout 5 http://$AI_AGENT_IP:3000/api/health | grep -q "ok"; then
|
||||
check_pass "Grafana is running on $AI_AGENT_IP:3000"
|
||||
else
|
||||
check_warn "Grafana not accessible"
|
||||
fi
|
||||
|
||||
# Check 11: Prometheus targets
|
||||
echo "[11/15] Checking Prometheus targets..."
|
||||
if curl -s --connect-timeout 5 "http://$AI_AGENT_IP:9090/api/v1/targets" | grep -q "up"; then
|
||||
TARGETS_UP=$(curl -s "http://$AI_AGENT_IP:9090/api/v1/targets" | grep -o '"health":"up"' | wc -l)
|
||||
check_pass "Prometheus monitoring $TARGETS_UP target(s)"
|
||||
else
|
||||
check_warn "Cannot verify Prometheus targets"
|
||||
fi
|
||||
|
||||
# Check 12: Internet connectivity through router
|
||||
echo "[12/15] Checking internet connectivity..."
|
||||
if ping -c 3 -W 5 8.8.8.8 &> /dev/null; then
|
||||
check_pass "Internet connectivity working"
|
||||
else
|
||||
check_warn "No internet connectivity (check WAN configuration)"
|
||||
fi
|
||||
|
||||
# Check 13: IPv6 connectivity
|
||||
echo "[13/15] Checking IPv6 connectivity..."
|
||||
if ping6 -c 3 -W 5 2001:4860:4860::8888 &> /dev/null; then
|
||||
check_pass "IPv6 connectivity working"
|
||||
else
|
||||
check_warn "No IPv6 connectivity (may be expected)"
|
||||
fi
|
||||
|
||||
# Check 14: AI Agent service
|
||||
echo "[14/15] Checking AI Agent service..."
|
||||
if ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no admin@$AI_AGENT_IP "systemctl is-active orion-agent" 2>/dev/null | grep -q "active"; then
|
||||
check_pass "AI Agent service is running"
|
||||
else
|
||||
check_warn "Cannot verify AI Agent service (SSH required)"
|
||||
fi
|
||||
|
||||
# Check 15: Node exporters
|
||||
echo "[15/15] Checking node exporters..."
|
||||
EXPORTER_COUNT=0
|
||||
for IP in $ROUTER_IP $AI_AGENT_IP $PROXMOX_IP; do
|
||||
if curl -s --connect-timeout 2 "http://$IP:9100/metrics" &> /dev/null; then
|
||||
((EXPORTER_COUNT++))
|
||||
fi
|
||||
done
|
||||
if [ $EXPORTER_COUNT -gt 0 ]; then
|
||||
check_pass "Node exporters running ($EXPORTER_COUNT/3)"
|
||||
else
|
||||
check_warn "No node exporters responding"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Validation Summary"
|
||||
echo "=========================================="
|
||||
echo -e "${GREEN}Passed:${NC} $CHECKS_PASSED"
|
||||
echo -e "${YELLOW}Warnings:${NC} $CHECKS_WARNED"
|
||||
echo -e "${RED}Failed:${NC} $CHECKS_FAILED"
|
||||
echo ""
|
||||
|
||||
if [ $CHECKS_FAILED -eq 0 ] && [ $CHECKS_PASSED -ge 10 ]; then
|
||||
echo -e "${GREEN}✓ Deployment successful!${NC}"
|
||||
echo ""
|
||||
echo "Access Points:"
|
||||
echo " - Proxmox: https://$PROXMOX_IP:8006"
|
||||
echo " - Grafana: http://$AI_AGENT_IP:3000 (admin/orion2025)"
|
||||
echo " - Prometheus: http://$AI_AGENT_IP:9090"
|
||||
echo " - Router SSH: ssh admin@$ROUTER_IP"
|
||||
echo " - iDRAC: https://$IDRAC_IP"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo " 1. Change Grafana password"
|
||||
echo " 2. Setup SSH keys for VMs"
|
||||
echo " 3. Configure backups"
|
||||
echo " 4. Review monitoring dashboards"
|
||||
echo ""
|
||||
exit 0
|
||||
elif [ $CHECKS_FAILED -gt 0 ]; then
|
||||
echo -e "${RED}✗ Deployment has issues${NC}"
|
||||
echo ""
|
||||
echo "Troubleshooting:"
|
||||
echo " - Check Proxmox VM status: qm list"
|
||||
echo " - View VM console: qm console <vmid>"
|
||||
echo " - Check VM logs in Proxmox web UI"
|
||||
echo " - Verify network bridge configuration"
|
||||
echo " - See ORION_HYBRID_ARCHITECTURE.md troubleshooting section"
|
||||
echo ""
|
||||
exit 1
|
||||
else
|
||||
echo -e "${YELLOW}⚠ Deployment partially complete${NC}"
|
||||
echo ""
|
||||
echo "Some services are not yet accessible."
|
||||
echo "This may be normal if VMs are still starting up."
|
||||
echo "Wait 5 minutes and run this script again."
|
||||
echo ""
|
||||
exit 0
|
||||
fi
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
#!/bin/bash
|
||||
# ORION Pre-Deployment Validation Script
|
||||
# Checks all prerequisites before starting deployment
|
||||
|
||||
set -e
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
IDRAC_IP="192.168.1.2"
|
||||
REQUIRED_PYTHON_VERSION="3.8"
|
||||
|
||||
echo "=========================================="
|
||||
echo "ORION Pre-Deployment Validation"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
|
||||
# Track overall status
|
||||
CHECKS_PASSED=0
|
||||
CHECKS_FAILED=0
|
||||
CHECKS_WARNED=0
|
||||
|
||||
# Helper functions
|
||||
check_pass() {
|
||||
echo -e "${GREEN}✓${NC} $1"
|
||||
((CHECKS_PASSED++))
|
||||
}
|
||||
|
||||
check_fail() {
|
||||
echo -e "${RED}✗${NC} $1"
|
||||
((CHECKS_FAILED++))
|
||||
}
|
||||
|
||||
check_warn() {
|
||||
echo -e "${YELLOW}⚠${NC} $1"
|
||||
((CHECKS_WARNED++))
|
||||
}
|
||||
|
||||
# Check 1: Python installation
|
||||
echo "[1/10] Checking Python installation..."
|
||||
if command -v python3 &> /dev/null; then
|
||||
PYTHON_VERSION=$(python3 --version | awk '{print $2}')
|
||||
PYTHON_MAJOR=$(echo $PYTHON_VERSION | cut -d. -f1)
|
||||
PYTHON_MINOR=$(echo $PYTHON_VERSION | cut -d. -f2)
|
||||
|
||||
if [ "$PYTHON_MAJOR" -ge 3 ] && [ "$PYTHON_MINOR" -ge 8 ]; then
|
||||
check_pass "Python $PYTHON_VERSION installed"
|
||||
else
|
||||
check_fail "Python $PYTHON_VERSION is too old (need 3.8+)"
|
||||
fi
|
||||
else
|
||||
check_fail "Python 3 not found"
|
||||
fi
|
||||
|
||||
# Check 2: Python requests library
|
||||
echo "[2/10] Checking Python dependencies..."
|
||||
if python3 -c "import requests" 2>/dev/null; then
|
||||
check_pass "Python requests library installed"
|
||||
else
|
||||
check_fail "Python requests library not installed (run: pip3 install requests)"
|
||||
fi
|
||||
|
||||
# Check 3: Network connectivity to iDRAC
|
||||
echo "[3/10] Checking iDRAC connectivity..."
|
||||
if ping -c 1 -W 2 $IDRAC_IP &> /dev/null; then
|
||||
check_pass "iDRAC at $IDRAC_IP is reachable"
|
||||
|
||||
# Try to connect to iDRAC HTTPS
|
||||
if curl -k -s --connect-timeout 5 https://$IDRAC_IP &> /dev/null; then
|
||||
check_pass "iDRAC HTTPS service responding"
|
||||
else
|
||||
check_warn "iDRAC HTTPS not responding (may need to wait for boot)"
|
||||
fi
|
||||
else
|
||||
check_fail "Cannot reach iDRAC at $IDRAC_IP"
|
||||
fi
|
||||
|
||||
# Check 4: Repository files
|
||||
echo "[4/10] Checking repository files..."
|
||||
REQUIRED_FILES=(
|
||||
"deploy-orion-hybrid.py"
|
||||
"orion-config.json"
|
||||
"vm-configs/router-vm/configuration.nix"
|
||||
"vm-configs/ai-agent-vm/configuration.nix"
|
||||
"vm-configs/ai-agent-vm/autonomous_agent.py"
|
||||
)
|
||||
|
||||
for file in "${REQUIRED_FILES[@]}"; do
|
||||
if [ -f "$file" ]; then
|
||||
check_pass "Found $file"
|
||||
else
|
||||
check_fail "Missing $file"
|
||||
fi
|
||||
done
|
||||
|
||||
# Check 5: Proxmox ISO availability
|
||||
echo "[5/10] Checking for Proxmox ISO..."
|
||||
if [ -f "/tmp/proxmox-ve.iso" ] || [ -f "$HOME/Downloads/proxmox-ve*.iso" ]; then
|
||||
check_pass "Proxmox ISO found locally"
|
||||
else
|
||||
check_warn "Proxmox ISO not found (download from: https://www.proxmox.com/en/downloads)"
|
||||
fi
|
||||
|
||||
# Check 6: NixOS ISO availability
|
||||
echo "[6/10] Checking for NixOS ISO..."
|
||||
if [ -f "/tmp/nixos-minimal.iso" ] || [ -f "$HOME/Downloads/nixos-*.iso" ]; then
|
||||
check_pass "NixOS ISO found locally"
|
||||
else
|
||||
check_warn "NixOS ISO not found (will be downloaded during deployment)"
|
||||
fi
|
||||
|
||||
# Check 7: Disk space
|
||||
echo "[7/10] Checking available disk space..."
|
||||
AVAILABLE_SPACE=$(df -BG . | tail -1 | awk '{print $4}' | sed 's/G//')
|
||||
if [ "$AVAILABLE_SPACE" -gt 10 ]; then
|
||||
check_pass "Sufficient disk space ($AVAILABLE_SPACE GB available)"
|
||||
else
|
||||
check_warn "Low disk space ($AVAILABLE_SPACE GB available, recommend 10GB+)"
|
||||
fi
|
||||
|
||||
# Check 8: Git repository status
|
||||
echo "[8/10] Checking git repository..."
|
||||
if git rev-parse --git-dir > /dev/null 2>&1; then
|
||||
check_pass "Git repository initialized"
|
||||
|
||||
# Check for uncommitted changes
|
||||
if git diff-index --quiet HEAD --; then
|
||||
check_pass "No uncommitted changes"
|
||||
else
|
||||
check_warn "You have uncommitted changes"
|
||||
fi
|
||||
else
|
||||
check_warn "Not a git repository (optional)"
|
||||
fi
|
||||
|
||||
# Check 9: SSH key availability
|
||||
echo "[9/10] Checking SSH keys..."
|
||||
if [ -f "$HOME/.ssh/id_rsa.pub" ] || [ -f "$HOME/.ssh/id_ed25519.pub" ]; then
|
||||
check_pass "SSH public key found"
|
||||
else
|
||||
check_warn "No SSH public key found (generate with: ssh-keygen -t ed25519)"
|
||||
fi
|
||||
|
||||
# Check 10: Network configuration
|
||||
echo "[10/10] Checking network configuration..."
|
||||
if ip route | grep -q "192.168.1.0/24"; then
|
||||
check_pass "Management network route configured"
|
||||
else
|
||||
check_warn "No route to 192.168.1.0/24 (iDRAC network)"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Validation Summary"
|
||||
echo "=========================================="
|
||||
echo -e "${GREEN}Passed:${NC} $CHECKS_PASSED"
|
||||
echo -e "${YELLOW}Warnings:${NC} $CHECKS_WARNED"
|
||||
echo -e "${RED}Failed:${NC} $CHECKS_FAILED"
|
||||
echo ""
|
||||
|
||||
if [ $CHECKS_FAILED -eq 0 ]; then
|
||||
echo -e "${GREEN}✓ Ready for deployment!${NC}"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo " 1. Review QUICKSTART_HYBRID.md"
|
||||
echo " 2. Run: python3 deploy-orion-hybrid.py"
|
||||
echo ""
|
||||
exit 0
|
||||
else
|
||||
echo -e "${RED}✗ Please resolve failed checks before deploying${NC}"
|
||||
echo ""
|
||||
echo "Common fixes:"
|
||||
echo " - Install Python 3.8+: apt-get install python3"
|
||||
echo " - Install requests: pip3 install requests"
|
||||
echo " - Check network connectivity to iDRAC"
|
||||
echo " - Ensure all repository files are present"
|
||||
echo ""
|
||||
exit 1
|
||||
fi
|
||||
Loading…
Reference in New Issue