15 KiB

Raw Blame History

Troubleshooting Guide

Common issues, solutions, and recovery procedures for Agent Guardrails Template

Version: 1.0 Last Updated: 2026-02-15

Quick Diagnostics
Common Errors
Debug Mode
Log Analysis
Performance Issues
Recovery Procedures
Getting Help

Quick Diagnostics

Run this checklist to quickly identify common issues:

# 1. Verify installation
curl -s http://localhost:8094/mcp/v1/health | jq .

# 2. Check configuration
ls -la .teams/
ls -la .guardrails/

# 3. Validate tools
python scripts/team_manager.py --validate

# 4. Test MCP connection
curl -s -X POST http://localhost:8094/mcp/v1/message \
  -d '{"jsonrpc":"2.0","method":"tools/list"}' | jq .

Expected Results:

Health endpoint returns {"status": "ok"}
.teams/ and .guardrails/ directories exist
Team manager script exits with code 0
Tools list returns available guardrail tools

Common Errors

MCP Server Connection Failed

Symptoms:

Error: Connection refused (localhost:8094)
Error: Could not connect to MCP server

Causes:

MCP server not running
Wrong port configuration
Firewall blocking connection

Solutions:

Start the MCP server:

python mcp_server.py
# or
./start-mcp-server.sh

Verify port configuration:

# Check if server is listening
netstat -tlnp | grep 8094
# or
lsof -i :8094

Check firewall rules:

# For Linux
sudo ufw allow 8094/tcp
# For macOS
sudo pfctl -e

Team Initialization Failed

Symptoms:

Error: TEAM-001: Team not found
Error: Failed to initialize project

Causes:

Project name contains invalid characters
.teams/ directory does not exist
Permission issues

Solutions:

Verify project name:

# Valid: my-project, project_123, team-alpha
# Invalid: my project, project;rm -rf /

Create required directories:
```
mkdir -p .teams/
chmod 755 .teams/
```

Check permissions:

ls -la .teams/
# Should be writable by current user

Team Size Violation (TEAM-007)

Symptoms:

Error: TEAM-005: Team size violation
Team 7 has 8 members (maximum is 6)

Causes:

Too many members assigned to a team
Batch assignment exceeded limits

Solutions:

Check current team sizes:

curl -X POST http://localhost:8094/mcp/v1/message \
  -d '{
    "jsonrpc":"2.0",
    "method":"tools/call",
    "params":{
      "name":"guardrail_team_size_validate",
      "arguments":{"project_name":"my-project"}
    }
  }'

Remove excess members:

curl -X POST http://localhost:8094/mcp/v1/message \
  -d '{
    "jsonrpc":"2.0",
    "method":"tools/call",
    "params":{
      "name":"guardrail_team_unassign",
      "arguments":{
        "project_name":"my-project",
        "team_id":7,
        "role_name":"Extra Role"
      }
    }
  }'

Rebalance across teams:
- Move members to teams with fewer than 4 members
- Split large teams into multiple smaller teams

Phase Gate Check Failed

Symptoms:

Error: Phase gate requirements not met
Missing deliverables: Architecture Decision Records

Causes:

Required deliverables not complete
Required teams not assigned
Approval not obtained

Solutions:

List gate requirements:

curl -X POST http://localhost:8094/mcp/v1/message \
  -d '{
    "jsonrpc":"2.0",
    "method":"tools/call",
    "params":{
      "name":"guardrail_phase_gate_check",
      "arguments":{
        "project_name":"my-project",
        "from_phase":1,
        "to_phase":2
      }
    }
  }'

Complete missing deliverables:
- See Phase Gates section
- Submit required documents
- Obtain approvals

Verify team assignments:

curl -X POST http://localhost:8094/mcp/v1/message \
  -d '{
    "jsonrpc":"2.0",
    "method":"tools/call",
    "params":{
      "name":"guardrail_team_status",
      "arguments":{
        "project_name":"my-project",
        "phase":"Phase 1"
      }
    }
  }'

Role Already Assigned

Symptoms:

Error: TEAM-004: Person already assigned
Role 'Technical Lead' already has 'Alice Johnson' assigned

Causes:

Attempted to assign to an occupied role
Duplicate assignment in batch script

Solutions:

Unassign current person:

curl -X POST http://localhost:8094/mcp/v1/message \
  -d '{
    "jsonrpc":"2.0",
    "method":"tools/call",
    "params":{
      "name":"guardrail_team_unassign",
      "arguments":{
        "project_name":"my-project",
        "team_id":7,
        "role_name":"Technical Lead"
      }
    }
  }'

Assign new person:

curl -X POST http://localhost:8094/mcp/v1/message \
  -d '{
    "jsonrpc":"2.0",
    "method":"tools/call",
    "params":{
      "name":"guardrail_team_assign",
      "arguments":{
        "project_name":"my-project",
        "team_id":7,
        "role_name":"Technical Lead",
        "person":"New Lead Name"
      }
    }
  }'

API Key Authentication Failed

Symptoms:

Error: AUTH-001: Authentication required
Error: AUTH-002: Invalid API key

Causes:

Missing Authorization header
Expired or revoked API key
Incorrect API key format

Solutions:

Verify header format:

curl -H "Authorization: Bearer YOUR_API_KEY" ...

Generate new API key:
```
python scripts/generate_api_key.py
```

Check key permissions:

python scripts/verify_api_key.py YOUR_API_KEY

Debug Mode

Enable debug mode to get detailed logging for troubleshooting.

Enable Debug Logging

Option 1: Environment Variable

export MCP_DEBUG=1
export MCP_LOG_LEVEL=debug
python mcp_server.py

Option 2: Configuration File

// .mcp/config.json
{
  "logging": {
    "level": "debug",
    "file": ".mcp/mcp.log",
    "console": true
  }
}

Option 3: Command Line Flag

python mcp_server.py --debug --log-file .mcp/debug.log

Debug Output Examples

Normal Operation:

[DEBUG] Received request: tools/call
[DEBUG] Tool: guardrail_team_list
[DEBUG] Parameters: {"project_name": "my-project"}
[DEBUG] Execution time: 45ms
[INFO] Response sent successfully

Error Condition:

[DEBUG] Received request: tools/call
[DEBUG] Tool: guardrail_team_assign
[DEBUG] Parameters: {"team_id": 99, ...}
[ERROR] Validation failed: Invalid team_id
[ERROR] Error code: TEAM-002
[DEBUG] Stack trace:
  File "scripts/team_manager.py", line 45, in validate_team
    raise InvalidTeamError(f"Team {team_id} not found")
[INFO] Error response sent

Log Analysis

Log File Locations

Component	Log File	Description
MCP Server	`.mcp/mcp.log`	Main server logs
Team Manager	`.mcp/team_manager.log`	Team operations
Validation	`.mcp/validation.log`	Guardrail checks
Audit	`.mcp/audit.log`	Security events

Log Format

[TIMESTAMP] [LEVEL] [COMPONENT] Message

Example:

2026-02-15 14:32:15 [INFO] [MCP] Server started on port 8094
2026-02-15 14:32:18 [DEBUG] [TEAM] Validating team assignment
2026-02-15 14:32:18 [ERROR] [VALID] TEAM-002: Invalid team ID

Common Log Patterns

Startup Issues:

# Check for port binding errors
grep "Address already in use" .mcp/mcp.log

# Check for permission denied
grep "Permission denied" .mcp/mcp.log

# Check for missing files
grep "No such file" .mcp/mcp.log

Authentication Issues:

# Find failed authentication attempts
grep "AUTH-" .mcp/audit.log

# List unauthorized access attempts
grep "403\|Unauthorized" .mcp/audit.log

Performance Issues:

# Find slow requests
grep "slow\|timeout" .mcp/mcp.log

# High response times
awk '/Execution time/ && $NF > 1000 {print}' .mcp/mcp.log

Log Rotation

Configure automatic log rotation to prevent disk space issues:

# Add to crontab
crontab -e

# Rotate logs daily at midnight
0 0 * * * /usr/sbin/logrotate /etc/logrotate.d/mcp

logrotate configuration:

# /etc/logrotate.d/mcp
/mnt/ollama/git/agent-guardrails-template/.mcp/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 644 user user
}

Performance Issues

Slow Team Operations

Symptoms:

Team list takes > 5 seconds
Batch assignments timeout
Phase gate checks are slow

Diagnosis:

# Check response times
time curl -s -X POST http://localhost:8094/mcp/v1/message \
  -d '...team_list...'

# Monitor server resources
htop
iostat -x 1

Solutions:

Enable caching:

{
  "cache": {
    "enabled": true,
    "ttl": 30
  }
}

Optimize batch operations:

# Use parallel processing
python scripts/batch_execute.py --parallel 8

Increase timeouts:
```
curl --max-time 30 ...
```

High Memory Usage

Symptoms:

MCP server using > 500MB RAM
System swapping
Out of memory errors

Diagnosis:

# Check memory usage
ps aux | grep mcp_server
free -h

# Monitor over time
watch -n 1 'ps -o pid,rss,cmd -p $(pgrep -f mcp_server)'

Solutions:

Limit cache size:

{
  "cache": {
    "max_size": 100,
    "ttl": 30
  }
}

Restart server periodically:

# Add to cron
0 */6 * * * systemctl restart mcp-server

Profile memory usage:

python -m memory_profiler mcp_server.py

Rate Limiting

Symptoms:

Error: RATE-001: Rate limit exceeded
Retry after 60 seconds

Solutions:

Implement backoff:

import time
import random

def with_backoff(func, max_retries=5):
    for i in range(max_retries):
        try:
            return func()
        except RateLimitError:
            wait = (2 ** i) + random.random()
            time.sleep(wait)
    raise MaxRetriesExceeded()

Use batch endpoints:

# Instead of multiple single calls
python scripts/batch_execute.py --file operations.json

Request limit increase: Contact support to increase rate limits for your use case.

Recovery Procedures

Restore from Backup

Prerequisites:

Backup files in .teams/backups/
Valid project configuration

Steps:

# 1. Stop MCP server
pkill -f mcp_server

# 2. Backup current state
cp -r .teams/ .teams/emergency-backup-$(date +%Y%m%d)

# 3. Restore from backup
cp .teams/backups/project-backup-20260214.json .teams/my-project.json

# 4. Restart server
python mcp_server.py

# 5. Verify restoration
curl -X POST http://localhost:8094/mcp/v1/message \
  -d '{"jsonrpc":"2.0","method":"tools/call","params":{"name":"guardrail_team_list","arguments":{"project_name":"my-project"}}}'

Reset Project State

Warning: This will remove all team assignments and reset to initial state.

# 1. Archive current state
mv .teams/my-project.json .teams/my-project-$(date +%Y%m%d).json.bak

# 2. Re-initialize project
curl -X POST http://localhost:8094/mcp/v1/message \
  -d '{
    "jsonrpc":"2.0",
    "method":"tools/call",
    "params":{
      "name":"guardrail_team_init",
      "arguments":{"project_name":"my-project"}
    }
  }'

# 3. Re-assign team members from backup reference

Repair Corrupted Configuration

Symptoms:

Error: Invalid JSON in team configuration
Error: PROJ-002: Project configuration missing

Steps:

# 1. Validate JSON syntax
python -m json.tool .teams/my-project.json > /dev/null

# 2. If invalid, try to recover
python scripts/repair_config.py .teams/my-project.json

# 3. If recovery fails, restore from backup
cp .teams/backups/my-project.json .teams/my-project.json

Emergency Rollback

Use when critical errors occur during batch operations:

#!/bin/bash
# emergency_rollback.sh

PROJECT_NAME="$1"
BACKUP_FILE=".teams/backups/${PROJECT_NAME}-pre-batch.json"

if [ ! -f "$BACKUP_FILE" ]; then
    echo "No backup found for $PROJECT_NAME"
    exit 1
fi

echo "Rolling back $PROJECT_NAME..."
cp "$BACKUP_FILE" ".teams/${PROJECT_NAME}.json"
echo "Rollback complete."

echo "Verifying..."
curl -s -X POST http://localhost:8094/mcp/v1/message \
  -d "{\"jsonrpc\":\"2.0\",\"method\":\"tools/call\",\"params\":{\"name\":\"guardrail_team_list\",\"arguments\":{\"project_name\":\"$PROJECT_NAME\"}}}"

Getting Help

Self-Service Resources

Documentation:
- TEAM_TOOLS.md - Tool reference
- AGENT_GUARDRAILS.md - Safety protocols
- TEAM_STRUCTURE.md - Team definitions
Error Code Lookup:
- See TEAM_TOOLS.md Error Handling section
- Search logs for error codes
Community:
- GitHub Issues: Report bugs and feature requests
- Discussions: Ask questions, share solutions

Support Channels

Issue Type	Channel	Response Time
Critical outage	Email: oncall@example.com	15 minutes
Security issue	security@example.com	4 hours
Feature request	GitHub Issues	2-3 days
General question	GitHub Discussions	1-2 days

Required Information

When reporting issues, include:

Error message: Full text or screenshot
Log snippets: Relevant sections from .mcp/mcp.log
Steps to reproduce: Minimal example

Environment:

python --version
uname -a
git log --oneline -1

Configuration: (sanitized)

cat .mcp/config.json | grep -v password

Diagnostic Script

Run this script to gather diagnostic information:

#!/bin/bash
# diagnose.sh - Collect diagnostic information

echo "=== Agent Guardrails Diagnostics ==="
echo "Date: $(date)"
echo ""

echo "=== System Information ==="
uname -a
echo ""

echo "=== Python Version ==="
python --version
echo ""

echo "=== MCP Server Status ==="
pgrep -f mcp_server || echo "MCP server not running"
echo ""

echo "=== Recent Log Entries ==="
tail -50 .mcp/mcp.log 2>/dev/null || echo "No log file found"
echo ""

echo "=== Configuration ==="
ls -la .teams/ 2>/dev/null || echo "No .teams directory"
ls -la .guardrails/ 2>/dev/null || echo "No .guardrails directory"
echo ""

echo "=== Health Check ==="
curl -s http://localhost:8094/mcp/v1/health 2>/dev/null || echo "Health check failed"
echo ""

echo "=== Diagnostics Complete ==="

Last Updated: 2026-02-15 Version: 1.0

15 KiB Raw Blame History

Troubleshooting Guide

Table of Contents

Quick Diagnostics

Common Errors

MCP Server Connection Failed

Team Initialization Failed

Team Size Violation (TEAM-007)

Phase Gate Check Failed

Role Already Assigned

API Key Authentication Failed

Debug Mode

Enable Debug Logging

Debug Output Examples

Log Analysis

Log File Locations

Log Format

Common Log Patterns

Log Rotation

Performance Issues

Slow Team Operations

High Memory Usage

Rate Limiting

Recovery Procedures

Restore from Backup

Reset Project State

Repair Corrupted Configuration

Emergency Rollback

Getting Help

Self-Service Resources

Support Channels

Required Information

Diagnostic Script

15 KiB

Raw Blame History