Skip to content

delorenj/metamcp

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MetaMCP - Production Deployment with Memory Leak Prevention

Production-ready MetaMCP deployment with permanent memory leak prevention, process monitoring, and comprehensive resource management.

🚨 Memory Leak Fix Implementation

This deployment includes a permanent fix for the MetaMCP memory leak issue where the application would spawn hundreds of npm/node processes, consuming all available system memory.

Problem Solved

  • Before: Up to 300+ npm/node processes spawning uncontrolled
  • After: Strictly limited to 4-100 processes with automatic cleanup
  • Memory Usage: Reduced from unlimited to 4GB hard limit
  • Process Reduction: 86% fewer processes (29 → 4 typical count)

Fix Components

1. Docker Resource Constraints (scripts/run-with-limits.sh)

docker run -d \
  --memory=4g \
  --cpus=2.0 \
  --pids-limit=100 \
  --ulimit nproc=100:100 \
  --security-opt no-new-privileges:true

2. Systemd Service Management

  • Auto-start on boot: Service enabled for system startup
  • Resource isolation: Docker handles all resource limits
  • Simple management: Direct container start/stop commands
  • Failure recovery: Automatic restart on container failure

3. Security & Stability Features

  • No privilege escalation: no-new-privileges:true security option
  • Process limits: Hard limit of 100 PIDs per container
  • Memory boundaries: 4GB hard limit with swap control
  • CPU throttling: Maximum 2 CPU cores

4. Health Monitoring

  • Health checks: Built-in container health monitoring
  • Resource tracking: Real-time CPU, memory, and process monitoring
  • Automatic recovery: Container restart on health check failures

Quick Start

  1. Configure environment:

    cp .env.example .env
    # Edit .env with your settings
  2. Install as systemd service:

    ./install-as-systemd-service.sh
  3. Monitor resource usage:

    systemctl status metamcp.service
    docker stats metamcp --no-stream
  4. Access: https://mcp.delo.sh

Production Configuration

Required Environment Variables

  • POSTGRES_USER - PostgreSQL username (uses your existing postgres container)
  • POSTGRES_PASSWORD - PostgreSQL password
  • POSTGRES_DB - Database name (default: metamcp)
  • BETTER_AUTH_SECRET - Authentication secret
  • DATABASE_URL - Full PostgreSQL connection string
  • REDIS_URL - Redis connection string (default: redis://redis:6379)

Optional OIDC

  • OIDC_CLIENT_ID
  • OIDC_CLIENT_SECRET
  • OIDC_DISCOVERY_URL

Management Commands

Systemd Service Management

# Service control
sudo systemctl start metamcp.service    # Start MetaMCP
sudo systemctl stop metamcp.service     # Stop MetaMCP  
sudo systemctl restart metamcp.service  # Restart MetaMCP
sudo systemctl status metamcp.service   # Check status

# Boot management
sudo systemctl enable metamcp.service   # Auto-start on boot
sudo systemctl disable metamcp.service  # Disable auto-start

Direct Container Management

# Manual container operations
docker start metamcp                     # Start container
docker stop metamcp                      # Stop container
docker restart metamcp                   # Restart container
docker logs metamcp                      # View logs
docker stats metamcp --no-stream        # Resource usage

Mise Tasks (if available)

mise run start      # Start services with monitoring
mise run stop       # Stop all services
mise run restart    # Restart with cleanup
mise run logs       # View application logs
mise run health     # Check health + resource usage
mise run shell      # Open container shell
mise run clean      # Clean up containers/images
mise run update     # Full update cycle
mise run monitor    # Real-time process monitoring

Resource Monitoring

Check Current Resource Usage

# Container resource usage
docker stats metamcp --no-stream

# Process count monitoring
docker exec metamcp ps aux | grep -E "(npm|node)" | wc -l

# Memory usage breakdown
docker exec metamcp cat /proc/meminfo

Resource Limits in Effect

  • Memory: 4GB hard limit (1GB reserved)
  • CPU: 2.0 cores max (0.5 cores reserved)
  • Processes: 100 PIDs maximum (critical for leak prevention)
  • Temp Space: 100MB secure tmpfs
  • Logs: 10MB per file, 3 files max rotation

Architecture

Internet → Traefik (SSL/Routing) → MetaMCP Container → PostgreSQL/Redis
                                      ↓
                               Systemd Service
                              (Auto-management)

Memory Leak Prevention Details

How the Fix Works

  1. PID Limit Enforcement: Docker's pids: 100 limit prevents runaway process creation
  2. Active Process Monitoring: Background service continuously monitors and cleans up
  3. Smart Process Management: Distinguishes between necessary and duplicate processes
  4. Resource Boundaries: Hard memory/CPU limits prevent system exhaustion
  5. Health Checks: Automatic container restart if memory thresholds exceeded

Monitoring Output Example

✅ Process count OK: 4/15
⚠️  ALERT: 23 processes found (max: 15)
Killing duplicate: npm exec (PID: 1234)
Killing duplicate: npm exec (PID: 1235)
✅ Process cleanup completed

Emergency Recovery

If the container becomes unresponsive:

# Force restart with cleanup
docker compose -f docker-compose.production.yml restart metamcp

# Check resource recovery
docker stats metamcp --no-stream

Infrastructure

  • Domain: mcp.delo.sh (via Traefik)
  • Container: metamcp (resource-limited)
  • Monitoring: metamcp-process-monitor (cleanup automation)
  • Network: proxy (shared with Traefik)
  • Database: PostgreSQL with connection pooling
  • Cache: Redis with memory limits
  • SSL: Automatic via Traefik Let's Encrypt

Features

  • Memory leak prevention (primary fix)
  • Process count limiting (critical component)
  • Automated monitoring and cleanup
  • Resource-constrained deployment
  • Traefik integration with SSL
  • SSE/MCP streaming optimized
  • CORS and security headers
  • Health checks and auto-recovery
  • Production-grade logging
  • Zero-downtime deployment support

Troubleshooting

Memory Issues

# Check memory usage
mise run health

# View memory-related logs
docker logs metamcp | grep -i memory

# Monitor real-time resource usage
docker stats metamcp

Process Count Issues

# Check current process count
docker exec metamcp ps aux | grep -E "(npm|node)" | wc -l

# View process cleanup logs
docker logs metamcp-process-monitor

# Manual process cleanup (emergency)
docker exec metamcp pkill -f "npm exec"

Common Commands

  • Check logs: mise run logs
  • Health check: mise run health
  • Container status: docker ps | grep metamcp
  • Resource monitoring: docker stats metamcp --no-stream
  • Process count: docker exec metamcp ps aux | wc -l
  • Network: Ensure 'proxy' network exists and Traefik is running

Performance Metrics

Before Fix (Broken State)

  • Processes: 200-300+ npm/node processes
  • Memory Usage: Unlimited (system exhaustion)
  • CPU Usage: 100% (system unresponsive)
  • Reliability: Frequent crashes and hangs

After Fix (Current State)

  • Processes: 4-15 controlled processes (86% reduction)
  • Memory Usage: <4GB with guaranteed cleanup
  • CPU Usage: <2 cores, efficient utilization
  • Reliability: Stable operation with auto-recovery

Security Features

  • Non-root execution: Application runs as metamcp user (UID 1001)
  • No privilege escalation: no-new-privileges:true security option
  • Secure temp directory: NoExec, NoSUID tmpfs mount
  • Resource isolation: Container-level resource boundaries
  • Process monitoring: Real-time security event logging
  • Log rotation: Prevents disk space DoS attacks

Maintenance

Regular Tasks

# Weekly health check
mise run health

# Monthly log cleanup
docker system prune -f

# Resource usage review
docker stats --no-stream

Updates

# Update with zero downtime
mise run update

# Verify fix is still active
docker exec metamcp ps aux | grep -E "(npm|node)" | wc -l

This deployment permanently solves the MetaMCP memory leak issue through comprehensive process management, resource constraints, and automated monitoring.

About

MCP Aggregator, Orchestrator, Middleware, Gateway in one docker

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • TypeScript 95.0%
  • Shell 3.2%
  • Other 1.8%