Inky MBTA Tracker or IMT is a multi-purpose, async Python application tracking the Massachusetts Bay Transit Authority. It relies on Redis for data storage in conjunction with inky-display for e-inky display functionality. Additionally, it provides a comprehensive API using FastAPI which serves ryanwallace.cloud.

You need a .env file. Here are the supported options:

# MBTA API Configuration
AUTH_TOKEN=<MBTA_API_TOKEN> # https://www.mbta.com/developers/v3-api

# Application Configuration
IMT_CONFIG=./config.json # optional to specify a different config file
IMT_REDIS_BACKUP_TIME=21:50 # set to backup every night at this time, you are responsible for cleaning up backups

# prometheus exporter configuration
IMT_PROMETHEUS_ENABLE="true"  # disabled by default
IMT_PROM_PORT="8000"

# use these settings for real time self-monitoring health checks via prometheus
IMT_PROMETHEUS_JOB="imt_dev"
IMT_PROMETHEUS_ENDPOINT="http://prometheus.local"
IMT_METRIC_NAME="mbta_server_side_events:rate5m"

# Logging
IMT_LOG_FILE=./logs/inky.log # optional to also log to a file
LOG_LEVEL=INFO # DEBUG, INFO, WARNING, ERROR supported log levels
IMT_COLOR=true  # to log with colors

# monitoring with Pyroscope (optional)
IMT_PYROSCOPE_ENABLED=true
IMT_PYROSCOPE_HOST=http://pyroscope.local
IMT_PYROSCOPE_NAME=inky-local

# Redis Configuration (required)
IMT_REDIS_ENDPOINT=127.0.0.1
IMT_REDIS_PORT=6379
IMT_REDIS_PASSWORD=mbta # change this!

# MQTT Configuration
IMT_ENABLE_MQTT=true/false
IMT_MQTT_HOST=127.0.0.1
IMT_MQTT_USER=username
IMT_MQTT_PASS=mqtt_pass # change this!

# API Timeouts
IMT_API_REQUEST_TIMEOUT=30 # API request timeout in seconds
IMT_TRACK_PREDICTION_TIMEOUT=15 # Track prediction timeout in seconds

# Feature Flags
IMT_RATE_LIMITING_ENABLED=true # Enable/disable rate limiting
IMT_SSE_ENABLED=true # Enable/disable Server-Sent Events

# machine learning configuration for track predictions
IMT_ML=true
KERAS_BACKEND=jax # Jax is set by default and works well even on low spec hardware
IMT_ML_CONF_GAMMA=1.3
IMT_ML_CONF_HIST_WEIGHT=0.3
IMT_BAYES_ALPHA=0.65
IMT_ML_SAMPLE_PCT=0.1
IMT_ML_REPLACE_DELTA=0.05
IMT_ML_COMPARE=true # enable to compare pattern based predictions versus machine-learning approaches
IMT_ML_MIN_CONFIDENCE=0.25
IMT_ML_STORE_CONFIDENCE=0.25
IMT_CR_LIMIT=10 # how many departures to fetch from each commuter rail station for date prediction requests

# precaching
IMT_PRECACHE_MAX_ATTEMPTS=5
IMT_PRECACHE_BASE_BACKOFF=1.0
IMT_PRED_JITTER_MS=200

Prometheus is available at port 8000.

New metric: imt_track_predictions_ml_wins — counts how often the live ML vs. historical/pattern comparison chose the ML result. Metric labels are station_id, route_id, and instance. This is exposed alongside the other imt_ metrics on the same Prometheus endpoint.

You can use my dashboard JSON for a Grafana dashboard combining the Prom metrics & a Loki datasource for logs.

Spin up the local Redis server using the provided docker compose file.

Then run task run to start up the tracker.

Note: On Linux/macOS, uvloop is enabled automatically when available for faster asyncio performance. If not present, the default loop is used.

The ML ensemble for track prediction is optional and off by default. To enable it:

• Environment

  • IMT_ML=true to enable the ML worker
  • KERAS_BACKEND=jax to use the Keras-on-Jax backend

• Behavior at a glance

  • ML runs asynchronously; the predictor never blocks waiting for ML.
  • The ML worker writes a TrackPrediction to Redis and clears any corresponding negative_… cache key to avoid suppression.
  • Predictions with confidence below 25% are filtered out (hard floor).
  • Confidence combines margin‑based probability, optional sharpening, historical accuracy, and (when available) Bayes fusion with latest ML outputs.

• Tuning (env vars)

  • IMT_ML_CONF_GAMMA (default 1.3): probability sharpening factor used for display confidence.
  • IMT_CONF_HIST_WEIGHT (default 0.3): blend weight for historical accuracy into display confidence.
  • IMT_BAYES_ALPHA (default 0.65): weight for pattern vs. ML in Bayes fusion.
  • IMT_ML_SAMPLE_PCT (default 0.10): % of successful traditional predictions also queued to ML for exploration.
  • IMT_ML_REPLACE_DELTA (default 0.05): minimum ML confidence improvement to overwrite an existing non‑ML prediction.
  • IMT_ML_COMPARE (default false): when true the predictor will attempt a short live comparison between the traditional/pattern result and the ML ensemble and choose the higher-confidence result. This is opt-in to avoid blocking the prediction flow.
  • IMT_ML_COMPARE_WAIT_MS (default 200): maximum time in milliseconds to wait for a recent ML result when IMT_ML_COMPARE is enabled. If no ML result is produced within this window the predictor falls back to the traditional result and continues (the ML worker still runs asynchronously).

• Observability

  • New Prometheus metric: imt_track_predictions_ml_wins — counts how often the live ML vs. pattern comparison chose the ML result. Labels: station_id, route_id, instance.

The track prediction system analyzes historical track assignments to predict future track assignments for MBTA commuter rail trains before they are officially announced. This helps solve the "mad scramble" problem at major stations like South Station and North Station.

1. Data Collection: The system automatically captures track assignments from the MBTA API when trains arrive at stations
2. Pattern Analysis: Historical data is analyzed to identify patterns based on:
   • Headsign and destination
   • Time of day and day of week
   • Direction of travel
   • Route information
3. Prediction Generation: Before official track announcements, the system generates predictions with confidence scores
4. Validation: Predictions are validated against actual track assignments to improve accuracy over time

Track predictions are automatically integrated into the existing display system:

• API Access: Use the track prediction API to get detailed information:

  # Get predictions for a station
  curl http://localhost:8080/predictions/place-sstat
  
  # Get prediction statistics
  curl http://localhost:8080/stats/place-sstat/CR-Providence
  
  # Get historical data
  curl http://localhost:8080/historical/place-sstat/CR-Providence?days=30

Add this environment variable to enable track prediction features:

# Optional: Port for track prediction API (default: 8080)
IMT_TRACK_API_PORT=8080

To enable automatic track prediction precaching, add these fields to your config.json:

{
  "enable_track_predictions": true,
  "track_prediction_routes": ["CR-Worcester", "CR-Providence"],
  "track_prediction_stations": ["place-sstat", "place-north", "place-bbsta"],
  "track_prediction_interval_hours": 2
}

• enable_track_predictions: Boolean to enable/disable precaching (default: false)
• track_prediction_routes: List of commuter rail routes to precache (optional, defaults to all CR routes)
• track_prediction_stations: List of station IDs to precache for (optional, defaults to major stations)
• track_prediction_interval_hours: Hours between precaching runs (default: 2)

Note that you will also need to add the following stations to your configuration:

  "stops": [
    {
      "stop_id": "place-north",
      "show_on_display": false,
      "transit_time_min": 1,
    },
    {
      "stop_id": "place-sstat",
      "show_on_display": false,
      "transit_time_min": 1
    },
    {
      "stop_id": "place-bbsta",
      "show_on_display": false,
      "transit_time_min": 1
    }
  ]

This creates real time departure/arrival trackers for these stations which is required to generate track predictions but it doesn't show them on the Inky display or in MQTT.

flowchart TD
    subgraph inky-mbta-tracker
    B(schedule and prediction workers)
    D{{queue}}
    C(queue processor)
    TP[Track Predictor]
    AI[AI Summarizer]
    AQ{{AI Job Queue}}
    end
    B --> D
    C --> D
    B --> TP
    TP --> B
    B --> AI
    AI --> AQ
    AQ --> AI

    C -->E@{ shape: cyl, label: "Redis"}
    C -->F@{ shape: bow-rect, label: "MQTT" }
    TP --> E
    AI --> E
    AI --> O[Ollama API]

    G@{ shape: curv-trap, label: "inky-display" } -->|reads| E
    H[/Home Assistant/] -->|reads| F
    I[Track Prediction API] --> TP
    J[AI Summary API] --> AI

At a base level, this project makes use of the MBTA V3 API, especially the streaming API for predictions to setup individual workers for stops which are configured by the user. Optionally, a user can request static schedules via the configuration file (explained below), and there is behavior that will retrieve static schedules if no real-time predictions are available for a stop. From anecdotal experience the V3 streaming API appears to start dropping events after several hours without any errors reported. Therefore, each stop watcher thread making use of the streaming API will restart after 1-3 hours which is cleanly handled through the Python Async APIs.

This project works with inky-display which checks the Redis server a few times a minute to refresh the display. Additionally, the departures can be integrated with Home Assistant MQTT Sensors to create a real-time departure dashboard.