Skip to content

weka/wtop

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

30 Commits
 
 
 
 
 
 
 
 

Repository files navigation

WTOP

License Issues

WTOP (WEKA TOP) is an open source, real-time terminal-based monitoring interface for WEKA cluster performance metrics, built with Python 3.6.8+ and urwid.

Features

  • Real-time monitoring of WEKA cluster performance metrics
  • Host-based rows with configurable columns
  • Adjustable refresh rate (default: 1 second)
  • Dynamic column management - add, remove, and cycle through different metrics
  • Portable - works on Linux hosts with Python 3.6.8+
  • Clean TUI interface with color-coded metrics
  • Distributed as a Python script and as a single-file binary

Requirements

  • Python 3.6.8 or greater
  • Linux host with WEKA CLI tools installed
  • Access to execute weka status and weka stats realtime commands

Quick Start

Choose one method.

Clone and run

git clone https://github.com/weka/wtop
cd wtop
./wtop

Download binary directly

curl -L -o wtop https://raw.githubusercontent.com/weka/wtop/main/wtop
chmod +x wtop
./wtop

Installation from Source

Clone the repository and install dependencies:

git clone https://github.com/weka/wtop.git
cd wtop/src
pip install -r requirements.txt
python3 wtop.py

Dependencies

  • urwid
  • json (Python standard library)
  • csv (Python standard library)

Install with:

pip install urwid

Usage

Run wtop in your terminal.

  • Default mode shows client (frontend) metrics.
  • Switch to backend view with the m key.

Command Line Options

The application runs with default settings and can be controlled entirely through the TUI interface.

TUI Controls

Navigation & Control

  • q - Quit the application
  • h - Show help dialog
  • +/- - Increase/decrease refresh rate
  • Ctrl+C - Quit (alternative)

Column Management

  • 1-9 - Cycle through available metrics for columns 1-9
  • a - Add new column (with next available metric)
  • r - Remove last column

Available Metrics

The TUI monitors the following metrics from WEKA:

Real-time Stats (from weka stats realtime)

  • Node ID - Node identifier
  • Hostname - Host name
  • Writes/s - Write operations per second
  • Write Throughput - Write bandwidth in bytes per second (only for clients)
  • Write Latency (µs) - Write latency in microseconds
  • Reads/s - Read operations per second
  • Read Throughput - Read bandwidth in bytes per second (only for clients)
  • Read Latency (µs) - Read latency in microseconds
  • Ops/s - Total operations per second
  • CPU % - CPU utilization percentage
  • L6 Recv - Level 6 receive metrics
  • L6 Sent - Level 6 send metrics
  • OBS Upload - Object storage upload metrics
  • OBS Download - Object storage download metrics
  • RDMA Recv - RDMA receive metrics
  • RDMA Sent - RDMA send metrics
  • Roles - Node roles
  • Mode - Operation mode

Status Data (from weka status -J)

  • Total Ops - Total operations count
  • Total Reads - Total read operations
  • Total Writes - Total write operations
  • Total Bytes Read - Total bytes read
  • Total Bytes Written - Total bytes written

Default Layout

The application starts with these default columns:

  1. Hostname
  2. CPU %
  3. Ops/s
  4. Reads/s
  5. Writes/s
  6. Read Latency (µs)
  7. Write Latency (µs)

Customization

Adding Columns

  • Press a to add a new column with the next available metric
  • The new column will appear on the right side

Removing Columns

  • Press r to remove the rightmost column
  • At least one column (hostname) will always remain visible

Cycling Metrics

  • Press 1-9 to cycle through available metrics for that specific column
  • Each column can display any available metric
  • No duplicate metrics are allowed across columns

Adjusting Refresh Rate

  • Press + to increase refresh rate (max: 10 seconds)
  • Press - to decrease refresh rate (min: 0.5 seconds)

Data Sources

The TUI executes two WEKA commands to gather data:

  1. weka status -J - Provides JSON output with cluster-wide activity statistics
  2. weka stats realtime -F mode=client -f csv -R -v - Provides CSV output with per-host real-time performance metrics

Error Handling

  • Command failures are gracefully handled with error messages in the status bar
  • Network issues or WEKA service problems are reported without crashing the application
  • Invalid data is displayed as "N/A" rather than causing errors

Troubleshooting

Common Issues

  1. "Command not found: weka"

    • Ensure WEKA CLI tools are installed and in PATH
    • Verify you have appropriate permissions to execute WEKA commands
  2. "Permission denied"

    • Check that you have access to execute WEKA commands
    • Verify your user has appropriate cluster access
  3. No data displayed

    • Check weka status
    • Verify network connectivity to cluster nodes
    • Check weka local status
  4. Display issues

    • Ensure your terminal supports the required colors
    • Try resizing your terminal window

Performance Considerations

  • Refresh rate: Lower refresh rates (0.5-1s) provide more real-time data but increase system load
  • Column count: More columns increase display complexity but provide more information
  • Network impact: Each refresh executes WEKA commands, consider cluster load

Contributing

This is a focused monitoring tool for WEKA clusters. Contributions are welcome for:

  • Bug fixes
  • Performance improvements
  • Additional metric support
  • Enhanced error handling

License

Licensed under the Apache License, Version 2.0

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages