Handy

A free, open source, and extensible speech-to-text application that works completely offline.

Handy is a cross-platform desktop application built with Tauri (Rust + React/TypeScript) that provides simple, privacy-focused speech transcription. Press a shortcut, speak, and have your words appear in any text field—all without sending your voice to the cloud.

Why Handy?

Handy was created to fill the gap for a truly open source, extensible speech-to-text tool. As stated on handy.computer:

Free: Accessibility tooling belongs in everyone's hands, not behind a paywall
Open Source: Together we can build further. Extend Handy for yourself and contribute to something bigger
Private: Your voice stays on your computer. Get transcriptions without sending audio to the cloud
Simple: One tool, one job. Transcribe what you say and put it into a text box

Handy isn't trying to be the best speech-to-text app—it's trying to be the most forkable one.

How It Works

Press a configurable keyboard shortcut to start/stop recording (or use push-to-talk mode)
Speak your words while the shortcut is active
Release and Handy processes your speech using Whisper
Get your transcribed text pasted directly into whatever app you're using

The process is entirely local:

Silence is filtered using VAD (Voice Activity Detection) with Silero
Transcription uses your choice of models:
- Whisper models (Small/Medium/Turbo/Large) with GPU acceleration when available
- Parakeet V3 - CPU-optimized model with excellent performance and automatic language detection
Works on Windows, macOS, and Linux

Quick Start

Installation

Download the latest release from the releases page or the website
Install the application following platform-specific instructions
Launch Handy and grant necessary system permissions (microphone, accessibility)
Configure your preferred keyboard shortcuts in Settings
Start transcribing!

Development Setup

For detailed build instructions including platform-specific requirements, see BUILD.md.

Architecture

Handy is built as a Tauri application combining:

Frontend: React + TypeScript with Tailwind CSS for the settings UI
Backend: Rust for system integration, audio processing, and ML inference
Core Libraries:
- whisper-rs: Local speech recognition with Whisper models
- transcription-rs: CPU-optimized speech recognition with Parakeet models
- cpal: Cross-platform audio I/O
- vad-rs: Voice Activity Detection
- rdev: Global keyboard shortcuts and system events
- rubato: Audio resampling

Debug Mode

Handy includes an advanced debug mode for development and troubleshooting. Access it by pressing:

macOS: Cmd+Shift+D
Windows/Linux: Ctrl+Shift+D

Known Issues & Current Limitations

This project is actively being developed and has some known issues. We believe in transparency about the current state:

Major Issues (Help Wanted)

Whisper Model Crashes:

Whisper models crash on certain system configurations (Windows and Linux)
Does not affect all systems - issue is configuration-dependent
- If you experience crashes and are a developer, please help to fix and provide debug logs!

Wayland Support (Linux):

Limited or no support for Wayland display server
On Wayland the clipboard-based paste options (Clipboard (CTRL+V) / Clipboard (Shift+Insert)) copy the transcription once, then try to run wtype (preferred) or dotool to fire the paste keystroke. Install one of these tools to let Handy drive the compositor-friendly paste shortcut; otherwise it falls back to Enigo-generated key events, which may not work on Wayland.

Linux Notes

The recording overlay is disabled by default on Linux (Overlay Position: None) because certain compositors treat it as the active window. When the overlay is visible it can steal focus, which prevents Handy from pasting back into the application that triggered transcription. If you enable the overlay anyway, be aware that clipboard-based pasting might fail or end up in the wrong window.
If you are having trouble with the app, running with the environment variable WEBKIT_DISABLE_DMABUF_RENDERER=1 may help
You can manage global shortcuts outside of Handy and still control the app via signals. Sending SIGUSR2 to the Handy process toggles recording on/off, which lets Wayland window managers or other hotkey daemons keep ownership of keybindings. Example (Sway):
```
bindsym $mod+o exec pkill -USR2 -n handy
```
pkill here simply delivers the signal—it does not terminate the process.

Platform Support

macOS (both Intel and Apple Silicon)
x64 Windows
x64 Linux

System Requirements/Recommendations

The following are recommendations for running Handy on your own machine. If you don't meet the system requirements, the performance of the application may be degraded. We are working on improving the performance across all kinds of computers and hardware.

For Whisper Models:

macOS: M series Mac, Intel Mac
Windows: Intel, AMD, or NVIDIA GPU
Linux: Intel, AMD, or NVIDIA GPU
- Ubuntu 22.04, 24.04

For Parakeet V3 Model:

CPU-only operation - runs on a wide variety of hardware
Minimum: Intel Skylake (6th gen) or equivalent AMD processors
Performance: ~5x real-time speed on mid-range hardware (tested on i5)
Automatic language detection - no manual language selection required

Roadmap & Active Development

We're actively working on several features and improvements. Contributions and feedback are welcome!

In Progress

Debug Logging:

Adding debug logging to a file to help diagnose issues

macOS Keyboard Improvements:

Support for Globe key as transcription trigger
A rewrite of global shortcut handling for MacOS, and potentially other OS's too.

Opt-in Analytics:

Collect anonymous usage data to help improve Handy
Privacy-first approach with clear opt-in

Settings Refactoring:

Cleanup and refactor settings system which is becoming bloated and messy
Implement better abstractions for settings management

Tauri Commands Cleanup:

Abstract and organize Tauri command patterns
Investigate tauri-specta for improved type safety and organization

Troubleshooting

Manual Model Installation (For Proxy Users or Network Restrictions)

If you're behind a proxy, firewall, or in a restricted network environment where Handy cannot download models automatically, you can manually download and install them. The URLs are publicly accessible from any browser.

Step 1: Find Your App Data Directory

Open Handy settings
Navigate to the About section
Copy the "App Data Directory" path shown there, or use the shortcuts:
- macOS: Cmd+Shift+D to open debug menu
- Windows/Linux: Ctrl+Shift+D to open debug menu

The typical paths are:

macOS: ~/Library/Application Support/com.pais.handy/
Windows: C:\Users\{username}\AppData\Roaming\com.pais.handy\
Linux: ~/.config/com.pais.handy/

Step 2: Create Models Directory

Inside your app data directory, create a models folder if it doesn't already exist:

# macOS/Linux
mkdir -p ~/Library/Application\ Support/com.pais.handy/models

# Windows (PowerShell)
New-Item -ItemType Directory -Force -Path "$env:APPDATA\com.pais.handy\models"

Step 3: Download Model Files

Download the models you want from below

Whisper Models (single .bin files):

Small (487 MB): https://blob.handy.computer/ggml-small.bin
Medium (492 MB): https://blob.handy.computer/whisper-medium-q4_1.bin
Turbo (1600 MB): https://blob.handy.computer/ggml-large-v3-turbo.bin
Large (1100 MB): https://blob.handy.computer/ggml-large-v3-q5_0.bin

Parakeet Models (compressed archives):

V2 (473 MB): https://blob.handy.computer/parakeet-v2-int8.tar.gz
V3 (478 MB): https://blob.handy.computer/parakeet-v3-int8.tar.gz

Step 4: Install Models

For Whisper Models (.bin files):

Simply place the .bin file directly into the models directory:

{app_data_dir}/models/
├── ggml-small.bin
├── whisper-medium-q4_1.bin
├── ggml-large-v3-turbo.bin
└── ggml-large-v3-q5_0.bin

For Parakeet Models (.tar.gz archives):

Extract the .tar.gz file
Place the extracted directory into the models folder
The directory must be named exactly as follows:
- Parakeet V2: parakeet-tdt-0.6b-v2-int8
- Parakeet V3: parakeet-tdt-0.6b-v3-int8

Final structure should look like:

{app_data_dir}/models/
├── parakeet-tdt-0.6b-v2-int8/     (directory with model files inside)
│   ├── (model files)
│   └── (config files)
└── parakeet-tdt-0.6b-v3-int8/     (directory with model files inside)
    ├── (model files)
    └── (config files)

Important Notes:

For Parakeet models, the extracted directory name must match exactly as shown above
Do not rename the .bin files for Whisper models—use the exact filenames from the download URLs
After placing the files, restart Handy to detect the new models

Step 5: Verify Installation

Restart Handy
Open Settings → Models
Your manually installed models should now appear as "Downloaded"
Select the model you want to use and test transcription

How to Contribute

Check existing issues at github.com/cjpais/Handy/issues
Fork the repository and create a feature branch
Test thoroughly on your target platform
Submit a pull request with clear description of changes
Join the discussion - reach out at contact@handy.computer

The goal is to create both a useful tool and a foundation for others to build upon—a well-patterned, simple codebase that serves the community.

Sponsors

We're grateful for the support of our sponsors who help make Handy possible:

Related Projects

Handy CLI - The original Python command-line version
handy.computer - Project website with demos and documentation

License

MIT License - see LICENSE file for details.

Acknowledgments

Whisper by OpenAI for the speech recognition model
whisper.cpp and ggml for amazing cross-platform whisper inference/acceleration
Silero for great lightweight VAD
Tauri team for the excellent Rust-based app framework
Community contributors helping make Handy better

"Your search for the right speech-to-text tool can end here—not because Handy is perfect, but because you can make it perfect for you."

Uh oh!

License

cjpais/Handy

About

Topics

Resources

License

Contributing

Uh oh!

Stars

Watchers

Forks

Releases 39

Sponsor this project

Uh oh!

Packages 0

Contributors 38

Uh oh!

Languages

Packages