Skip to content

ThomasBury/mlops-uv

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

7 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Mastering Python Project Management with UV: MLOps

How to use

You have two options to follow along with this guide:

  1. Build the project from scratch by manually setting up the structure and copy-pasting the provided code base (src and tests folders).

  2. Clone the repository, install dependencies using the command uv sync, and run the commands explained below directly to:

    • Execute the test suite
    • Build the Docker image
    • Modify and test GitHub Actions

Ship it like it's hot! 🚒πŸ”₯

Introduction

MLOps (Machine Learning Operations) is all about bringing DevOps principles into machine learning, making model deployment, versioning, and monitoring more efficient. However, managing dependencies, ensuring reproducibility, and streamlining deployments can be a major headache for ML/DS teams.

That's where UV comes inβ€”a fast, modern package manager that simplifies dependency management, build processes, and CI/CD for Python projects.

In this article, we'll explore how UV can enhance MLOps workflows through AceBet, a mock-up FastAPI app that predicts the winner of an ATP match (for demonstration purposes onlyβ€”don’t bet your savings on it!). We'll cover:

  • Setting up a UV-based MLOps project
  • Managing dependencies and lockfiles
  • Automating CI/CD with GitHub Actions
  • Building and deploying with Docker

Prerequisites

Make sure to read:


πŸ“¦ Initializing an MLOps Project with UV

When working on an MLOps project, structuring your codebase properly is crucial. We'll start by setting up a packaged application using UV:

uv init --package acebet

A packaged application follows the src-based structure, where the source code is contained within a dedicated package directory (src/acebet). This approach is beneficial for:

βœ… Large applications with multiple modules
βœ… Projects that need to be distributed (e.g., PyPI packages, CLI tools)
βœ… Better namespace isolation, preventing import conflicts
βœ… Improved testability and modularity

Example Directory Structure

acebet/
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ acebet/
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”œβ”€β”€ data_prep.py
β”‚   β”‚   β”œβ”€β”€ model.py
β”‚   β”‚   β”œβ”€β”€ predict.py
β”‚   β”‚   β”œβ”€β”€ api.py
β”‚   β”‚   └── utils.py
β”œβ”€β”€ tests/
β”‚   β”œβ”€β”€ test_model.py
β”œβ”€β”€ pyproject.toml
└── README.md

This structure ensures: βœ” Encapsulation: The application is a proper Python package, avoiding accidental name conflicts.
βœ” Reusability: Can be installed via pip install . or published to PyPI.
βœ” Cleaner Imports: Enforces absolute imports (from acebet.utils import foo) instead of relative imports.
βœ” Better CI/CD Support: Easier to package and distribute in Docker, PyPI, or GitHub Actions.

Regular App vs. Packaged App

πŸ‘‰ For quick scripts or internal projects? Use a regular application.
πŸ‘‰ For scalable, maintainable, and deployable projects? Use a packaged application.

Since AceBet is a full MLOps project, we’ll use a packaged application.


πŸ”§ Managing Dependencies with UV

Installing Core Dependencies

Once your project is initialized, install the necessary dependencies for developing AceBet, including FastAPI and machine learning libraries like Scikit-learn:

uv add fastapi scikit-learn pandas lightgbm

UV will automatically resolve versions and install the required packages.

Creating a Lockfile for Reproducibility

One of UV's key advantages is ensuring dependency reproducibility with a lockfile. This guarantees that all environments (local, staging, production) use the same dependency versions.

Once you're satisfied with the initial codebase, generate a lockfile:

uv lock

Or, if you want to sync all dependencies in one go:

uv sync

This process ensures version consistency across environments, which is an essential practice in MLOps.


πŸ›  Adding Testing Dependencies & Running Tests

Testing is just as important as model accuracy in MLOps. UV provides three different ways to manage testing tools (e.g., pytest):

Method Command Use Case
Adding as Dev Dependency uv add --dev pytest When pytest is part of your Python project
Running Temporarily uvx pytest tests When you only need to run it occasionally
Installing Persistently uv tool install pytest When you need pytest in Docker or as a global CLI

For MLOps best practices, we add testing dependencies:

uv add --dev pytest

To run tests:

uv run pytest tests

πŸ‘‰ Best Practice: Explicitly list all required dependencies in pyproject.toml for consistent test environments.


πŸš€ Automating CI/CD with GitHub Actions

A CI/CD pipeline ensures your models and applications remain production-ready. UV makes GitHub Actions setup seamless.

A simple workflow to run tests on every commit to main:

name: Testing
on:
  push:
    branches:
      - "main"
jobs:
  uv-example:
    name: Python
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install UV
        uses: astral-sh/setup-uv@v5
      - name: Install the project
        run: uv sync --all-extras --dev
      - name: Run tests
        run: uv run pytest tests

βœ… Installs UV
βœ… Syncs dependencies
βœ… Runs unit tests using Pytest


🐳 Building a Docker Image with UV

A well-built Docker image ensures consistent deployment across environments. UV simplifies containerization.

Dockerfile for AceBet

FROM python:3.12-slim

# Install UV
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

# Copy the application into the container
COPY . /app

# Set working directory
WORKDIR /app

# Install dependencies
RUN uv sync --frozen --no-cache

# Run the FastAPI app
CMD ["/app/.venv/bin/fastapi", "run", "src/acebet/api.py", "--port", "80", "--host", "0.0.0.0"]

For production-ready builds, use a multi-stage Docker build to keep the image lightweight.


🌟 Why UV for MLOps?

Feature UV πŸš€ (Rust-based) Poetry πŸ› οΈ (Python-based)
⏩ Performance & Speed ⚑ 8-10x faster than pip & pip-tools (80-115x with cache). Ideal for CI/CD. 🐒 Slower dependency resolution and package installation. Can be a bottleneck in large projects.
πŸ”§ Dependency Management Uses a global cache to avoid redundant installations. Faster and more efficient. Uses a custom resolver, but can be slow for large projects.
πŸ“¦ Environment Handling Manages Python versions natively (no need for pyenv). Creates fast, lightweight virtual environments. Supports virtual environments but requires external tools for Python version management.
🐳 Docker Efficiency βœ… Smaller images & faster builds. Simplifies deployment by combining Python & dependency management. ❌ Larger image footprint due to reliance on multiple tools. Longer build times.
πŸš€ CI/CD Pipelines βœ… Faster builds due to Rust-based optimizations. Reduces install time in GitHub Actions, Docker, and cloud environments. ❌ Slower CI/CD performance due to Python-based dependency resolution.
πŸ”„ Migration & Ecosystem βœ… Follows PEP standards closely, making migration easier. Less tightly integrated, offering flexibility. ❌ More opinionated ecosystem, making migration or integration with existing tools more complex.
πŸ”‘ Authentication & Config βœ… Simplifies authentication using environment variables. Ensures cross-platform consistency. ❌ Configuration can be complex, requiring additional setup for cross-platform consistency.
πŸ“œ Unified Tooling βœ… All-in-one tool: Handles package management, virtual environments, and Python versions. No need for extra tools. ❌ Depends on multiple tools (e.g., pyenv for Python versioning), increasing setup complexity.
πŸ—οΈ Build & Deployment βœ… Optimized for modern workflows. Generates smaller wheels and installs faster in Docker, Kubernetes, and cloud deployments. ❌ Traditional package builds, not as optimized for modern DevOps/MLOps pipelines.

🎯 Key Takeaways

  • If you need speed, lightweight builds, and a streamlined DevOps workflow, UV is the better choice. πŸš€
  • If you prefer a well-established but slower tool with more integrated features, Poetry remains viable. πŸ› οΈ
  • For MLOps & CI/CD, UV's speed and efficiency make it the preferred option. πŸ’‘

🎯 Conclusion

By integrating UV into your MLOps workflow, you get a fast, reproducible, and efficient setup for managing dependencies, testing, and deployment.

With AceBet, we demonstrated how to: βœ”οΈ Initialize a UV-based project
βœ”οΈ Manage dependencies & lockfiles
βœ”οΈ Automate testing with GitHub Actions
βœ”οΈ Build Docker images for deployment

Give UV a tryβ€”it might just replace Pip and Poetry in your workflow! πŸš€

Happy Coding!

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •