Changes from upstream at cvat-ai/cvat

Added convenience compose files and envvars to more easily run annotation models (nuclio serverless functions) either locally or on a separate server
Added deploy_cpu.ps1 and deploy_gpu.ps1 for easier Windows deployment of models
Added support for running SAM2 (Credit to hashJoe/cvat:feature/sam2, added code to build UI container with SAM2 components)
Fixed CUDA version issues in yolov7
Fixed dependency issues in faster-rcnn
Turned on auto-save by default

The following CVAT versions are available - see below for instructions on how to update to newer versions:

v2.32.0-sam2

Running this fork of CVAT

Requirements:

Docker & Docker Compose

# Clone our fork
git clone https://github.com/dk-teknologisk-jahs/cvat.git
cd cvat

# Switch to branch with changes
git switch v2.32.0-sam2

# Check .env and change any variables as necessary, such as CVAT_NUCLIO_HOST

# Build & Start CVAT using the provided compose files (should use compose.yaml by default, add -f compose.yaml if not)
docker compose up -d --build --force-recreate

Commands used to create this fork (Bash)

# Clone our fork
git clone https://github.com/dk-teknologisk-jahs/cvat.git
cd cvat

# Add required remotes
git remote add upstream https://github.com/cvat-ai/cvat.git
git remote add hashJoe https://github.com/hashJoe/cvat.git

# Fetch all tags from upstream
git fetch upstream --tags

# Find latest stable version
git tag -l | sort -V
# Let's assume v2.32.0 is the latest stable

# Create a branch based on this stable version
git checkout -b v2.32.0-sam2 v2.32.0

# Merge the SAM2 feature
git merge hashJoe/feature/sam2

# Add necessary files to run CVAT with SAM2 and either local or remote nuclio server
cat <<'EOF' >compose.yaml
include:
  - path:
    - ./docker-compose.yml
    #- ./docker-compose.dev.yml # should not be necessary (warning: will expose DB on port 5432 unless modified in compose.override.yaml)
    - ./components/serverless/docker-compose.serverless.yml
    - ./compose.override.yaml
EOF
cat <<'EOF' >compose.override.yaml
services:
  # cvat_db: # For modifying port mapping of DB to avoid conflicts with existing PostgreSQL instances
  #   ports: [] # Remove mappings (if using docker-compose.dev.yml)
  #   #  - '5433:5432' # Or instead change host port 5432-->5433

  cvat_server:
    environment:
      CVAT_SERVERLESS: 1
      CVAT_NUCLIO_HOST: ${CVAT_NUCLIO_HOST:-localhost}
      CVAT_NUCLIO_INVOKE_METHOD: 'dashboard'

  cvat_worker_annotation:
    environment:
      CVAT_NUCLIO_HOST: ${CVAT_NUCLIO_HOST:-localhost}
      CVAT_NUCLIO_INVOKE_METHOD: 'dashboard'

  cvat_ui: # adapted from docker-compose.dev.yml, needed for custom plugins with UI components such as SAM2
    build:
      context: .
      args:
        http_proxy:
        https_proxy:
        no_proxy:
        socks_proxy:
        CLIENT_PLUGINS: ${CLIENT_PLUGINS}
      dockerfile: Dockerfile.ui
EOF
cat <<'EOF' >.env
CLIENT_PLUGINS=plugins/sam2
CVAT_HOST=tjorn.local # CVAT has some issues with 404 erros if we don't set this
CVAT_SERVERLESS=1
CVAT_NUCLIO_HOST=172.17.155.175 # set to localhost if using local nuclio
CVAT_NUCLIO_INVOKE_METHOD=dashboard
EOF
git add -f compose.yaml compose.override.yaml .env
git commit -m 'Added compose.yml, compose.override.yaml and .env'

# Add deploy_cpu.ps1 and deploy_gpu.ps1 for Windows deployment of models
cat <<'EOF' >serverless/deploy_cpu.ps1
# deploy_cpu.ps1 - Windows version of deploy_cpu.sh
# Sample commands to deploy nuclio functions on CPU

# Enable stopping on errors
$ErrorActionPreference = "Stop"

# Get the script directory
$SCRIPT_DIR = $PSScriptRoot
$FUNCTIONS_DIR = if ($args[0]) { $args[0] } else { $SCRIPT_DIR }

# Enable Docker BuildKit
$env:DOCKER_BUILDKIT = 1

# Build base OpenVINO image
docker build -t cvat.openvino.base "$SCRIPT_DIR\openvino\base"

# Create the CVAT project
nuctl create project cvat --platform local

# Find and deploy all function.yaml files
Get-ChildItem -Path $FUNCTIONS_DIR -Recurse -Filter "function.yaml" | ForEach-Object {
    $func_config = $_.FullName
    $func_root = Split-Path -Parent $func_config
    $func_parent_dir = Split-Path -Parent $func_root

    # Calculate relative path similar to Linux's realpath
    $func_rel_path = $func_parent_dir.Replace($SCRIPT_DIR, "").TrimStart("\")

    # Build Docker image if Dockerfile exists
    if (Test-Path -Path "$func_root\Dockerfile") {
        $docker_tag = "cvat." + ($func_rel_path -replace "\\", ".") + ".base"
        docker build -t $docker_tag $func_root
    }

    Write-Host "Deploying $func_rel_path function..."
    nuctl deploy --project-name cvat --path $func_root `
        --file $func_config --platform local `
        --env CVAT_FUNCTIONS_REDIS_HOST=cvat_redis_ondisk `
        --env CVAT_FUNCTIONS_REDIS_PORT=6666 `
        --platform-config '{\"attributes\": {\"network\": \"cvat_cvat\"}}'
}

# List deployed functions
nuctl get function --platform local
EOF
cat <<'EOF' >serverless/deploy_gpu.ps1
# deploy_gpu.ps1 - Windows version of deploy_gpu.sh
# Sample commands to deploy nuclio functions on GPU

# Enable stopping on errors
$ErrorActionPreference = "Stop"

# Get the script directory
$SCRIPT_DIR = $PSScriptRoot
$FUNCTIONS_DIR = if ($args[0]) { $args[0] } else { $SCRIPT_DIR }

# Create the CVAT project
nuctl create project cvat --platform local

# Find and deploy all function-gpu.yaml files
Get-ChildItem -Path $FUNCTIONS_DIR -Recurse -Filter "function-gpu.yaml" | ForEach-Object {
    $func_config = $_.FullName
    $func_root = Split-Path -Parent $func_config
    $func_parent_dir = Split-Path -Parent $func_root

    # Calculate relative path similar to Linux's realpath
    $func_rel_path = $func_parent_dir.Replace($SCRIPT_DIR, "").TrimStart("\")

    Write-Host "Deploying $func_rel_path function..."
    nuctl deploy --project-name cvat --path $func_root `
        --file $func_config --platform local `
        --env CVAT_FUNCTIONS_REDIS_HOST=cvat_redis_ondisk `
        --env CVAT_FUNCTIONS_REDIS_PORT=6666 `
        --platform-config '{\"attributes\": {\"network\": \"cvat_cvat\"}}'
}

# List deployed functions
nuctl get function --platform local
EOF
git add -f serverless/deploy_cpu.ps1 serverless/deploy_gpu.ps1
git commit -m 'Added deploy_cpu.ps1 and deploy_gpu.ps1 for Windows'

# Compatibility fix for yolov7
sed -i 's/baseImage: nvidia\/cuda:12\.6\.3-cudnn-runtime-ubuntu22\.04/baseImage: nvidia\/cuda:12\.4\.1-cudnn-runtime-ubuntu22\.04/g' serverless/onnx/WongKinYiu/yolov7/nuclio/function-gpu.yaml
git commit -m 'Downgrade cuda of yolov7 from 2.6.3 to 2.4.1 for better compatibility'

# Fix dependency issues with faster-rcnn
sed -i '/^  build:/a\    commands:\n      - pip install msgpack # Added to fix bug - https://github.com/nuclio/nuclio/issues/3472' /home/jahs/GitHub/cvat/serverless/tensorflow/faster_rcnn_inception_v2_coco/nuclio/function*.yaml
git commit -m 'Fix dependency issues with faster-rcnn'

# Turn on auto-save by default
sed -i 's/autoSave: false,/autoSave: true,/g' cvat-ui/src/reducers/settings-reducer.ts
git commit -m 'Turn on auto-save by default'

# Resolve any conflicts if necessary
# Then commit and push
git push -u origin v2.32.0-sam2

Updating to a new CVAT version

When CVAT releases a new version (e.g. v2.45.0), you can either just merge the changes since the last stable version:

# If you have made changes to the .env etc, make sure you are on the previous stable
# version w. SAM2 (v2.32.0-sam2 in this case), so you can stash the local changes
git checkout v2.32.0-sam2 # replace with the branch you are using now
git stash push -u -m "local_changes"

# Create a new branch from the previous stable version w. SAM2 (v2.32.0-sam2 in this case)
git fetch upstream --tags
git checkout -b v2.45.0-sam2 v2.32.0-sam2 # replace with the new CVAT version you want to use and the branch you are using now

# Now merge the changes since previous stable version
git merge upstream/v2.45.0 # replace with the new CVAT version you want to use

# Pop the stashed changes if necessary
git stash apply stash^{/local_changes}

# Resolve conflicts, test, then optionally push
git push -u origin v2.45.0-sam2 # replace with the new CVAT version you want to use

Or alternatively, start from scratch, reapply the SAM2 changes and add all necessary files:

# If you have made changes to the .env etc, make sure you are on the previous stable
# version w. SAM2 (v2.32.0-sam2 in this case), so you can stash the local changes
git checkout v2.32.0-sam2 # replace with the branch you are using now
git stash push -u -m "local_changes"

# Create a new branch from the new stable version wo. SAM2 (v2.45.0 in this case)
git fetch upstream --tags
git checkout -b v2.45.0-sam2 upstream/v2.45.0 # replace with the new CVAT version you want to use and the branch you are using now

# Now merge the changes from the SAM2 feature branch
git merge hashJoe/feature/sam2

# Add necessary files to run CVAT with SAM2 and either local or remote nuclio server
cat <<'EOF' >compose.yaml
include:
  - path:
    - ./docker-compose.yml
    #- ./docker-compose.dev.yml # should not be necessary (warning: will expose DB on port 5432 unless modified in compose.override.yaml)
    - ./components/serverless/docker-compose.serverless.yml
    - ./compose.override.yaml
EOF
cat <<'EOF' >compose.override.yaml
services:
  # cvat_db: # For modifying port mapping of DB to avoid conflicts with existing PostgreSQL instances
  #   ports: [] # Remove mappings (if using docker-compose.dev.yml)
  #   #  - '5433:5432' # Or instead change host port 5432-->5433

  cvat_server:
    environment:
      CVAT_SERVERLESS: 1
      CVAT_NUCLIO_HOST: ${CVAT_NUCLIO_HOST:-localhost}
      CVAT_NUCLIO_INVOKE_METHOD: 'dashboard'

  cvat_worker_annotation:
    environment:
      CVAT_NUCLIO_HOST: ${CVAT_NUCLIO_HOST:-localhost}
      CVAT_NUCLIO_INVOKE_METHOD: 'dashboard'

  cvat_ui: # adapted from docker-compose.dev.yml, needed for custom plugins with UI components such as SAM2
    build:
      context: .
      args:
        http_proxy:
        https_proxy:
        no_proxy:
        socks_proxy:
        CLIENT_PLUGINS: ${CLIENT_PLUGINS}
      dockerfile: Dockerfile.ui
EOF
cat <<'EOF' >.env
CLIENT_PLUGINS=plugins/sam2
CVAT_HOST=tjorn.local # CVAT has some issues with 404 erros if we don't set this
CVAT_SERVERLESS=1
CVAT_NUCLIO_HOST=172.17.155.175 # set to localhost if using local nuclio
CVAT_NUCLIO_INVOKE_METHOD=dashboard
EOF
git add -f compose.yaml compose.override.yaml .env
git commit -m 'Added compose.yml, compose.override.yaml and .env'

# Add deploy_cpu.ps1 and deploy_gpu.ps1 for Windows deployment of models
cat <<'EOF' >serverless/deploy_cpu.ps1
# deploy_cpu.ps1 - Windows version of deploy_cpu.sh
# Sample commands to deploy nuclio functions on CPU

# Enable stopping on errors
$ErrorActionPreference = "Stop"

# Get the script directory
$SCRIPT_DIR = $PSScriptRoot
$FUNCTIONS_DIR = if ($args[0]) { $args[0] } else { $SCRIPT_DIR }

# Enable Docker BuildKit
$env:DOCKER_BUILDKIT = 1

# Build base OpenVINO image
docker build -t cvat.openvino.base "$SCRIPT_DIR\openvino\base"

# Create the CVAT project
nuctl create project cvat --platform local

# Find and deploy all function.yaml files
Get-ChildItem -Path $FUNCTIONS_DIR -Recurse -Filter "function.yaml" | ForEach-Object {
    $func_config = $_.FullName
    $func_root = Split-Path -Parent $func_config
    $func_parent_dir = Split-Path -Parent $func_root

    # Calculate relative path similar to Linux's realpath
    $func_rel_path = $func_parent_dir.Replace($SCRIPT_DIR, "").TrimStart("\")

    # Build Docker image if Dockerfile exists
    if (Test-Path -Path "$func_root\Dockerfile") {
        $docker_tag = "cvat." + ($func_rel_path -replace "\\", ".") + ".base"
        docker build -t $docker_tag $func_root
    }

    Write-Host "Deploying $func_rel_path function..."
    nuctl deploy --project-name cvat --path $func_root `
        --file $func_config --platform local `
        --env CVAT_FUNCTIONS_REDIS_HOST=cvat_redis_ondisk `
        --env CVAT_FUNCTIONS_REDIS_PORT=6666 `
        --platform-config '{\"attributes\": {\"network\": \"cvat_cvat\"}}'
}

# List deployed functions
nuctl get function --platform local
EOF
cat <<'EOF' >serverless/deploy_gpu.ps1
# deploy_gpu.ps1 - Windows version of deploy_gpu.sh
# Sample commands to deploy nuclio functions on GPU

# Enable stopping on errors
$ErrorActionPreference = "Stop"

# Get the script directory
$SCRIPT_DIR = $PSScriptRoot
$FUNCTIONS_DIR = if ($args[0]) { $args[0] } else { $SCRIPT_DIR }

# Create the CVAT project
nuctl create project cvat --platform local

# Find and deploy all function-gpu.yaml files
Get-ChildItem -Path $FUNCTIONS_DIR -Recurse -Filter "function-gpu.yaml" | ForEach-Object {
    $func_config = $_.FullName
    $func_root = Split-Path -Parent $func_config
    $func_parent_dir = Split-Path -Parent $func_root

    # Calculate relative path similar to Linux's realpath
    $func_rel_path = $func_parent_dir.Replace($SCRIPT_DIR, "").TrimStart("\")

    Write-Host "Deploying $func_rel_path function..."
    nuctl deploy --project-name cvat --path $func_root `
        --file $func_config --platform local `
        --env CVAT_FUNCTIONS_REDIS_HOST=cvat_redis_ondisk `
        --env CVAT_FUNCTIONS_REDIS_PORT=6666 `
        --platform-config '{\"attributes\": {\"network\": \"cvat_cvat\"}}'
}

# List deployed functions
nuctl get function --platform local
EOF
git add -f serverless/deploy_cpu.ps1 serverless/deploy_gpu.ps1
git commit -m 'Added deploy_cpu.ps1 and deploy_gpu.ps1 for Windows'

# Compatibility fix for yolov7
sed -i 's/baseImage: nvidia\/cuda:12\.6\.3-cudnn-runtime-ubuntu22\.04/baseImage: nvidia\/cuda:12\.4\.1-cudnn-runtime-ubuntu22\.04/g' serverless/onnx/WongKinYiu/yolov7/nuclio/function-gpu.yaml
git commit -m 'Downgrade cuda of yolov7 from 2.6.3 to 2.4.1 for better compatibility'

# Fix dependency issues with faster-rcnn
sed -i '/^  build:/a\    commands:\n      - pip install msgpack # Added to fix bug - https://github.com/nuclio/nuclio/issues/3472' /home/jahs/GitHub/cvat/serverless/tensorflow/faster_rcnn_inception_v2_coco/nuclio/function*.yaml
git commit -m 'Fix dependency issues with faster-rcnn'

# Turn on auto-save by default
sed -i 's/autoSave: false,/autoSave: true,/g' cvat-ui/src/reducers/settings-reducer.ts
git commit -m 'Turn on auto-save by default'

# Pop the stashed changes if necessary
 apply stash^{/local_changes}

# Resolve conflicts, test, then optionally push
git push -u origin v2.45.0-sam2 # replace with the new CVAT version you want to use

This approach gives a clean upgrade path while maintaining the customizations.

Example of running SAM2 on CPU as serverless function on the same PC as CVAT:

Run on the same PC to set up the SAM2 serverless plugin for CVAT, running on the CPU (change to deploy_gpu script to run on GPU).

For Linux PCs (Bash)

Requirements:

Docker & Docker Compose
NVIDIA Container Toolkit (if using deploy_gpu)

# Set variables
CVAT_ROOT_DIR=/home/jahs/GitHub/cvat # Change this to your CVAT directory
NUCLIO_BIN_DIR=/home/jahs/GitHub/bin # Change this to the directory where you want to store nuctl
USE_NUCLIO_VERSION=1.14.0 # Should match nuclio version used by CVAT (1.13.22 and 1.14.1 both failed for CVAT v2.32.0, so sticking with 1.14.0 for now)

# Create directory for nuctl if it doesn't exist
mkdir -p "$NUCLIO_BIN_DIR"

# Download nuctl executable
mkdir -p "$NUCLIO_BIN_DIR"
cd "$NUCLIO_BIN_DIR"
wget "https://github.com/nuclio/nuclio/releases/download/$USE_NUCLIO_VERSION/nuctl-$USE_NUCLIO_VERSION-linux-amd64"
chmod +x "nuctl-$USE_NUCLIO_VERSION-linux-amd64"
ln -sf "nuctl-$USE_NUCLIO_VERSION-linux-amd64" nuctl
chmod +x nuctl

# Navigate to CVAT root directory, switch to correct branch and pull latest changes
cd "$CVAT_ROOT_DIR"
git switch v2.32.0-sam2 # replace with the branch you are using now
git pull

# Add nuctl to PATH temporarily for this session
export PATH="$NUCLIO_BIN_DIR:$PATH"

# Create & start SAM2 serverless function on CPU
./serverless/deploy_cpu.sh serverless/pytorch/facebookresearch/sam2

For Windows PCs (PowerShell) - Not Tested

Requirements:

Docker & Docker Compose
- Make sure to use the WSL2 backend for Docker Desktop
NVIDIA Container Toolkit (if using deploy_gpu)

# Set variables
$CVAT_ROOT_DIR = "C:\GitHub\cvat"  # Change this to your CVAT directory
$NUCLIO_BIN_DIR = "C:\GitHub\bin"  # Change this to the directory where you want to store nuctl
$USE_NUCLIO_VERSION = "1.14.0"     # Should match nuclio version used by CVAT (1.13.22 and 1.14.1 both failed for CVAT v2.32.0, so sticking with 1.14.0 for now)

# Create directory for nuctl if it doesn't exist
if (-not (Test-Path -Path $NUCLIO_BIN_DIR)) {
    New-Item -ItemType Directory -Path $NUCLIO_BIN_DIR -Force
}

# Download nuctl executable
$nuctl_url = "https://github.com/nuclio/nuclio/releases/download/$USE_NUCLIO_VERSION/nuctl-$USE_NUCLIO_VERSION-windows-amd64"
$nuctl_path = Join-Path -Path $NUCLIO_BIN_DIR -ChildPath "nuctl"

Write-Host "Downloading nuctl from $nuctl_url..."
Invoke-WebRequest -Uri $nuctl_url -OutFile $nuctl_path
icacls "$nuctl_path" /grant:r "$env:USERNAME:(RX)"

# Navigate to CVAT root directory, switch to correct branch and pull latest changes
Set-Location -Path $CVAT_ROOT_DIR
git switch v2.32.0-sam2 # replace with the branch you are using now
git pull

# Add nuctl to PATH temporarily for this session
$env:PATH = "$NUCLIO_BIN_DIR;$env:PATH"

# Create & start SAM2 serverless function on CPU
.\serverless\deploy_cpu.ps1 "$CVAT_ROOT_DIR\serverless\pytorch\facebookresearch\sam2"

This setup assumes that CVAT and the SAM2 serverless function are running on the same machine. Ensure that the .env file is configured correctly with CVAT_NUCLIO_HOST=localhost.

Remember to stop and restart CVAT:

cd "$CVAT_ROOT_DIR"

# Stop and remove existing containers (should use compose.yaml by default, add -f compose.yaml if not)
docker compose down

# Rebuild and restart containers (should use compose.yaml by default, add -f compose.yaml if not)
docker compose up -d --build --force-recreate

Example of running SAM2 on GPU as serverless function on the same PC as CVAT:

Run on the same PC to set up the SAM2 serverless plugin for CVAT, running on the GPU (change to deploy_cpu script to run on CPU).

For Windows PCs (WSL)

Requirements:

Docker Desktop. Follow this link to install.
WSL with a distribution installed. Follow this link to install.

Open Powershell and enter the WSL shell (in this example I am using Ubuntu 24.04):

wsl -d Ubuntu-24.04

In the WSL shell enter the following commands:

NOTE: change CVAT_ROOT_DIR and NUCLIO_BIN_DIR to the correct path with reference to your system. CVAT_ROOT_DIR should be the path where the cvat source code was downloaded in the setup step.

# Set variables
CVAT_ROOT_DIR=/mnt/c/GitHub/cvat # Change this to your CVAT directory
NUCLIO_BIN_DIR=/mnt/c/GitHub/bin # Change this to the directory where you want to store nuctl
USE_NUCLIO_VERSION=1.14.0 # Should match nuclio version used by CVAT (1.13.22 and 1.14.1 both failed for CVAT v2.32.0, so sticking with 1.14.0 for now)

# Create directory for nuctl if it doesn't exist
mkdir -p "$NUCLIO_BIN_DIR"

# Download nuctl executable
mkdir -p "$NUCLIO_BIN_DIR"
cd "$NUCLIO_BIN_DIR"
wget "https://github.com/nuclio/nuclio/releases/download/$USE_NUCLIO_VERSION/nuctl-$USE_NUCLIO_VERSION-linux-amd64"
chmod +x "nuctl-$USE_NUCLIO_VERSION-linux-amd64"
ln -sf "nuctl-$USE_NUCLIO_VERSION-linux-amd64" nuctl
chmod +x nuctl

# Navigate to CVAT root directory, switch to correct branch and pull latest changes
cd "$CVAT_ROOT_DIR"
git switch v2.32.0-sam2 # replace with the branch you are using now
git pull

# Add nuctl to PATH temporarily for this session
export PATH="$NUCLIO_BIN_DIR:$PATH"


# Create & start SAM2 serverless function on CPU
./serverless/deploy_gpu.sh serverless/pytorch/facebookresearch/sam2

This setup assumes that CVAT and the SAM2 serverless function are running on the same machine.

Ensure that the .env file is configured correctly:

CVAT_NUCLIO_HOST=${IP_LOCAL_COMPUTER}. Substitute ${IP_LOCAL_COMPUTER} with the IP address of your computer (with localhost it will not work when you access the CVAT WebUI from outside the local PC).
CVAT_HOST with the domain name or the IP address of your computer.

Remember to stop and restart CVAT:

cd "$CVAT_ROOT_DIR"

# Stop and remove existing containers (should use compose.yaml by default, add -f compose.yaml if not)
docker compose down

# Rebuild and restart containers (should use compose.yaml by default, add -f compose.yaml if not)
docker compose up -d --build --force-recreate

Example of running SAM2 on GPU as serverless function on separate PC (w. IP: 172.17.155.175):

Run on separate server from CVAT server:

For Linux PCs (Bash)

Requirements:

Docker & Docker Compose
NVIDIA Container Toolkit (if using deploy_gpu)

# Set variables
CVAT_ROOT_DIR=/home/kristian/GitHub/cvat # Change this to your CVAT directory
NUCLIO_BIN_DIR=/home/kristian/GitHub/bin # Change this to the directory where you want to store nuctl
USE_NUCLIO_VERSION=1.14.0 # Should match nuclio version used by CVAT (1.13.22 and 1.14.1 both failed for CVAT v2.32.0, so sticking with 1.14.0 for now)
USE_NUCLIO_ADDRESS=172.17.155.175 # set to actual IP of GPU server (hostname might work, didn't in my case though)

# Create directory for nuctl if it doesn't exist
mkdir -p "$NUCLIO_BIN_DIR"

# Download nuctl executable
mkdir "$NUCLIO_BIN_DIR"
cd "$NUCLIO_BIN_DIR"
wget "https://github.com/nuclio/nuclio/releases/download/$USE_NUCLIO_VERSION/nuctl-$USE_NUCLIO_VERSION-linux-amd64"
chmod +x "nuctl-$USE_NUCLIO_VERSION-linux-amd64"
ln -sf "nuctl-$USE_NUCLIO_VERSION-linux-amd64" nuctl
chmod +x nuctl

# Clone our CVAT fork, navigate to CVAT root directory, switch to correct branch and pull latest changes
git clone https://github.com/dk-teknologisk-jahs/cvat.git "$CVAT_ROOT_DIR"
cd "$CVAT_ROOT_DIR"
git switch v2.32.0-sam2 # replace with the branch you are using now
git pull

# Add nuctl to PATH temporarily for this session
export PATH="$NUCLIO_BIN_DIR:$PATH"

# Create & start SAM2 (or any other, checkout the builtin models) serverless function on GPU
PATH="$NUCLIO_BIN_DIR:$PATH" ./serverless/deploy_gpu.sh serverless/pytorch/facebookresearch/sam2

# Start nuclio dashboard server to monitor and provide access to the function over HTTP API
# It might be possible to get it working without this (directly access node port), but I couldn't make it work
docker run -d \
  -p 8070:8070 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --network cvat_cvat \
  --name nuclio-dashboard \
  -e NUCLIO_DASHBOARD_EXTERNAL_IP_ADDRESSES=$USE_NUCLIO_ADDRESS \
  quay.io/nuclio/dashboard:$USE_NUCLIO_VERSION-amd64

For Windows PCs (PowerShell) - Not Tested

Requirements:

Docker & Docker Compose
- Make sure to use the WSL2 backend for Docker Desktop
- If you have issues with binding to the docker daemon socket, try some of the solutions here and please create an issue with any improvement suggestions
Ensure you have installed the NVIDIA Container Toolkit for Windows and configured Docker to use your GPU
You might need to set the execution policy to run scripts: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
Make sure ports are open in Windows Firewall for the nuclio dashboard (8070)

# Set variables
$CVAT_ROOT_DIR = "C:\GitHub\cvat"  # Change this to your CVAT directory
$NUCLIO_BIN_DIR = "C:\GitHub\bin"  # Change this to the directory where you want to store nuctl
$USE_NUCLIO_VERSION = "1.14.0"     # Should match nuclio version used by CVAT (1.13.22 and 1.14.1 both failed for CVAT v2.32.0, so sticking with 1.14.0 for now)
$USE_NUCLIO_ADDRESS = "172.17.155.175" # set to actual IP of GPU server (hostname might work, didn't in my case though)

# Create directory for nuctl if it doesn't exist
if (-not (Test-Path -Path $NUCLIO_BIN_DIR)) {
    New-Item -ItemType Directory -Path $NUCLIO_BIN_DIR -Force
}

# Download nuctl executable
$nuctl_url = "https://github.com/nuclio/nuclio/releases/download/$USE_NUCLIO_VERSION/nuctl-$USE_NUCLIO_VERSION-windows-amd64"
$nuctl_path = Join-Path -Path $NUCLIO_BIN_DIR -ChildPath "nuctl"

Write-Host "Downloading nuctl from $nuctl_url..."
Invoke-WebRequest -Uri $nuctl_url -OutFile $nuctl_path
icacls "$nuctl_path" /grant:r "$env:USERNAME:(RX)"

# Clone our CVAT fork, navigate to CVAT root directory, switch to correct branch and pull latest changes
git clone https://github.com/dk-teknologisk-jahs/cvat.git "$CVAT_ROOT_DIR"
cd "$CVAT_ROOT_DIR"
git switch v2.32.0-sam2 # replace with the branch you are using now
git pull

# Add nuctl to PATH temporarily for this session
$env:PATH = "$NUCLIO_BIN_DIR;$env:PATH"

# Create & start SAM2 (or any other, checkout the builtin models) serverless function on GPU
.\deploy_gpu.ps1 "$CVAT_ROOT_DIR\serverless\pytorch\facebookresearch\sam2"

# Start nuclio dashboard server to monitor and provide access to the function over HTTP API
# It might be possible to get it working without this (directly access node port), but I couldn't make it work
docker run -d \
  -p 8070:8070 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --network cvat_cvat \
  --name nuclio-dashboard \
  -e NUCLIO_DASHBOARD_EXTERNAL_IP_ADDRESSES=$USE_NUCLIO_ADDRESS \
  quay.io/nuclio/dashboard:$USE_NUCLIO_VERSION-amd64

This setup assumes that CVAT and the SAM2 serverless function are running on separate machines. Ensure that the .env file is configured correctly with CVAT_NUCLIO_HOST pointing to the IP address of the machine running the serverless functions. Be aware that you can't use both local nuclio and remote nuclio at the same time.

Original README from here on

Computer Vision Annotation Tool (CVAT)

CVAT is an interactive video and image annotation tool for computer vision. It is used by tens of thousands of users and companies around the world. Our mission is to help developers, companies, and organizations around the world to solve real problems using the Data-centric AI approach.

Start using CVAT online: cvat.ai. You can use it for free, or subscribe to get unlimited data, organizations, autoannotations, and Roboflow and HuggingFace integration.

Or set CVAT up as a self-hosted solution: Self-hosted Installation Guide. We provide Enterprise support for self-hosted installations with premium features: SSO, LDAP, Roboflow and HuggingFace integrations, and advanced analytics (coming soon). We also do trainings and a dedicated support with 24 hour SLA.

Quick start ⚡

Partners ❤️

CVAT is used by teams all over the world. In the list, you can find key companies which help us support the product or an essential part of our ecosystem. If you use us, please drop us a line at [email protected].

Human Protocol uses CVAT as a way of adding annotation service to the Human Protocol.
FiftyOne is an open-source dataset curation and model analysis tool for visualizing, exploring, and improving computer vision datasets and models that are tightly integrated with CVAT for annotation and label refinement.

Public datasets

ATLANTIS, an open-source dataset for semantic segmentation of waterbody images, developed by iWERS group in the Department of Civil and Environmental Engineering at the University of South Carolina is using CVAT.

For developing a semantic segmentation dataset using CVAT, see:

CVAT online: cvat.ai

This is an online version of CVAT. It's free, efficient, and easy to use.

cvat.ai runs the latest version of the tool. You can create up to 10 tasks there and upload up to 500Mb of data to annotate. It will only be visible to you or the people you assign to it.

For now, it does not have analytics features like management and monitoring the data annotation team. It also does not allow exporting images, just the annotations.

We plan to enhance cvat.ai with new powerful features. Stay tuned!

Prebuilt Docker images 🐳

Prebuilt docker images are the easiest way to start using CVAT locally. They are available on Docker Hub:

The images have been downloaded more than 1M times so far.

Screencasts 🎦

Here are some screencasts showing how to use CVAT.

Computer Vision Annotation Course: we introduce our course series designed to help you annotate data faster and better using CVAT. This course is about CVAT deployment and integrations, it includes presentations and covers the following topics:

Speeding up your data annotation process: introduction to CVAT and Datumaro. What problems do CVAT and Datumaro solve, and how they can speed up your model training process. Some resources you can use to learn more about how to use them.
Deployment and use CVAT. Use the app online at app.cvat.ai. A local deployment. A containerized local deployment with Docker Compose (for regular use), and a local cluster deployment with Kubernetes (for enterprise users). A 2-minute tour of the interface, a breakdown of CVAT’s internals, and a demonstration of how to deploy CVAT using Docker Compose.

Product tour: in this course, we show how to use CVAT, and help to get familiar with CVAT functionality and interfaces. This course does not cover integrations and is dedicated solely to CVAT. It covers the following topics:

Pipeline. In this video, we show how to use app.cvat.ai: how to sign up, upload your data, annotate it, and download it.

For feedback, please see Contact us

API

Documentation

SDK

Install with pip install cvat-sdk
PyPI package homepage
Documentation

CLI

Install with pip install cvat-cli
PyPI package homepage
Documentation

Supported annotation formats

CVAT supports multiple annotation formats. You can select the format after clicking the Upload annotation and Dump annotation buttons. Datumaro dataset framework allows additional dataset transformations with its command line tool and Python library.

For more information about the supported formats, see: Annotation Formats.

Annotation format	Import	Export
CVAT for images	✔️	✔️
CVAT for a video	✔️	✔️
Datumaro	✔️	✔️
PASCAL VOC	✔️	✔️
Segmentation masks from PASCAL VOC	✔️	✔️
YOLO	✔️	✔️
MS COCO Object Detection	✔️	✔️
MS COCO Keypoints Detection	✔️	✔️
MOT	✔️	✔️
MOTS PNG	✔️	✔️
LabelMe 3.0	✔️	✔️
ImageNet	✔️	✔️
CamVid	✔️	✔️
WIDER Face	✔️	✔️
VGGFace2	✔️	✔️
Market-1501	✔️	✔️
ICDAR13/15	✔️	✔️
Open Images V6	✔️	✔️
Cityscapes	✔️	✔️
KITTI	✔️	✔️
Kitti Raw Format	✔️	✔️
LFW	✔️	✔️
Supervisely Point Cloud Format	✔️	✔️
Ultralytics YOLO Detection	✔️	✔️
Ultralytics YOLO Oriented Bounding Boxes	✔️	✔️
Ultralytics YOLO Segmentation	✔️	✔️
Ultralytics YOLO Pose	✔️	✔️
Ultralytics YOLO Classification	✔️	✔️

Deep learning serverless functions for automatic labeling

CVAT supports automatic labeling. It can speed up the annotation process up to 10x. Here is a list of the algorithms we support, and the platforms they can be run on:

Name	Type	Framework	CPU	GPU
Segment Anything	interactor	PyTorch	✔️	✔️
Deep Extreme Cut	interactor	OpenVINO	✔️
Faster RCNN	detector	OpenVINO	✔️
Mask RCNN	detector	OpenVINO	✔️
YOLO v3	detector	OpenVINO	✔️
YOLO v7	detector	ONNX	✔️	✔️
Object reidentification	reid	OpenVINO	✔️
Semantic segmentation for ADAS	detector	OpenVINO	✔️
Text detection v4	detector	OpenVINO	✔️
SiamMask	tracker	PyTorch	✔️	✔️
TransT	tracker	PyTorch	✔️	✔️
f-BRS	interactor	PyTorch	✔️
HRNet	interactor	PyTorch		✔️
Inside-Outside Guidance	interactor	PyTorch	✔️
Faster RCNN	detector	TensorFlow	✔️	✔️
RetinaNet	detector	PyTorch	✔️	✔️
Face Detection	detector	OpenVINO	✔️

License

The code is released under the MIT License.

The code contained within the /serverless directory is released under the MIT License. However, it may download and utilize various assets, such as source code, architectures, and weights, among others. These assets may be distributed under different licenses, including non-commercial licenses. It is your responsibility to ensure compliance with the terms of these licenses before using the assets.

This software uses LGPL-licensed libraries from the FFmpeg project. The exact steps on how FFmpeg was configured and compiled can be found in the Dockerfile.

FFmpeg is an open-source framework licensed under LGPL and GPL. See https://www.ffmpeg.org/legal.html. You are solely responsible for determining if your use of FFmpeg requires any additional licenses. CVAT.ai Corporation is not responsible for obtaining any such licenses, nor liable for any licensing fees due in connection with your use of FFmpeg.

Contact us

Gitter to ask CVAT usage-related questions. Typically questions get answered fast by the core team or community. There you can also browse other common questions.

Discord is the place to also ask questions or discuss any other stuff related to CVAT.

LinkedIn for the company and work-related questions.

YouTube to see screencast and tutorials about the CVAT.

GitHub issues for feature requests or bug reports. If it's a bug, please add the steps to reproduce it.

#cvat tag on StackOverflow is one more way to ask questions and get our support.

[email protected] to reach out to us if you need commercial support.

Name		Name	Last commit message	Last commit date
Latest commit History 5,103 Commits
.github		.github
.regal		.regal
.vscode		.vscode
changelog.d		changelog.d
components		components
cvat-canvas		cvat-canvas
cvat-canvas3d		cvat-canvas3d
cvat-cli		cvat-cli
cvat-core		cvat-core
cvat-data		cvat-data
cvat-sdk		cvat-sdk
cvat-ui		cvat-ui
cvat		cvat
dev		dev
helm-chart		helm-chart
serverless		serverless
site		site
supervisord		supervisord
tests		tests
utils		utils
.bandit		.bandit
.coveragerc		.coveragerc
.dockerignore		.dockerignore
.editorconfig		.editorconfig
.eslintignore		.eslintignore
.eslintrc.cjs		.eslintrc.cjs
.gitattributes		.gitattributes
.gitignore		.gitignore
.gitmodules		.gitmodules
.nycrc		.nycrc
.prettierignore		.prettierignore
.prettierrc		.prettierrc
.pylintrc		.pylintrc
.remarkignore		.remarkignore
.remarkrc.js		.remarkrc.js
.stylelintrc.json		.stylelintrc.json
CHANGELOG.md		CHANGELOG.md
CITATION.cff		CITATION.cff
Dockerfile		Dockerfile
Dockerfile.ci		Dockerfile.ci
Dockerfile.ui		Dockerfile.ui
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
backend_entrypoint.sh		backend_entrypoint.sh
docker-compose.ci.yml		docker-compose.ci.yml
docker-compose.dev.yml		docker-compose.dev.yml
docker-compose.external_db.yml		docker-compose.external_db.yml
docker-compose.https.yml		docker-compose.https.yml
docker-compose.yml		docker-compose.yml
lint-staged.config.js		lint-staged.config.js
manage.py		manage.py
package.json		package.json
pyproject.toml		pyproject.toml
rqscheduler.py		rqscheduler.py
wait_for_deps.sh		wait_for_deps.sh
yarn.lock		yarn.lock

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

Table of Contents

Changes from upstream at cvat-ai/cvat

Running this fork of CVAT

Commands used to create this fork (Bash)

Updating to a new CVAT version

Example of running SAM2 on CPU as serverless function on the same PC as CVAT:

For Linux PCs (Bash)

For Windows PCs (PowerShell) - Not Tested

Example of running SAM2 on GPU as serverless function on the same PC as CVAT:

For Windows PCs (WSL)

Example of running SAM2 on GPU as serverless function on separate PC (w. IP: 172.17.155.175):

For Linux PCs (Bash)

For Windows PCs (PowerShell) - Not Tested

Original README from here on

Computer Vision Annotation Tool (CVAT)

Quick start ⚡

Partners ❤️

Public datasets

CVAT online: cvat.ai

Prebuilt Docker images 🐳

Screencasts 🎦

API

SDK

CLI

Supported annotation formats

Deep learning serverless functions for automatic labeling

License

Contact us

Links

About

Uh oh!

Releases

Packages

Languages

License

dk-teknologisk-jahs/cvat

Folders and files

Latest commit

History

Repository files navigation

Table of Contents

Changes from upstream at cvat-ai/cvat

Running this fork of CVAT

Commands used to create this fork (Bash)

Updating to a new CVAT version

Example of running SAM2 on CPU as serverless function on the same PC as CVAT:

For Linux PCs (Bash)

For Windows PCs (PowerShell) - Not Tested

Example of running SAM2 on GPU as serverless function on the same PC as CVAT:

For Windows PCs (WSL)

Example of running SAM2 on GPU as serverless function on separate PC (w. IP: 172.17.155.175):

For Linux PCs (Bash)

For Windows PCs (PowerShell) - Not Tested

Original README from here on

Computer Vision Annotation Tool (CVAT)

Quick start ⚡

Partners ❤️

Public datasets

CVAT online: cvat.ai

Prebuilt Docker images 🐳

Screencasts 🎦

API

SDK

CLI

Supported annotation formats

Deep learning serverless functions for automatic labeling

License

Contact us

Links

About

Resources

License

Security policy

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages