Skip to content

K3S install fails when network interface has multiple addresses #1362

@Nevon

Description

@Nevon

Description

Running colima start --kubernetes fails with the following error:

TRAC[0016] cmd ["lima" "sh" "-c" "INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_SKIP_ENABLE=true k3s-install.sh --write-kubeconfig-mode 644 --disable=traefik --advertise-address 192.168.105.3\n192.168.105.2 --flannel-iface col0 --docker --https-listen-port 63823"] 
> [INFO]  Skipping k3s download and verify
> [INFO]  Skipping installation of SELinux RPM
> [INFO]  Skipping /usr/local/bin/kubectl symlink to k3s, already exists
> [INFO]  Skipping /usr/local/bin/crictl symlink to k3s, already exists
> [INFO]  Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
> [INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
> [INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
> [INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
> [INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
> sh: 2: 192.168.105.2: not found
FATA[0016] error provisioning kubernetes: exit status 127 

This happens because of adding a second ip address to col0, which limautil.GetIpAddress doesn't handle very well and returns a newline delimited string of ip addresses (see the value being passed in --advertise-address above).

func getIPAddress(profileID, interfaceName string) string {
var buf bytes.Buffer
// TODO: this should be less hacky
cmd := Limactl("shell", profileID, "sh", "-c",
`ip -4 addr show `+interfaceName+` | grep inet | awk -F' ' '{print $2 }' | cut -d/ -f1`)
cmd.Stderr = nil
cmd.Stdout = &buf
_ = cmd.Run()
return strings.TrimSpace(buf.String())
}

You can also see this newline in for example colima list:

PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK      RUNTIME       ADDRESS
default    Running    aarch64    3       8GiB      100GiB    docker+k3s    192.168.105.3
192.168.105.2

Version

colima version 0.8.1
git commit: 96598cc

runtime: docker
arch: aarch64
client: v28.3.2
server: v27.3.1
limactl version 1.1.1

Operating System

  • macOS Intel <= 13 (Ventura)
  • macOS Intel >= 14 (Sonoma)
  • Apple Silicon <= 13 (Ventura)
  • Apple Silicon >= 14 (Sonoma)
  • Linux

Output of colima status

INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] address: 192.168.105.3
192.168.105.2
INFO[0000] socket: unix:///$HOME/.colima/default/docker.sock

Reproduction Steps

  1. Create a config with a fixed ip address:
provision:
  - mode: system
    script: ip address add 192.168.105.2/24 brd 192.168.105.255 scope global dev col0
  1. Run colima start --kubernetes --profile my-profile

Expected behaviour

It's a little tricky, because I don't know how colima could pick which address it should use as the advertised ip for k3s, but even just picking the primary would be better than crashing.

Additional context

I also tried to override --advertise-address using --k3s-arg, but the argument is not replaced and instead just tacked on to the end of the command:

$ colima start -k --k3s-arg="--disable=traefik" --k3s-arg="--advertise-address=192.168.105.2" --very-verbose
...
TRAC[0025] cmd ["lima" "sh" "-c" "INSTALL_K3S_SKIP_DOWNLOAD=true INSTALL_K3S_SKIP_ENABLE=true k3s-install.sh --write-kubeconfig-mode 644 --disable=traefik --advertise-address=192.168.105.2 --advertise-address 192.168.105.5\n192.168.105.2 --flannel-iface col0 --docker --https-listen-port 57908"]

I've filed that as a separate bug in #1363

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions