Skip to content

use MSI identity to login for az cli when useManagedIdentityExtension is set to true and aadClientSecret is not provided #2413

New issue
New issue
Open
Open
use MSI identity to login for az cli when useManagedIdentityExtension is set to true and aadClientSecret is not provided#2413
Labels
kind/enhancementteam/area2
Milestone
v1.5.x
@chaudhryfaisal

Description

@chaudhryfaisal
chaudhryfaisal
opened on Jan 13, 2021

RKE version:
1.2.3

Docker version: (docker version,docker info preferred)

Client: Docker Engine - Community
 Version:           19.03.14
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        5eb3275d40
 Built:             Tue Dec  1 19:20:42 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.14
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       5eb3275d40
  Built:            Tue Dec  1 19:19:17 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Operating system and kernel: (cat /etc/os-release, uname -r preferred)

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)
Azure
cluster.yml file:

cloud_provider:
  azureCloudProvider:
    subscriptionId: subscriptionId
    tenantId: tenantId
    vmType: vmss
    useManagedIdentityExtension: true
  name: azure
cluster_name: cluster_name
ignore_docker_version: true
ingress:
  provider: nginx
kubernetes_version: v1.18.12-rancher1-1
network:
  plugin: calico
nodes:
- address: IP_ADDRESS
  hostname_override: vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000
  role:
  - controlplane
  - etcd
  - worker
  user: azureuser

Steps to Reproduce:

rke up

Results:

rke-tools seems to always try to login using aadClientId and aadClientSecret however when aadClientSecret is not provided and useManagedIdentityExtension is set to true then it should use MSI identity to login for az cli

rke up logs

time="2021-01-13T15:18:43Z" level=info msg="[sync] Syncing nodes Labels and Taints"
time="2021-01-13T15:18:43Z" level=debug msg="worker [9] starting sync for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000]"
time="2021-01-13T15:18:43Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #1"
time="2021-01-13T15:18:48Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #2"
time="2021-01-13T15:18:53Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #3"
time="2021-01-13T15:18:58Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #4"
time="2021-01-13T15:19:03Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #5"
time="2021-01-13T15:19:08Z" level=debug msg="[hosts] Can't find node by name [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], error:  \"vmss-k8s-cluster-demo-01-single-node-dev-
eastus-fic1000000\" not found"
time="2021-01-13T15:19:10Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #1"
time="2021-01-13T15:19:15Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #2"
time="2021-01-13T15:19:20Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #3"
time="2021-01-13T15:19:25Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #4"
time="2021-01-13T15:19:30Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #5"
time="2021-01-13T15:19:35Z" level=debug msg="[hosts] Can't find node by name [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], error:  \"vmss-k8s-cluster-demo-01-single-node-dev-
eastus-fic1000000\" not found"
time="2021-01-13T15:19:37Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #1"
time="2021-01-13T15:19:42Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #2"
time="2021-01-13T15:19:47Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #3"
time="2021-01-13T15:19:52Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #4"
time="2021-01-13T15:19:57Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #5"
time="2021-01-13T15:20:02Z" level=debug msg="[hosts] Can't find node by name [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], error:  \"vmss-k8s-cluster-demo-01-single-node-dev-
eastus-fic1000000\" not found"
time="2021-01-13T15:20:04Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #1"
time="2021-01-13T15:20:09Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #2"
time="2021-01-13T15:20:14Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #3"
time="2021-01-13T15:20:19Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #4"
time="2021-01-13T15:20:24Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #5"
time="2021-01-13T15:20:29Z" level=debug msg="[hosts] Can't find node by name [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], error:  \"vmss-k8s-cluster-demo-01-single-node-dev-
eastus-fic1000000\" not found"
time="2021-01-13T15:20:31Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #1"
time="2021-01-13T15:20:36Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #2"
time="2021-01-13T15:20:41Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #3"
time="2021-01-13T15:20:46Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #4"
time="2021-01-13T15:20:51Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #5"
time="2021-01-13T15:20:56Z" level=debug msg="[hosts] Can't find node by name [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], error:  \"vmss-k8s-cluster-demo-01-single-node-dev-
eastus-fic1000000\" not found"
time="2021-01-13T15:20:58Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #1"
time="2021-01-13T15:21:03Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #2"
time="2021-01-13T15:21:08Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #3"
time="2021-01-13T15:21:13Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #4"
time="2021-01-13T15:21:18Z" level=debug msg="Checking node list for node [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], try #5"
time="2021-01-13T15:21:23Z" level=debug msg="[hosts] Can't find node by name [vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000], error:  \"vmss-k8s-cluster-demo-01-single-node-dev-
eastus-fic1000000\" not found"
time="2021-01-13T15:21:25Z" level=fatal msg="[ \"vmss-k8s-cluster-demo-01-single-node-dev-eastus-fic1000000\" not found]"

suspected root cause from kubelet, kube-controller-manager and kube-apiserver container logs

 kubelet.log:Some variables were not populated correctly, using the passed config!
 kubelet.log:Some variables were not populated correctly, using the passed config!
 kubelet.log:Some variables were not populated correctly, using the passed config!
 kubelet.log:Some variables were not populated correctly, using the passed config!
 kubelet.log:Some variables were not populated correctly, using the passed config!
 kubelet.log:Some variables were not populated correctly, using the passed config!
 kubelet.log:Some variables were not populated correctly, using the passed config!
 kubelet.log:Some variables were not populated correctly, using the passed config!
 kubelet.log:Some variables were not populated correctly, using the passed config!
 kubelet.log:Some variables were not populated correctly, using the passed config!
 kube-controller-manager.log:Some variables were not populated correctly, using the passed config!
 kube-apiserver.log:Some variables were not populated correctly, using the passed config!

proposed fix
rancher/rke-tools#116

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/enhancementteam/area2

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions