---
title: "Health Configuration"
date: 2019-03-06T13:55:38+02:00
draft: false
type: "documentation"
weight: 5
---

:linkattrs:
:sectlinks:

= Health Configuration
:sectnums:
:toc: left
toc::[]
:toc-title: Custom Health
:keywords: Kiali Documentation Health
:icons: font
:imagesdir: /images/documentation/health-configuration/


== Health

Kiali calculates health by combining the individual health of several indicators, such as pods and request traffic.  The _global health_ of a resource reflects the most severe health of its indicators.

=== Health Indicators

The table below lists the current health indicators and whether the indicator supports custom configuration for its health calculation.

++++
<div style="display: flex;">
    <table style="width: 100%; border: 1px solid black">
      <tr style="width: 100%; border: 1px solid black">
        <th>Indicator</th>
        <th>Supports Configuration</th>
      </tr>
      <tr style="width: 100%; border: 1px solid black">
        <td>Pod Status</td><td>No</td>
      </tr>
      <tr style="width: 100%; border: 1px solid black">
        <td>Traffic Health</td><td>Yes</td>
      </tr>
    </table>
</div>
++++

=== Icons and colors

Kiali use icons and colors to indicate the health of resources and associated request traffic.

++++
<div style="display: flex;">
<ul>
<li>
 <img src="/images/documentation/health-configuration/no_health.png" style="width: 40px;height: 40px" /> No Health Information (NA)
</li>
<li>
 <img src="/images/documentation/health-configuration/healthy.png" style="width: 40px;height: 40px" /> Healthy
</li>
<li>
 <img src="/images/documentation/health-configuration/degraded.png" style="width: 40px;height: 40px" /> Degraded
</li>
<li>
 <img src="/images/documentation/health-configuration/failure.png" style="width: 40px;height: 40px" /> Failure
</li>
</ul>
  </div>
++++

=== Default Values

==== Request Traffic

By default Kiali uses the traffic rate configuration shown below.  Application errors have minimal tolerance while client errors have a higher tolerance reflecting that some level of client errors is often normal (e.g. 404 Not Found):

* For _http_ protocol 4xx are client errors and 5xx codes are application errors.
* For _grpc_ protocol all 1-16 are errors (0 is success).

So, for example, if the rate of application errors is >= 0.1% kiali will show `Degraded` health and if > 10% will show `Failure` health.

```yaml
# ...
  health_config:
    rate:
      - namespace: ".*"
        kind: ".*"
        name: ".*"
        tolerance:
          - code: "^5\\d\\d$"
            direction: ".*"
            protocol: "http"
            degraded: 0
            failure: 10
          - code: "^4\\d\\d$"
            direction: ".*"
            protocol: "http"
            degraded: 10
            failure: 20
          - code: "^[1-9]$|^1[0-6]$"
            direction: ".*"
            protocol: "grpc"
            degraded: 0
            failure: 10
# ...
```

=== Configuration

Custom health configuration is specified in the Kiali CR. To see the supported configuration syntax for `health_config` visit link:https://github.com/kiali/kiali-operator/blob/master/deploy/kiali/kiali_cr.yaml[Kiali CR].

Kiali applies *the first matching rate configuration (namespace, kind, etc)* and calculates the status for each tolerance. The reported health will be the status with highest priority (see below).


++++
<table style="width: 100%; border: 1px solid black">
<tr style="width: 100%; border: 1px solid black"><th>Rate Option</th><th>Definition</th><th>Default</th>
<tr style="width: 100%; border: 1px solid black">
<td>namespace</td><td>Matching Namespaces (regex)</td><td>".*" (match all)</td>
</tr>
<tr style="width: 100%; border: 1px solid black">
<td>kind</td><td>Matching Resource Types (workload|app|service) (regex)</td><td>".*" (match all)</td>
</tr>
<tr style="width: 100%; border: 1px solid black">
<td>name</td><td>Matching Resource Names (regex)</td><td>".*" (match all)</td>
</tr>
<tr>
<td rowspan="2">Tolerance</td><td colspan="2">Array of tolerances to apply.</td>
</tr>
<tr>
<td colspan="2">
 <table style="width: 100%; border: 1px solid black" >
    <tr style="width: 100%; border: 1px solid black">
        <th>Tolerance Option</th>
        <th>Definition</th>
        <th>Default</th>
    </tr>
    <tr>
      <td>code</td>
      <td>Matching Response Status Codes (regex) [1]</td>
      <td><strong>required</strong></td>
    </tr>
    <tr>
      <td>direction</td>
      <td>Matching Request Directions (inbound|outbound) (regex)</td>
      <td>".*" (match all)</td>
    </tr>
    <tr>
      <td>protocol</td>
      <td>Matching Request Protocols (http|grpc) (regex)</td>
      <td>".*" (match all)</td>
    </tr>
    <tr>
      <td>degraded</td>
      <td>Degraded Threshold(% matching requests >= value)</td>
      <td>0</td>
    </tr>
    <tr>
      <td>failure</td>
      <td>Failure Threshold (% matching requests >= value)</td>
      <td>0</td>
    </tr>
 </table>
</td>
</tr>
</table>
++++
_[1] The status code typically depends on the request protocol. The special code **-**, a single dash, is used for requests that don't receive a response, and therefore no response code._

Kiali reports traffic health with the following top-down status priority :

++++
 <table style="width: 100%; border: 1px solid black" >
    <tr>
        <th>Priority</th>
        <th>Rule (value=% matching requests)</th>
        <th>Status</th>
    </tr>
    <tr>
      <td>1</td>
      <td>value >= FAILURE threshold</td>
      <td><img src="/images/documentation/health-configuration/failure.png" style="width: 40px;height: 40px" />FAILURE</td>
    </tr>
    <tr>
      <td>2</td>
      <td>value >= DEGRADED threshold AND value < FAILURE threshold</td>
      <td><img src="/images/documentation/health-configuration/degraded.png" style="width: 40px;height: 40px" />DEGRADED</td>
    </tr>
    <tr>
      <td>3</td>
      <td>value > 0 AND value < DEGRADED threshold</td>
      <td><img src="/images/documentation/health-configuration/healthy.png" style="width: 40px;height: 40px" />HEALTHY</td>
    </tr>
    <tr>
      <td>4</td>
      <td>value = 0</td>
      <td><img src="/images/documentation/health-configuration/healthy.png" style="width: 40px;height: 40px" />HEALTHY</td>
    </tr>
    <tr>
      <td>5</td>
      <td>No traffic</td>
      <td><img src="/images/documentation/health-configuration/no_health.png" style="width: 40px;height: 40px" />No Health Information</td>
    </tr>

 </table>
++++

== Examples

These examples use the repo _https://github.com/kiali/demos/tree/master/error-rates_.

In this repo we can see 2 namespaces: alpha and beta (link:https://github.com/kiali/demos/tree/master/error-rates#error-rates-demo-design[Demo design]).
++++
<table style="width: 100%">
<tr style="text-align: center">
<td>Alpha</td>
</tr>
<tr style="text-align: center">
<td>
<img src="https://raw.githubusercontent.com/kiali/demos/master/error-rates/doc/Kiali-AlphaNamespace.png" style="width: 80%; height: 60%" />
</td>
</tr>
</table>
++++

Where nodes return the responses (You can configure responses link:https://github.com/kiali/demos/tree/master/error-rates#configurable-error-rates[here] ):

- link:https://github.com/kiali/demos/blob/master/error-rates/alpha.yaml[Alpha deployment]
- link:https://github.com/kiali/demos/blob/master/error-rates/beta.yaml[Beta deployment]

++++
<table style="width: 100%; border: 1px solid black">
<tr style="text-align: center; border: 1px solid black; background-color: #F0F0F0">
<td style="border: 1px solid black" rowspan="2">App</td><td style="border: 1px solid black" colspan="2">Alpha/Beta</td>
</tr>
<tr style="text-align: center; background-color: #F0F0F0">
<td style="text-align: center; border: 1px solid black"> Code </td>
<td style="text-align: center; border: 1px solid black"> Rate </td>
</tr>
<tr>
    <td style="text-align: center; border: 1px solid black" rowspan="2"> x-server</td>
    <td style="text-align: center; border: 1px solid black"> 200</td>
    <td style="text-align: center; border: 1px solid black"> 9</td>
</tr>
<tr>
    <td style="text-align: center; border: 1px solid black"> 404</td>
    <td style="text-align: center; border: 1px solid black"> 1  </td>
</tr>
<tr>
    <td style="text-align: center; border: 1px solid black" rowspan="2"> y-server</td>
    <td style="text-align: center; border: 1px solid black"> 200</td>
    <td style="text-align: center; border: 1px solid black"> 9</td>
</tr>
<tr>
    <td style="text-align: center; border: 1px solid black"> 500</td>
    <td style="text-align: center; border: 1px solid black"> 1</td>
</tr>
<tr>
    <td style="text-align: center; border: 1px solid black" rowspan="3"> z-server</td>
    <td style="text-align: center; border: 1px solid black"> 200</td>
    <td style="text-align: center; border: 1px solid black"> 8</td>
</tr>
<tr>
    <td style="text-align: center; border: 1px solid black"> 201</td>
    <td style="text-align: center; border: 1px solid black"> 1</td>
</tr>
<tr>
    <td style="text-align: center; border: 1px solid black"> 202</td>
    <td style="text-align: center; border: 1px solid black"> 1</td>
</tr>
</table>
++++

The applied traffic rate configuration is:

```yaml
# ...
health_config:
  rate:
   - namespace: "alpha"
     tolerance:
       - code: "404"
         failure: 10
         protocol: "http"
       - code: "[45]\\d[^\\D4]"
         protocol: "http"
   - namespace: "beta"
     tolerance:
       - code: "[4]\\d\\d"
         degraded: 30
         failure: 40
         protocol: "http"
       - code: "[5]\\d\\d"
         protocol: "http"
# ...
```

After Kiali adds default configuration we have the following (Debug Info Kiali):

```json
{
  "healthConfig": {
    "rate": [
      {
        "namespace": "/alpha/",
        "kind": "/.*/",
        "name": "/.*/",
        "tolerance": [
          {
            "code": "/404/",
            "degraded": 0,
            "failure": 10,
            "protocol": "/http/",
            "direction": "/.*/"
          },
          {
            "code": "/[45]\\d[^\\D4]/",
            "degraded": 0,
            "failure": 0,
            "protocol": "/http/",
            "direction": "/.*/"
          }
        ]
      },
      {
        "namespace": "/beta/",
        "kind": "/.*/",
        "name": "/.*/",
        "tolerance": [
          {
            "code": "/[4]\\d\\d/",
            "degraded": 30,
            "failure": 40,
            "protocol": "/http/",
            "direction": "/.*/"
          },
          {
            "code": "/[5]\\d\\d/",
            "degraded": 0,
            "failure": 0,
            "protocol": "/http/",
            "direction": "/.*/"
          }
        ]
      },
      {
        "namespace": "/.*/",
        "kind": "/.*/",
        "name": "/.*/",
        "tolerance": [
          {
            "code": "/^5\\d\\d$/",
            "degraded": 0,
            "failure": 10,
            "protocol": "/http/",
            "direction": "/.*/"
          },
          {
            "code": "/^4\\d\\d$/",
            "degraded": 10,
            "failure": 20,
            "protocol": "/http/",
            "direction": "/.*/"
          },
          {
            "code": "/^[1-9]$|^1[0-6]$/",
            "degraded": 0,
            "failure": 10,
            "protocol": "/grpc/",
            "direction": "/.*/"
          }
        ]
      }
    ]
  }
}
```

What are we applying?

- For namespace alpha, all resources
  - Protocol http if % requests with error code 404 are >= 10 then FAILURE, if they are > 0 then DEGRADED
  - Protocol http if % requests with others error codes are> 0 then FAILURE.

- For namespace beta, all resources
  - Protocol http if % requests with error code 4xx are >= 40 then FAILURE, if they are >= 30 then DEGRADED
  - Protocol http if % requests with error code 5xx are > 0 then FAILURE

- For other namespaces kiali apply defaults.
  - Protocol http if % requests with error code 5xx are >= 20 then FAILURE, if they are >= 0.1 then DEGRADED
  - Protocol grpc if % requests with error code match /^[1-9]$|^1[0-6]$/ are >= 20 then FAILURE, if they are >= 0.1 then DEGRADED



++++
 <table style="width: 100%; border: 1px solid black" >
    <tr style="text-align: center;width: 100%; border: 1px solid black">
        <td> Alpha </td>
        <td> Beta </td>
    </tr>
    <tr>
      <td><img src="/images/documentation/health-configuration/alpha.png" style="width: 100%;height: 800px" /></td>
      <td><img src="/images/documentation/health-configuration/beta.png" style="width: 100%;height: 800px" /></td>
    </tr>
 </table>
++++
