anycastd is aimed to bring new age approaches to classic applications by allowing to announce virtual addresses via BGP w/ service-level checks.
Let's imaging you gonna create your own DNS/HTTP/whatever else service so how to utilize L3 load balancing and remove announce when something goes wrong on particular node? anycastd is exactly for that! No bunch of dedicated services and stateful control on BGP routes and no semi-tested Python scripts anymore!
anycastd is written as a pretty compact daemon like application so it could run in two main ways: via systemd or via docker/podman/any other container engine so that's why there's a container image in packages is available in parallel with traditional Go binaries.
For monitoring purposes anycastd listens HTTP socket with Prometheus metrics endpoint so once routes are removed from the announce SRE's could go and check why that happened.
anycastd longs to follow 12factor principles however it's not always possible to encode complex data structures into environment variable. So configuration is divided into two ways:
anycastd allows to set the following options via environment variables:
- CONFIG_PATH(string, default:- /config.yaml) - path to the configuration file
- LOG_LEVEL(ENUM, default:- WARN) - logging verbosity level, could one of the:- TRACE
- DEBUG
- INFO
- WARNING
- ERROR
- FATAL
- PANIC
 
Configuration file contains service configuration, i.e. how to announce, where to announce, checks to perform before announce. JSON and YAML formats for the same data structure is supported. Example:
---
announcer:
  router_id: 10.3.3.3
  local_address: 10.0.0.1
  local_asn: 65999
  routes:
    - 10.0.0.128/32
  peers:
    - name: some_router_1
      remote_address: 10.0.0.252
      remote_asn: 65000
    - name: some_router_2
      remote_address: 10.0.0.253
      remote_asn: 65000
services:
  - name: http
    check_interval: 10s
    checks:
      - kind: dns_lookup
        spec:
          query: google.com
          resolver: 127.0.0.1:53
          tries: 3
          interval: 100ms
          timeout: 3s
      - kind: http_2xx
        spec:
          url: http://127.0.0.1:8080
          method: GET
          headers:
            Host: example.com
          payload: ping
          path: /
          tries: 3
          interval: 100ms
          timeout: 2s
      - kind: tls_certificate
        spec:
          local:
            path: /etc/ssl/pki/cert.pem
          common_name: Test certificate
          dns_names:
            - site.example.org
          ip_addresses:
            - 127.0.0.1
          issuer: Test Issuer
      - kind: assigned_address
        spec:
          interface: dummy0
          ipv4: 33.22.11.0
      - kind: icmp_ping
        spec:
          static:
            host: google.com
          tries: 3
          interval: 100ms
          timeout: 5s
      - kind: tftp_rrq
        spec:
          url: tftp://127.0.0.1:69/lpxelinux.0
          tries: 3
          interval: 100ms
          timeout: 5s
metrics:
  enabled: true
  address: 127.0.0.1:9090
Check (implemented via Checker interface) is core concept in anycastd, allows to write well-tested piece of code to enable or disable announce. In anycastd most of checks could be written without any shell invocations i.e. in pure Go which is a preferable way. That's why there's no exec command check ;)
For now the following checks are available:
- assigned_address - ensures the address is assigned on interface
- dns_lookup - performs DNS lookup
- http_2xx - performs HTTP check and expects 2xx code
- icmp_ping - performs ICMP ping to the specified host
- tftp_rrq - performs TFTP GET request to specified URL
- tls_certificate - performs TLS certificate validation & provide expiration date via metrics
A special part of anycastd is a Prometheus-compatible metrics allows monitoring systems to notify operators about any changes in their service.
For now anycastd provides the following metrics:
Service could provide their metrics in order to aggregate current statuses.
| Metric name | Labels | Description | 
|---|---|---|
| anycastd_service_up | service, check | Service liveness status based on checks | 
Check running engine provides the following metrics.
| Metric name | Labels | Description | 
|---|---|---|
| anycastd_check_duration_seconds | service, check | Duration of check execution in seconds | 
In addition some checkers could provide their own metrics the list of them is bellow:
| Metric name | Labels | Description | 
|---|---|---|
| certificate_expires_in_seconds | check, path | Time the certificate expires in (in seconds) | 
| Metric name | Labels | Description | 
|---|---|---|
| anycastd_check_avg_rtt_seconds | check, host | Avg RTT of ICMP checks | 
| anycastd_check_loss_percent | check, host | Percent of packet loss | 
| anycastd_check_max_rtt_seconds | check, host | Max RTT of ICMP checks | 
| anycastd_check_min_rtt_seconds | check, host | Min RTT of ICMP checks | 
| anycastd_check_packets_received_duplicates_total | check, host | Total amount of duplicate packets received | 
| anycastd_check_packets_received_total | check, host | Total amount of packets received | 
| anycastd_check_packets_sent_total | check, host | Total amount of packets sent | 
| anycastd_check_std_dev_rtt_seconds | check, host | Standard deviation RTT of ICMP checks | 
The core of anycastd for BGP communication is GoBGP which allows so gather some details about peers, sessions and announces.
| Metric name | Labels | Description | 
|---|---|---|
| anycastd_gobgp_peer_admin_state | router_id, peer | Peer state 0=up, 1=down, 2=pfx_ct | 
| anycastd_gobgp_peer_count | router_id | Total amount of peers configured for the GoBGP instance | 
| anycastd_gobgp_peer_flops_count | router_id, peer | Peer flops count | 
| anycastd_gobgp_peer_out_queue_count | router_id, peer | Peer outgoing messages queue | 
| anycastd_gobgp_peer_password_set_flag | router_id, peer | Whether the peer have peer password set flag set | 
| anycastd_gobgp_peer_remove_private_flag | router_id, peer | Whether the peer have remove private flag set | 
| anycastd_gobgp_peer_send_community_flag | router_id, peer | Whether the peer have send community flag set | 
| anycastd_gobgp_peer_session_state | router_id, peer | Peer session state 0=unknown, 1=idle, 2=connect, 3=active, 4=opensent, 5=openconfirm, 6=established | 
| anycastd_gobgp_peer_type | router_id, peer | Peer type 0=internal, 1=external | 
| anycastd_gobgp_received_messages_keepalive | router_id, peer | Number of Keepalive messages received from the peer | 
| anycastd_gobgp_received_messages_notification | router_id, peer | Number of Notification messages received from the peer | 
| anycastd_gobgp_received_messages_open | router_id, peer | Number of Open messages received from the peer | 
| anycastd_gobgp_received_messages_refresh | router_id, peer | Number of Refresh messages received from the peer | 
| anycastd_gobgp_received_messages_total | router_id, peer | Total number of messages received from the peer | 
| anycastd_gobgp_received_messages_update | router_id, peer | Number of Update messages received from the peer | 
| anycastd_gobgp_received_messages_withdraw_update | router_id, peer | Number of Withdraw Update messages received from the peer | 
| anycastd_gobgp_sent_messages_keepalive | router_id, peer | Number of Keepalive messages sent to the peer | 
| anycastd_gobgp_sent_messages_notification | router_id, peer | Number of Notification messages sent to the peer | 
| anycastd_gobgp_sent_messages_open | router_id, peer | Number of Open messages sent to the peer | 
| anycastd_gobgp_sent_messages_refresh | router_id, peer | Number of Refresh messages sent to the peer | 
| anycastd_gobgp_sent_messages_total | router_id, peer | Total number of messages sent to the peer | 
| anycastd_gobgp_sent_messages_update | router_id, peer | Number of Update messages sent to the peer | 
| anycastd_gobgp_sent_messages_withdraw_prefix | router_id, peer | Number of Withdraw Prefix messages sent to the peer | 
| anycastd_gobgp_sent_messages_withdraw_update | router_id, peer | Number of Withdraw Update messages sent to the peer |