Skip to content

gekatateam/neptunus

Repository files navigation

Neptunus

Neptunus is a data processing engine for consuming, transforming, and producing events. Originally conceived as a central unit of a mediation platform, Neptunus can:

  • receive data from a number of different sources, either from message brokers or by acting as a server,
  • manage event streams based on filtering rules,
  • transform, enrich, and create new events,
  • and deliver events to consumers in various formats and protocols.

It also can collect and write metrics, directly related to your processes.

Neptunus is based on data processing pipelines - a composition of six types of plugins:

  • Inputs consume events from external sources
  • Processors transform events
  • Outputs produce events to external systems
  • Filters route events in pipeline by conditions
  • Parsers convert raw data into events
  • Serializers convert events into data formats for external systems

Configuration

Neptunus configuration has two parts - daemon config and pipelines.

See more in our documentation.

Getting Started

Get help:

neptunus --help

Run daemon:

neptunus run --config config.toml

Test pipelines configuration:

neptunus test --config config.toml

Get help about cli tool usage:

neptunus pipeline --help

How to build

This project uses Taskfile as a build tool. Out-of-the-box, there are three OS and two platform combinations: linux, windows, darwin and amd64, arm64. You can add more in builds and packs tasks if needed. All tasks should be cross-platform, but note that they are tested only on Windows 10 and Linux (Ubuntu 22.04).

Then, follow simple steps:

  1. Install Taskfile and go-licence-detector
  2. Run task build:{{ OS }}-{{ PLATFORM }} to build binary
  3. Run task build:notice to generate NOTICE.txt file
  4. Run task pack:{{ OS }}-{{ PLATFORM }} to pack your build
  5. Run task build:docker or task build:podman if you need container image
  6. Finally, run task cleanup to remove build artifacts from file system

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors 3

  •  
  •  
  •  

Languages