Apache Parquet

Apache Beam is a unified programming model for Batch and Streaming data processing.

Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.

🤖 A bot that helps you keep your projects up-to-date

sbt plugin for compiling Avro schemas, similar to sbt-protobuf

Apache Hadoop

An sbt plugin for missinglink

Build time tool for detecting link problems in java projects

socco-ng is a fork from criteo/socco: A Scala compiler plugin to generate documentation from Scala source files.

FlytePropeller is a Kubernetes native operator, that executes Flyte Workflows and Tasks. It has its own kubectl-flyte CLI to interact and is extensible using the flyteplugins/pluginmachinery interface

Flyte Backend Plugins contributed by the Flyte community.

"The path to execution", Styx is a service that schedules batch data processing jobs in Docker containers on Kubernetes.

Repository for www.lyh.me

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

A Scala API for Apache Beam and Google Cloud Dataflow.