pg_lake: Postgres with Iceberg and data lake access

The Future of Data Engineering — A CLI SQL client for the modern data stack, enabling AI-native context engineering for data.

AutoMQ is a diskless Kafka on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.

Hybrid in-memory and disk cache in Rust

Distributed query engine providing simple and reliable data processing for any modality and scale

Memory for AI Agents; SOTA in AI Agent Memory; Announcing OpenMemory MCP - local and secure memory management.

Apache DataFusion Ray

DataChain 🔗 Process and curate unstructured data using local ML models and LLM calls

Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks

A modular graph-based Retrieval-Augmented Generation (RAG) system

Open, Multi-modal Catalog for Data & AI

Polycat is a cutting-edge cloud-native metastore system, purpose-built to cater to the demands of modern data management in lakehouse deployments. It offers a comprehensive solution for organizatio…

小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫

The simplest, fastest repository for training/finetuning medium-sized GPTs.

Detect file content types with deep learning

An Open Standard for lineage metadata collection

LLMPerf is a library for validating and benchmarking LLMs

A unified streaming and historical data processing engine in one single binary, powered by ClickHouse

An embedded key-value store library specialized for building state machine and log store

Enhance Tesseract OCR output for scanned PDFs by applying Large Language Model (LLM) corrections, complete with options for text validation and hallucination filtering.

ClickBench: a Benchmark For Analytical Databases

Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, branching, and bottomless storage.

Auto parallelization for large-scale neural networks

Tigris is a modern, scalable backend for building real-time websites and apps.

These are the best resources for System Design on the Internet

A modular implementation of timely dataflow in Rust

🎨 Diagram as Code for prototyping cloud system architectures

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.

Modin: Speed up your Pandas workflows by changing a single line of code

Automatically visualize your pandas dataframe via a single print! 📊 💡