brozzler - distributed browser-based web crawler

WARC writing MITM HTTP/S proxy

rethinkdb python library

Python client package for the playback rules engine

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

A youtube-dl fork with additional features and fixes

The Python programming language

A rules engine for managing playback.

IA's public Wayback Machine (moved from SourceForge)

A library that provides an embeddable, persistent key-value store for fast storage.

Wombat.js client-side rewriting library

Web archive index server based on RocksDB

A queue-controlled browser automation tool for improving web crawl quality

Trough: Big data, small databases.

Command-line program to download videos from YouTube.com and other video sites

Headless chrome/chromium automation library (unofficial port of puppeteer)

CiviCRM (Core Application and Framework)

Python script to create CDX index files of WARC data

utility to fetch provenance information from Internet Archive's Wayback Machine

Ansible Role - Apache Solr

Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications — automat…

Nagios Plugin for RabbitMQ

Mobile first "lite" Open Library

One webpage for every book ever published!