Skip to content

JaneliaSciComp/zarrcade

Repository files navigation

Zarrcade

logoz@0 1x

Python CI

Zarrcade makes it easy to generate simple web applications for browsing, searching, and visualizing collections of OME-NGFF (i.e. OME-Zarr) images. It implements the following features:

  • Automatic discovery of OME-Zarr images on any storage backend supported by fsspec including file system, AWS S3, Azure Blob, Google Cloud Storage, Dropbox, etc.
  • MIP/thumbnail generation
  • Web-based MIP gallery with convenient viewing links to OME-Zarr-compatible viewers
  • Searchable/filterable metadata and annotations
  • Build-in file proxy for non-public storage backends
  • Integration with the Allen Institute's BioFile Finder
  • Integration with external file proxies (e.g. x2s3)
Zarrcade screenshot

Prerequisites

1. Install pixi

Install pixi if you don't already have it.

2. Clone this repo

git clone https://github.com/JaneliaSciComp/zarrcade.git
cd zarrcade

3. Initialize the pixi environment

pixi install

Now you can run the pixi run zarrcade command.

Examples

To try a simple example, use one of following commands to import the example data before starting the server.

Example 1: Discover all OME-Zarr images stored within a given location

This example runs Zarr discovery on an S3 bucket. The metadata file adds textual annotations for each image.

pixi run zarrcade load examples/flyefish.yaml
pixi run zarrcade start

Example 2: Import OME-Zarr images specified in a spreadsheet

In this example, absolute paths are provided to Open Organelle Zarr images in the metadata file, along with absolute thumbnail paths.

pixi run zarrcade load examples/openorganelle.yaml
pixi run zarrcade start

Loading your own data

1. Create OME-Zarr images

If your images are not already in OME-Zarr format, you will need to convert them, e.g. using bioformats2raw:

bioformats2raw -w 128 -h 128 -z 64 --compression zlib /path/to/input /path/to/zarr

If you have many images to convert, use the nf-omezarr Nextflow pipeline to efficiently run bioformats2raw on a collection of images. This pipeline also lets you scale the conversion processes to your available compute resources (cluster, cloud, etc).

2. Create YAML configuration for your data collection

There is documentation on creating collection settings files, or you can follow one of the examples.

3. Import images and metadata into Zarrcade

You can import images into Zarrcade using the provided command line script:

pixi run zarrcade load path/to/mycollection.yaml

This will automatically create a local Sqlite database containing a Zarrcade collection named "mycollection" and populate it with information about the images in the specified directory. By default, this will also create MIPs and thumbnails for each image in ./static/.zarrcade (unless your metadata file already contains thumbnail paths).

Read more about the import options in the Data Import section of the documentation.

4. Run the Zarrcade web application

Start the development server, pointing it to your OME-Zarr data:

pixi run dev-launch

This is equivalent to running Uvicorn like this:

pixi run uvicorn zarrcade.serve:app --port 8000 --reload

Your images and annotations will be browseable at http://0.0.0.0:8000. Read the documentation below for more details on how to configure the web UI and deploy the service in production.

Documentation

  • Overview - learn about the data model and overall architecture
  • Configuration - configure the Zarrcade service using settings.yaml or environment variables
  • Deployment - instructions for deploying the service with Docker and in production mode
  • Development Notes - technical notes for developers working on Zarrcade itself

Known Limitations

  • The OmeZarrAgent does not currently support the full OME-Zarr specification, and may fail with certain types of images. If you encounter an error with your data, please open an issue on the Github repository.

Attributions

About

Create web-based OME-Zarr galleries

Topics

Resources

License

Stars

Watchers

Forks

Packages