This repo implements a celestia da-server for Alt-DA mode using generic commitments.

The da-server connects to a celestia-node running as a sidecar process.

celestia da-server accepts the following flags for celestia storage using celestia go client

   --celestia.addr value                      celestia rpc endpoint (default: "http://localhost:26658") [$OP_ALTDA_CELESTIA_ADDR]
   --celestia.auth-token value                celestia rpc auth token [$OP_ALTDA_CELESTIA_AUTH_TOKEN]
   --celestia.namespace value                 celestia namespace [$OP_ALTDA_CELESTIA_NAMESPACE]
   --celestia.core.grpc.addr value            celestia core grpc addr (default: "http://localhost:9090") [$OP_ALTDA_CELESTIA_CORE_GRPC_ADDR]
   --celestia.p2p-network value               celestia p2p network (default: "mocha-4") [$OP_ALTDA_CELESTIA_P2P_NETWORK]
   --addr value                               alt da server listening address (default: "127.0.0.1") [$OP_ALTDA_ADDR]
   --port value                               alt da server listening port (default: 3100) [$OP_ALTDA_PORT]
   --celestia.tls-enabled                     celestia rpc TLS (default: true) [$OP_ALTDA_CELESTIA_TLS_ENABLED]
   --celestia.core.grpc.tls-enabled           enable core grpc TLS (default: true) [$OP_ALTDA_CELESTIA_CORE_GRPC_TLS_ENABLED]
   --celestia.core.grpc.auth-token value      celestia core grpc auth token [$OP_ALTDA_CELESTIA_CORE_GRPC_AUTH_TOKEN]
   --celestia.keyring.default-key-name value  celestia default key name (default: "my_celes_key") [$OP_ALTDA_CELESTIA_DEFAULT_KEY_NAME]
   --celestia.keyring.path value              celestia keyring path e.g. ~/.celestia-light-mocha-4/keys [$OP_ALTDA_CELESTIA_KEYRING_PATH]
   --s3.credential-type value                 The way to authenticate to S3, options are [iam, static] [$OP_ALTDA_S3_CREDENTIAL_TYPE]
   --s3.bucket value                          bucket name for S3 storage [$OP_ALTDA_S3_BUCKET]
   --s3.path value                            path for S3 storage [$OP_ALTDA_S3_PATH]
   --s3.endpoint value                        endpoint for S3 storage [$OP_ALTDA_S3_ENDPOINT]
   --s3.access-key-id value                   access key id for S3 storage [$OP_ALTDA_S3_ACCESS_KEY_ID]
   --s3.access-key-secret value               access key secret for S3 storage [$OP_ALTDA_S3_ACCESS_KEY_SECRET]
   --s3.timeout value                         S3 timeout (default: 5s) [$OP_ALTDA_S3_TIMEOUT]
   --routing.fallback                         Enable fallback (default: false) [$OP_ALTDA_FALLBACK]
   --routing.cache                            Enable cache. (default: false) [$OP_ALTDA_CACHE]
   --log.level value                          The lowest log level that will be output (default: INFO) [$OP_ALTDA_LOG_LEVEL]
   --log.format value                         Format the log output. Supported formats: 'text', 'terminal', 'logfmt', 'json', 'json-pretty', (default: text) [$OP_ALTDA_LOG_FORMAT]
   --log.color                                Color the log output if in terminal mode (default: false) [$OP_ALTDA_LOG_COLOR]
   --log.pid                                  Show pid in the log (default: false) [$OP_ALTDA_LOG_PID]
   --help, -h                                 show help
   --version, -v                              print the version

The celestia server endpoint should be set to the celestia-node rpc server, usually http://127.0.0.1:26658.

A random valid namespace can be generated by running the following command:

export NAMESPACE=00000000000000000000000000000000000000$(openssl rand -hex 10)

The auth token is auto generated by celestia-node and is required to submit blob transactions to the celestia-node.

Follow the usual instructions for setting up devnet.

There are two ways to test the celestia da-server in devnet and testnet modes.

The first way uses a fork of rollop to automate deployment. This method is recommended for beginners.

Clone the rollop repository, checkout the alt-da branch and follow the instructions in the README to setup the rollop stack.

git clone https://github.com/celestiaorg/roll-op.git
cd roll-op
git checkout alt-da
./rollop setup --yes

rollop is configured to use mocha-4 celestia network by default in both devnet and testnet modes.

Once the rollop stack is setup, you can start the devnet with the following command:

./rollop --clean devnet

This will start the devnet in Alt-DA mode with celestia-node running in testnet mode.

You can verify that the devnet is running by checking da-server logs:

tail -f deployments/rollup/logs/da_server.log

Upon successful operation, the logs should contain the following message:

t=2024-05-30T19:08:30+0000 lvl=info msg="Using celestia storage" url=http://127.0.0.1:26658
t=2024-05-30T19:08:32+0000 lvl=info msg="celestia: blob successfully submitted" id=0900000000000000b25a32154ab00902cfc0269b3239b612ebfe7f7263545119ee7251cc72728142
t=2024-05-30T19:08:34+0000 lvl=info msg="celestia: blob successfully submitted" id=0a00000000000000cb559bc3c6a01b0819460ce86c13165fdc58ac9c81c1e52404f8c4b36097db87
t=2024-05-30T19:08:34+0000 lvl=info msg="celestia: blob request" id=010c0900000000000000b25a32154ab00902cfc0269b3239b612ebfe7f7263545119ee7251cc72728142

Copy example config config.toml.example to config.toml and update any fields as required.

Next start the testnet with the following command:

./rollop --name=celopstia-11177111 --preset=prod --config=config.toml l2

You can verify that the testnet is running by checking da-server logs:

tail -f deployments/celopstia-11177111/logs/da_server.log

Upon successful operation, the logs should contain the following message:

t=2024-06-03T21:36:32+0000 lvl=info msg="Using celestia storage" url=http://127.0.0.1:26658
t=2024-06-03T21:36:32+0000 lvl=info msg="Started DA Server"
t=2024-06-03T21:37:16+0000 lvl=debug msg=PUT url=/put/
t=2024-06-03T21:37:19+0000 lvl=info msg="celestia: blob successfully submitted" id=430f1e0000000000f34842f2c61d3b1fc1fad749ed442320fdce7023669f3116ea8df024ce792b7f
t=2024-06-03T21:38:23+0000 lvl=debug msg=PUT url=/put/
t=2024-06-03T21:38:28+0000 lvl=debug msg=GET url=/get/0x010c430f1e0000000000f34842f2c61d3b1fc1fad749ed442320fdce7023669f3116ea8df024ce792b7f
t=2024-06-03T21:38:28+0000 lvl=info msg="celestia: blob request" id=010c430f1e0000000000f34842f2c61d3b1fc1fad749ed442320fdce7023669f3116ea8df024ce792b7f

To configure the da-server to use a different namespace, update the da_namespace field in config.toml.

To configure the da-server to use a different celestia network, you'll need to update the --p2p.network flag in l2_batcher.py, l2_node.py and celestia_light_node.py for now. This will be configurable in the future.

Check the logs for the da-server to see if it is running successfully:

tail -f deployments/rollup/logs/da_server.log

Additionally verify that the batcher is able to submit transactions to the celestia-node:

tail -f deployments/rollup/logs/l2_batcher.log

The second way to test the celestia da-server uses docker-compose to deploy the rollup stack. This method reuses the bedrock devnet docker-compose setup and is recommended for developers.

For testing devnet, we'll use the local-celestia-devnet docker image which simulates the whole celestia network including a local celestia-node and does not require a peer-to-peer connection.

To run a local devnet with local-celestia-devnet:

DEVNET_ALT_DA="true" GENERIC_ALT_DA="true"  make devnet-up

This will start the devnet in Alt-DA mode with local-celestia-devnet.

You can verify that the devnet is running by checking da-server logs:

docker logs -f ops-da-server-1 | grep celestia

Upon successful operation, the logs should contain the following message:

t=2024-05-30T19:08:30+0000 lvl=info msg="Using celestia storage" url=http://da:26658
t=2024-05-30T19:08:32+0000 lvl=info msg="celestia: blob successfully submitted" id=0900000000000000b25a32154ab00902cfc0269b3239b612ebfe7f7263545119ee7251cc72728142
t=2024-05-30T19:08:34+0000 lvl=info msg="celestia: blob successfully submitted" id=0a00000000000000cb559bc3c6a01b0819460ce86c13165fdc58ac9c81c1e52404f8c4b36097db87
t=2024-05-30T19:08:34+0000 lvl=info msg="celestia: blob request" id=010c0900000000000000b25a32154ab00902cfc0269b3239b612ebfe7f7263545119ee7251cc72728142

Usually, it's better to deploy both OP stack and celestia-node in the same network, but for ease of testing, we will modify the devnet docker-compose file to submit to celestia testnet instead.

For running Alt-DA in celestia testnet mode, we'll use the celestia-node docker image which requires a celestia testnet network like mocha or arabica.

To run a celestia-node in testnet mode, follow the instructions for setting up a light node and changes required to the docker-compose file

Note that in this case, the auth token needs to be configured using the CELESTIA_NODE_AUTH_TOKEN environment variable.

To obtain the auth token for celestia-node, check the documentation for the node type you are using

Once the docker compose file is modified, you can run the devnet with the following command:

DEVNET_ALT_DA="true" GENERIC_ALT_DA="true"  make devnet-up

This will start the devnet in Alt-DA mode with celestia-node running in testnet mode.

You can verify that the devnet is running by checking da-server logs:

docker logs -f ops-bedrock-da-server-1 | grep celestia

Once again, upon successful operation, the logs should contain the following message:

t=2024-05-30T19:08:30+0000 lvl=info msg="Using celestia storage" url=http://da:26658
t=2024-05-30T19:08:32+0000 lvl=info msg="celestia: blob successfully submitted" id=0900000000000000b25a32154ab00902cfc0269b3239b612ebfe7f7263545119ee7251cc72728142
t=2024-05-30T19:08:34+0000 lvl=info msg="celestia: blob successfully submitted" id=0a00000000000000cb559bc3c6a01b0819460ce86c13165fdc58ac9c81c1e52404f8c4b36097db87
t=2024-05-30T19:08:34+0000 lvl=info msg="celestia: blob request" id=010c0900000000000000b25a32154ab00902cfc0269b3239b612ebfe7f7263545119ee7251cc72728142

As an alternative, you may wish to deploy OP stack and celestia-node both to testnets e.g. sepolia and mocha.

In this case, run a celestia light node and da-server, then follow the instructions for creating your own L2 rollup testnet including the following flags:

op-node
      --alt-da.enabled=true
      --alt-da.da-service=true

op-batcher
      --alt-da.enabled=true
      --alt-da.da-service=true

da-server
      --celestia.addr=http://localhost:26658
      --celestia.auth-token=$CELESTIA_NODE_AUTH_TOKEN
      --celestia.namespace=$NAMESPACE

where $CELESTIA_NODE_AUTH_TOKEN is the auth token for the celestia-node and $NAMESPACE is a random valid namespace.

Check the logs for the da-server to see if it is running successfully:

docker logs -f ops-bedrock-da-server-1

Additionally verify that the batcher is able to submit transactions to the celestia-node:

docker logs -f ops-bedrock-op-batcher-1

If you see the following error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x10964251c]

goroutine 1 [running]:
github.com/celestiaorg/celestia-node/api/client.(*Client).Close(0x140005daaf0)
        /Users/tuxcanfly/Work/celestia-node/api/client/client.go:147 +0x2c

it means that the client was not able to connect to the core gRPC server, please ensure your --celestia.core.grpc.addr, --celestia.core.grpc.auth-token and --celestia.core.grpc.tls-enabled flags are correct. Note that boolean flags like --celestia.tls-enabled and --celestia.tls-enabled should be set as --celestia.tls-enabled=false and --celestia.core.grpc.tls-enabled=true respectively, for example.