Skip to main content

Implementing a file archive with the DAL and a Smart Rollup

Experimental

The data availability layer is an experimental feature that is not yet available on Tezos Mainnet. The way the DAL works may change significantly before it is generally available.

The data availability layer (DAL) is a companion peer-to-peer network for the Tezos blockchain, designed to provide additional data bandwidth to Smart Rollups. It allows users to share large amounts of data in a way that is decentralized and permissionless, because anyone can join the network and post and read data on it.

In this tutorial, you will set up a file archive that stores and retrieves files with the DAL. You will learn:

  • How data is organized and shared with the DAL and the reveal data channel
  • How to read data from the DAL in a Smart Rollup
  • How to host a DAL node
  • How to publish data and files with the DAL

Because the DAL is not yet available on Tezos Mainnet, this tutorial uses the Weeklynet test network, which runs on a newer version of the protocol that includes the DAL.

See these links for more information about the DAL:

Prerequisites​

This article assumes some familiarity with Smart Rollups. If you are new to Smart Rollups, see the tutorial Deploy a Smart Rollup.

Set up a Weeklynet environment and account​

Because Weeklynet requires a specific version of the Octez suite, you can't use most wallet applications and installations of the Octez suite with it. Instead, you must set up an environment with a specific version of the Octez suite and use it to create and fund an account. Note that Weeklynet is reset every Wednesday, so you must recreate your environment and account after the network resets.

The easiest way to do this is to use the Docker image that is generated each time Weeklynet is reset and recreated. As another option, you can build the specific version of the Octez suite locally. For instructions, see the Weeklynet page at https://teztnets.com/weeklynet-about.

To set up an environment and account in a Docker container, follow these steps:

  1. From the Weeklynet page, find the Docker command to create a container from the correct Docker image, as in this example:

    docker run -it --entrypoint=/bin/sh tezos/tezos:master_7f3bfc90_20240116181914

    The image tag in this command changes each time the network is reset.

  2. Copy the URL of the public RPC endpoint for Weeklynet, such as https://rpc.weeklynet-2024-01-17.teztnets.com. This endpoint also changes each time the network is reset.

  3. For convenience, you may want to set this endpoint as the value of the ENDPOINT environment variable.

  4. In the container, initialize the Octez client with that endpoint, such as this example:

    octez-client -E https://rpc.weeklynet-2024-01-17.teztnets.com config init

  5. Create an account with the command octez-client gen keys $MY_ACCOUNT, where $MY_ACCOUNT is an alias for your account.

  6. Get the public key hash of the new account by running the command octez-client show address $MY_ACCOUNT.

  7. From the Weeklynet page, open the Weeklynet faucet and send some tez to the account.

Now you can use this account to deploy Smart Rollups.

Install Rust​

To run this tutorial, install Rust by running the following command. The application in this tutorial uses Rust because of its support for WebAssembly (WASM), the language that Smart Rollups use to communicate. Rollups can use any language that has WASM compilation support.

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Then, add WASM as a compilation target for Rust by running this command:

rustup target add wasm32-unknown-unknown

You can see other ways of installing Rust at https://www.rust-lang.org.

Why the DAL?​

The DAL has earned the nickname "Rollup Booster" from its ability to address the last bottleneck Smart Rollups developers could not overcome without sacrificing decentralization: block space. Smart Rollups offload computation from layer 1, but the transactions that they process still need to originate from somewhere.

By default, that "somewhere" is the layer 1 blocks, yet the size of a Tezos block is limited to around 500KBytes. In this model, while Smart Rollups do not compete for layer 1 gas anymore, they still compete for block space.

Additionally, a Smart Rollup can fetch data from an additional source called the reveal data channel, which allows them to retrieve arbitrary data. The reveal channel is a powerful way to share data, because it allows a Smart Rollup operator to post hashes instead of full data files on layer 1. But it is a double-edged sword, because nothing enforces the availability of the data in the first place. Solutions exist to address this challenge, but they are purely off-chain ones, coming with no guarantee from layer 1.

The DAL allows third parties to publish data and have bakers attest that the data is available. When enough bakers have attested that the data is available, Smart Rollups can retrieve the data without the need for additional trusted third-parties.

How the DAL works​

In this tutorial, you create a file archive application that allows clients to upload data to the DAL. You also create a Smart Rollup that listens to the DAL and responds to that data.

The DAL works like this:

  1. Users post data to a DAL node.
  2. The DAL node returns a certificate. This certificate includes a commitment that the data is available and a proof of the data.
  3. Users post the certificate to layer 1 via the Octez client, which is much cheaper than posting the complete data.
  4. When the certificate is confirmed in a block, layer 1 splits the data into shards and assigns those shards to bakers, who verify that the data is available.
  5. Bakers verify that the data is available and attest that the data is available in their usual block attestations to layer 1. They have a certain number of blocks to do so, known as the attestation lag, and if they don't by the end of this period, the certificate is considered bogus and the related data is dropped.
  6. Other DAL nodes get the data from the initial DAL node through the peer-to-peer network.
  7. The Smart Rollup node monitors the blocks and when it sees attested DAL data, it connects to a DAL node to request the data.
  8. The Smart Rollup node stores the data in its durable storage, addressed by its hash. Smart Rollups must store the data because it is available on the DAL for only a short time.
  9. Users who know the hash of the data can download it from the Smart Rollup node.

The overall workflow is summarized in the following figure:

There are many steps in the DAL process, but the most complicated parts (storing and sharing data) are handled automatically by the various daemons in the Octez suite.

The Smart Rollup does not support the DAL

As of today, the Smart Rollup Installer does not support the DAL as a data availability solution. This means we will need to rely on the reveal channel to initialize our Smart Rollup correctly (which is not ideal for a decentralized file archive).

When your environment is ready, get started by going to Part 1: Getting the DAL parameters.

Last updated on