RSS3 Documentation
Deployment

Deploy a DSL Node

An easy guide for deploying an RSS3 Node on the Data Sublayer.

Introduction

This guide provides instructions for a deployment using containerized solutions including Docker and Kubernetes. The deployment focuses on the Decentralized component. For deploying more components, refer to the respective guides. It is assumed that the user has a basic understanding of terminal commands and containerization concepts.

The guide is opinionated towards a containerized deployment solution, the source code is available at: https://github.com/RSS3-Network/Node, if one prefers an alternative deployment method.

It's important to note that while the Node itself is production-ready, the deployment methods we outline here may not be universally applicable in all environments. Adaptation of these methods may be necessary to fit the specific requirements of your own setup.

This is different from an RSS3 VSL RPC Node.

Deployment

Prerequisites

Alternative Deployment Methods

You can also choose alternative deployment methods like:

Hardware Requirements

Since RSS3 Node's coverage is highly customizable, there is no universal configuration applicable to all.

Our interactive deployment configurator will provide an estimated hardware requirement.

We recommend a minimal configuration of 16 Core, 64 GB RAM, 500 GB SSD * 3 to operate a Node with full coverage to boost your work rate. For that, AlloyDB should be used to replace PostgreSQL, as AlloyDB generally requires fewer resources than a PostgreSQL cluster to achieve the same performance.

Local AI Capability

Running local AI models requires additional resources (mainly GPUs). It is currently in an internal testing phase, where OpenAgent will be part of the Node to provide AI capabilities.

We will release the relevant documentation soon.

Meanwhile, you can try out OpenAgent on your own server.

Deploy via docker-compose

A configuration file is required for deploying your Node.

👍 We strongly recommend that everyone use the interactive deployment configurator to minimize the risk of manual errors.

  1. During your Node registration https://explorer.rss3.io/nodes/register, follow the steps and generate a config.yaml, and download it.

  2. In the same directory as your config.yaml, download the automated deployment script by executing the following command:

    curl -s https://raw.githubusercontent.com/RSS3-Network/Node-Automated-Deployer/main/automated_deploy.sh | bash

🎉 And you are done! (Yes it's that simple.)

A successful deployment (if you have enabled at least 1 decentralized worker) will generate the following containers:

  1. rss3_node_core: This container is used to process incoming requests from Global Indexers.
  2. rss3_node_broadcaster: This container is used to broadcast the Node's status to Global Indexers.
  3. rss3_node_monitor: This container is used to monitor worker status.
  4. rss3_node_alloydb: This container is used to store data.
  5. rss3_node_redis: This container is used to cache data.
  6. worker containers: These containers are used to index data from their corresponding sources.

Upgrading Your Node

Maintaining an up-to-date Node is crucial for optimal performance and security. We recommend regularly checking the Changelog for updates and new features.

When upgrading, ensure that you update your config.yaml file or modify the configurations in accordance with the latest Deployment Guide.

Before proceeding with any upgrade, carefully review the release notes, particularly for major version changes. Major versions may introduce breaking changes that require additional steps or modifications.

Upgrade Process for v1.0.x or older to v1.1.x

When upgrading from v1.0.x or older versions to v1.1.x, please note that the rss component has undergone significant changes. Update your config.yaml file as follows:

component:
    rss:
---     id: rss-rsshub
+++     id: rsshub-core
---     network: rss
+++     network: rsshub
---     worker: rsshub
+++     worker: core

For operators using the automated deployer, the upgrade process is:

  1. Update your config.yaml file in the config/ directory.

  2. Execute the following command in the same directory as your existing docker-compose.yaml file to initiate the automated upgrade process:

    curl -s https://raw.githubusercontent.com/RSS3-Network/Node-Automated-Deployer/main/automated_deploy.sh | bash

This command will automatically upgrade all necessary components of your Node.

Sample config.yaml

For your reference, a config.yaml that contains the Decentralized component only should look like this:

# DO NOT change this value except for local debugging purposes.
environment: production
 
# `discovery` is used for Node discovery and verification.
discovery:
  operator:
    # `evm_address` is the wallet address used during your registration on the RSS3 Explorer.
    evm_address: 0x...address
    # The `signature` obtained during registration, it will be used to verify your ownership of this Node.
    # You MUST NOT share the signature with anyone.
    signature: 0x...signature
  server:
    # `endpoint` is how how Global Indexers access your Node.
    # You can use a domain or an IP address.
    endpoint: https://your.node.com
    # If your Node is registered on the VSL Testnet, use https://gi.rss3.dev.
    # Otherwise, DO NOT change this value.
    global_indexer_endpoint: https://gi.rss3.io
    # Use access_token to protect your Node from unauthorized access.
    access_token: your_access_token
 
# Database configuration
database:
  # `coverage_period` is the number of months that the database will store indexed data, the minimum value is 3.
  coverage_period: 3
  # `driver` is the database driver used by the Node, currently only `postgres` is supported.
  driver: postgres
  # `partition` is used to enable the partition feature of the database.
  partition: true
  # `uri` is the connection string of the database.
  uri: postgres://postgres:password@localhost:5432/postgres
 
# A redis is required for monitor service and will significantly improve the indexing performance of some workers.
redis:
  endpoint: localhost:6379
  username:
  password:
 
# `endpoints` are data access points for Workers.
# Endpoints defined here can be referenced in the configuration below.
# For example,
# - Blockchain networks require RPC endpoints,
# - Arweave requires a gateway endpoint,
# - Farcaster requires a Hubble.
# - Mastodon requires a public URL forwarding to specific localhost port.
endpoints:
  vsl:
    url: https://rpc.rss3.io
  arweave:
    url: https://arweave.net
 
# `component` is used to split different types of networks.
component:
  # `decentralized` component includes workers indexing data from decentralized networks such as blockchain networks, Arweave, etc.
  decentralized:
    # Each configuration here initializes a worker.
    # `id` is the unique identifier
    - id: vsl-core
      # `network` is the network that the worker is indexing from.
      network: vsl
      # `endpoint` is the data access endpoint used by the worker.
      # You can reference the endpoint defined above in the `endpoints` section.
      # You can also use a direct URL, such as `https://rpc.rss3.io`.
      endpoint: vsl
      # `worker` is the actual worker that processes the data.
      # You can find the list of available workers here: https://github.com/RSS3-Network/Node/blob/develop/README.md#supported-networks-and-workers.
      worker: core
    - id: arweave-mirror
      network: arweave
      endpoint: arweave
      worker: mirror
      # `ipfs_gateways` is used to specify the IPFS gateways used by the worker.
      # Supplying multiple gateways may improve the indexing performance and reliability.
      ipfs_gateways:
        - https://ipfs.io
        - https://cloudflare-ipfs.com
      parameters:
        # `concurrent_block_requests` is used to specify the number of concurrent block requests.
        concurrent_block_requests: 2

Deploy via Kubernetes

Prerequisites

  • Have kind installed
  • Have Helm installed

Create a Kubernetes cluster via kind

Create our Kubernetes cluster with the following command

kind create cluster --name node

Deploy using RSS3's Helm Chart

Since Helm uses a chart file for deployment. We can easily deploy a Node by using the chart files provided.

helm repo add rss3 https://rss3-network.github.io/helm-charts
helm repo update

Get Help

In case you run into any issues, please reach out to us on Discord.

Conclusion

This guide provides basic instructions for deploying an RSS3 Node in containerized environments. For more advanced configurations, refer to the respective Docker or Kubernetes documentation.

This guide adopts an opinionated stance in favor of containerized deployment. We advocate for this approach due to its benefits in ensuring consistency and scalability, promoting ease of portability across various environments. However, we appreciate the diverse preferences and needs within our community, for those who wish to compile and run the application locally, the source code is available on: https://github.com/RSS3-Network/Node.