Deploy a DSL Node

Introduction

This guide provides instructions for a deployment using containerized solutions including Docker and Kubernetes. The deployment focuses on the Decentralized component. For deploying more components, refer to the respective guides. It is assumed that the user has a basic understanding of terminal commands and containerization concepts.

The guide is opinionated towards a containerized deployment solution, the source code is available at: https://github.com/RSS3-Network/Node, if one prefers an alternative deployment method.

It's important to note that while the Node itself is production-ready, the deployment methods we outline here may not be universally applicable in all environments. Adaptation of these methods may be necessary to fit the specific requirements of your own setup.

This is different from an RSS3 VSL RPC Node.

Deployment

Prerequisites

Basic command line proficiency
Have your Node registered on the VSL
Have docker, docker-compose installed on the server

Alternative Deployment Methods

You can also choose alternative deployment methods like:

Light Mode

Learn how to deploy an RSS3 Node in light mode.

Serverless

An easy guide for deploying an RSS3 Node on serverless platforms.

Hardware Requirements

Since RSS3 Node's coverage is highly customizable, there is no universal configuration applicable to all.

Our interactive deployment configurator will provide an estimated hardware requirement.

We recommend a minimal configuration of 16 Core, 64 GB RAM, 500 GB SSD * 3 to operate a Node with full coverage to boost your work rate. For that, AlloyDB should be used to replace PostgreSQL, as AlloyDB generally requires fewer resources than a PostgreSQL cluster to achieve the same performance.

Local AI Capability

Running local AI models requires additional resources (mainly GPUs). It is currently in an internal testing phase, where OpenAgent will be part of the Node to provide AI capabilities.

We will release the relevant documentation soon.

Meanwhile, you can try out OpenAgent on your own server.

Deploy via docker-compose

A configuration file is required for deploying your Node.

👍 We strongly recommend that everyone use the interactive deployment configurator to minimize the risk of manual errors.

During your Node registration https://explorer.rss3.io/nodes/register, follow the steps and generate a config.yaml, and download it.
In the same directory as your config.yaml, download the automated deployment script by executing the following command:
curl -s https://raw.githubusercontent.com/RSS3-Network/Node-Automated-Deployer/main/automated_deploy.sh | bash

🎉 And you are done! (Yes it's that simple.)

A successful deployment will generate the following containers:

rss3_node_core: This container is used to process incoming requests from Global Indexers.
rss3_node_broadcaster: This container is used to broadcast the Node's status to Global Indexers.
rss3_node_monitor: This container is used to monitor worker status.
rss3_node_alloydb: This container is used to store data.
rss3_node_redis: This container is used to cache data.
rss3_node_agentdata: This container is used to provide AI capabilities.
worker containers: These containers are used to index data from their corresponding sources. (if you have enabled at least 1 decentralized/federated worker)

Sample config.yaml

For your reference, a config.yaml that contains the Decentralized component only should look like this:

# DO NOT change this value except for local debugging purposes.
environment: production
 
# `discovery` is used for Node discovery and verification.
discovery:
  operator:
    # `evm_address` is the wallet address used during your registration on the RSS3 Explorer.
    evm_address: 0x...address
    # The `signature` obtained during registration, it will be used to verify your ownership of this Node.
    # You MUST NOT share the signature with anyone.
    signature: 0x...signature
  server:
    # `endpoint` is how how Global Indexers access your Node.
    # You can use a domain or an IP address.
    endpoint: https://your.node.com
    # If your Node is registered on the VSL Testnet, use https://gi.rss3.dev.
    # Otherwise, DO NOT change this value.
    global_indexer_endpoint: https://gi.rss3.io
    # Use access_token to protect your Node from unauthorized access.
    access_token: your_access_token
 
# Database configuration
database:
  # `coverage_period` is the number of months that the database will store indexed data, the minimum value is 3.
  coverage_period: 3
  # `driver` is the database driver used by the Node, currently only `postgres` is supported.
  driver: postgres
  # `partition` is used to enable the partition feature of the database.
  partition: true
  # `uri` is the connection string of the database.
  uri: postgres://postgres:password@localhost:5432/postgres
 
# A redis is required for monitor service and will significantly improve the indexing performance of some workers.
redis:
  endpoint: localhost:6379
  username:
  password:
 
# `endpoints` are data access points for Workers.
# Endpoints defined here can be referenced in the configuration below.
# For example,
# - Blockchain networks require RPC endpoints,
# - Arweave requires a gateway endpoint,
# - Farcaster requires a Hubble.
# - Mastodon requires a public URL forwarding to specific localhost port.
endpoints:
  vsl:
    url: https://rpc.rss3.io
  arweave:
    url: https://arweave.net
 
# `component` is used to split different types of networks.
component:
  # `ai` type includes the worker indexing data in AI format.
  ai:
    id: agentdata-core
    worker: core
    endpoint: https://your.agentdata.com/
    parameters:
      agentdata_db_url:
      openai_api_key:
      ollama_host:
      kaito_api_token:
      twitter:
        bearer_token:
        api_key:
        api_secret:
        access_token:
        access_token_secret:
  # `decentralized` component includes workers indexing data from decentralized networks such as blockchain networks, Arweave, etc.
  decentralized:
    # Each configuration here initializes a worker.
    # `id` is the unique identifier
    - id: vsl-core
      # `network` is the network that the worker is indexing from.
      network: vsl
      # `endpoint` is the data access endpoint used by the worker.
      # You can reference the endpoint defined above in the `endpoints` section.
      # You can also use a direct URL, such as `https://rpc.rss3.io`.
      endpoint: vsl
      # `worker` is the actual worker that processes the data.
      # You can find the list of available workers here: https://github.com/RSS3-Network/Node/blob/develop/README.md#supported-networks-and-workers.
      worker: core
    - id: arweave-mirror
      network: arweave
      endpoint: arweave
      worker: mirror
      # `ipfs_gateways` is used to specify the IPFS gateways used by the worker.
      # Supplying multiple gateways may improve the indexing performance and reliability.
      ipfs_gateways:
        - https://ipfs.io
        - https://cloudflare-ipfs.com
      parameters:
        # `concurrent_block_requests` is used to specify the number of concurrent block requests.
        concurrent_block_requests: 2
  # `rss` network type includes the worker indexing data in RSS format.
  rss:
    id: rsshub-core
    network: rsshub
    worker: core
    endpoint: https://your.rsshub.com/
    parameters:
      authentication:
        access_key:
  # `federated` network type includes workers indexing data from federated networks such as ActivityPub, Atprotocol.
  federated:
    # mastodon
    - id: mastodon-mastodon
      network: mastodon
      worker: mastodon
      endpoint: mastodon
      parameters:
        relay_url_list: [ "https://relay.fedi.buzz/instance/mastodon.social"]
        port: 8181

Deploy via Kubernetes

Prerequisites

Have kind installed
Have Helm installed

Create a Kubernetes cluster via kind

Create our Kubernetes cluster with the following command

kind create cluster --name node

Deploy using RSS3's Helm Chart

Since Helm uses a chart file for deployment. We can easily deploy a Node by using the chart files provided.

helm repo add rss3 https://rss3-network.github.io/helm-charts
helm repo update

Get Help

In case you run into any issues, please reach out to us on Discord.

Conclusion

This guide provides basic instructions for deploying an RSS3 Node in containerized environments. For more advanced configurations, refer to the respective Docker or Kubernetes documentation.

This guide adopts an opinionated stance in favor of containerized deployment. We advocate for this approach due to its benefits in ensuring consistency and scalability, promoting ease of portability across various environments. However, we appreciate the diverse preferences and needs within our community, for those who wish to compile and run the application locally, the source code is available on: https://github.com/RSS3-Network/Node.

Light Mode

Serverless

On this page