Implementing a High-Performance Arweave Bundle Parser

Background

After Lens Protocol announced the launch of Momoka, a data availability layer that uses Arweave Bundle, we immediately started working on incorporating it into the RSS3 Network. During the process, we realized that existing solutions for parsing bundled transactions do not provide a satisfactory result: they all require that the complete transactions be loaded (either from disk or memory).

For a larger transaction, take nhErSntmWK5gP5CFDNgtJ1vQqgW34kD5siZlG1PU6HQ as an example, it causes the following problems:

It must be loaded into the memory and occupies a whopping 1.05 GB of memory, plus additional allocations generated by the parser.
It means that the RSS3 Network will expect a long delay in delivering the latest data generated by applications that generate bundled transactions.

Clearly, scaling our hardware on-the-fly to align with bundle sizes is not feasible, so often it leads to out of memory (OOM).

A bundle transaction that consumes over 5 GB memory.

What is an Arweave Bundle?

The Arweave Bundle specification currently has two versions, ANS-102 (JSON serialization) and ANS-104 (binary serialization). We focus on ANS-104 here.

A bundled transaction aggregates multiple transactions into one, thus reducing transaction costs. The only requirement is to fill in 2 tags: Bundle-Format and Bundle-Version. Here is an example: rmJZGbi9_UY2yvQPnnwaEGMoLqIVmD5-XJi_aH6FvR0.

{
  "tags": [
    {
      // Bundle-Format 
      "name": "QnVuZGxlLVZlcnNpb24",
      // binary
      "value": "Mi4wLjA"
    },
    {
      // Bundle-Version
      "name": "UHJvdG9jb2wtbmFtZQ",
      // 2.0.0
      "value": "QkFS"
    }
  ]
}

Implementation

We implement the Arweave Bundle parser in Golang.

The implementation is mainly based on byte streams to avoid loading the entire bundle into memory. We handle the TCP data stream directly in memory, with a mechanism to control the memory allocation. In Golang, io.Reader is the interface representing a readable byte stream.

type Decoder struct {
	reader io.Reader
}

func NewDecoder(reader io.Reader) *Decoder {
	decoder := Decoder{
		reader: reader,
	}

	return &decoder
}

We then proceed to reading the bundle header and body separately.

Reading Bundle Header

The bundle header uses 32 bytes to record the number of data Item. Each subsequent 64 bytes are used to record a pair of Size and ID for every Item in the bundle. So conveniently, we can reuse the same buffer since all 3 parts are 32 bytes in length.

type Decoder struct {
	reader    io.Reader
	dataInfos []DataInfo
}

func (d *Decoder) DecodeDataInfos() ([]DataInfo, error) {
	buffer := make([]byte, 32)

	if _, err := io.ReadFull(d.reader, buffer); err != nil {
		return nil, fmt.Errorf("read number of infos: %w", err)
	}

	d.dataInfos = make([]DataInfo, binary.LittleEndian.Uint64(buffer))

	for index := 0; index < len(d.dataInfos); index++ {
		var dataInfo DataInfo

		if _, err := io.ReadFull(d.reader, buffer); err != nil {
			return nil, fmt.Errorf("read size of info %d: %w", index, err)
		}

		dataInfo.Size = binary.LittleEndian.Uint64(buffer)

		if _, err := io.ReadFull(d.reader, buffer); err != nil {
			return nil, fmt.Errorf("read id of info %d: %w", index, err)
		}

		dataInfo.ID = base64.RawURLEncoding.EncodeToString(buffer)

		d.dataInfos[index] = dataInfo
	}

	return d.dataInfos, nil
}

Reading Bundle Body

The bundle body that contains the actual transaction data is included as Items, in the previous step we already obtained their Size and ID. If needed, we can actually use ID to skip the Item we don’t need, or terminate the process to expedite parsing.

We maintain a Cursor variable and provide a Next() bool function to avoid triggering the io.EOF error.

type Decoder struct {
	reader    io.Reader
	dataInfos []DataInfo
	cursor    int
}

func (d *Decoder) Next() bool {
	return d.cursor < len(d.dataInfos)
}

We now begin parsing the Items.

Step 1. Reading Signature

The first thing to handle is the Signature Type and Signature fields. Signature Type is fixed at 2 bytes, which dictates the length of the corresponding Signature.

We obtained the length table by looking into Irys’s Rust SDK implementation.

Type	Signer	Signature Length	Public Key Length
1	Arweave	512	512
2	Curve 25519	64	32
3	Ethereum	64 + 1	65
4	Solana	64	32
5	Aptos	64	32
6	Multi Aptos	64 * 32 + 4	32 * 32 + 1
7	Typed Ethereum	64 + 1	42

Since Go does not have an enum type, we use github.com/dmarkham/enumer to generate a mapping between unit and string.

var ErrorUnsupportedType = errors.New("unsupported type")

//go:generate go run --mod=mod github.com/dmarkham/enumer --values --type=Type --linecomment --output type_string.go
type Type uint16

const (
	TypeArweave       Type = iota + 1 // arweave
	TypeED25519                       // ed25519
	...
)

func (t Type) SignatureLength() (int, error) {
	switch t {
	case TypeArweave:
		return 512, nil
	case TypeED25519:
		return 64, nil
	...
	default:
		return 0, fmt.Errorf("%w: %d", ErrorUnsupportedType, t)
	}
}

func (t Type) PublicKeyLength() (int, error) {
	switch t {
	case TypeArweave:
		return 512, nil
	case TypeED25519:
		return 32, nil
	...
	default:
		return 0, fmt.Errorf("%w: %d", ErrorUnsupportedType, t)
	}
}

type DataItem struct {
	SignatureType signature.Type
	Signature     []byte
}

As mentioned earlier, the Signature Type is 2 bytes in length. Let’s read it and initialize the buffer for the Signature after parsing it into uint16.

func (d *Decoder) decodeDataItemSignature(dataItem *DataItem) error {
	buffer := make([]byte, 2)
	if _, err := io.ReadFull(d.buffer, buffer); err != nil {
		return err
	}

	dataItem.SignatureType = signature.Type(binary.LittleEndian.Uint16(buffer))

	signatureLength, err := dataItem.SignatureType.SignatureLength()
	if err != nil {
		return err
	}

	dataItem.Signature = make([]byte, signatureLength)
	if _, err := io.ReadFull(d.buffer, dataItem.Signature); err != nil {
		return err
	}

	return nil
}

Step 2. Reading Owner

The length and format of the Owner are also dictated by Signature Type, which is the length of the signer’s public key.

func (d *Decoder) decodeDataItemOwner(dataItem *DataItem) error {
	publicKeyLength, err := dataItem.SignatureType.PublicKeyLength()
	if err != nil {
		return err
	}

	dataItem.Owner = make([]byte, publicKeyLength)
	if _, err := io.ReadFull(d.buffer, dataItem.Owner); err != nil {
		return err
	}

	return nil
}

Step 3. Reading Target

Unlike the Owner field, Target is an optional field as an Arweave transaction may not have a recipient. The first byte is the presence byte used to indicate whether this field is empty, so the length is either 1 or 32 + 1.

Before parsing the target, we can implement a common function to parse an optional field to be reused later.

func (d *Decoder) decodeDataItemOptionField(buffer []byte) (bool, error) {
	if _, err := io.ReadFull(d.buffer, buffer[:1]); err != nil {
		return false, err
	}

	switch flag := buffer[0]; flag {
	case 0x0:
		return false, nil
	case 0x1:
		_, err := io.ReadFull(d.buffer, buffer[1:])

		return true, err
	default:
		return false, fmt.Errorf("invalid presence byte: %d", flag)
	}
}

And we only read the optional field when it’s present.

func (d *Decoder) decodeDataItemTarget(dataItem *DataItem) error {
	buffer := make([]byte, 32+1)

	presence, err := d.decodeDataItemOptionField(buffer)
	if err != nil {
		return err
	}

	if presence {
		dataItem.Target = buffer[1:]
	}

	return nil
}

Step 4. Reading Anchor

Anchor as an optional field is used to prevent replay attacks. It too follows the 1 presence byte + 32 data bytes design.

func (d *Decoder) decodeDataItemAnchor(dataItem *DataItem) error {
	buffer := make([]byte, 32+1)

	presence, err := d.decodeDataItemOptionField(buffer)
	if err != nil {
		return err
	}

	if presence {
		dataItem.Anchor = buffer[1:]
	}

	return nil
}

Step 5. Reading Tags

Three fields are present here,

count: indicates the number of Tags.
size: indicate the number of bytes.
block: the actual data for Tags.

Tags conform to Apache Avro, with the following schema.

{
  "type": "array",
  "items": {
    "type": "record",
    "name": "Tag",
    "fields": [
      { "name": "name", "type": "bytes" },
      { "name": "value", "type": "bytes" }
    ]
  }
}

Count and size are not needed here since each tag is prefixed by its size.

func (d *Decoder) decodeDataItemTags(dataItem *DataItem) error {
	buffer := make([]byte, 8)

	// Read the count, but discard it
	if _, err := io.ReadFull(d.buffer, buffer); err != nil {
		return err
	}

	// Read the size, but discard it
	if _, err := io.ReadFull(d.buffer, buffer); err != nil {
		return err
	}

	dataItem.Tags = make([]byte, binary.LittleEndian.Uint16(buffer))

	if _, err := io.ReadFull(d.buffer, dataItem.Tags); err != nil {
		return err
	}

	return nil
}

We provide an additional function DecodeDataItemTags for Decoder to parse it into []DataTag, where each DataTag is stored as bytes. We then use github.com/hamba/avro/v2 for deserialization and verification.

type DataTag struct {
	Name  []byte `avro:"name"`
	Value []byte `avro:"value"`
}

const SchemaDataTags = `{"type":"array","items":{"type":"record","name":"Tag","fields":[{"name":"name","type":"bytes"},{"name":"value","type":"bytes"}]}}`

type Decoder struct {
    reader, buffer io.Reader
    dataInfos      []DataInfo
    dataTagsSchema avro.Schema
    cursor         int
}

func (d *Decoder) DecodeDataItemTags(dataItem DataItem) ([]DataTag, error) {
    var tags []DataTag

    if err := avro.Unmarshal(d.dataTagsSchema, dataItem.Tags, &tags); err != nil {
       return nil, err
    }

    return tags, nil
}

func NewDecoder(reader io.Reader) *Decoder {
    decoder := Decoder{
       reader:         reader,
       dataTagsSchema: lo.Must(avro.Parse(SchemaDataTags)),
    }

    return &decoder
}

Step 6. Reading Item

To correctly maintain the state of the cursor and TCP stream, we need to ensure that the current Item is completely consumed before parsing the next Item.

To do this, we extend io.Reader to satisfy io.ReadCloser interface and discard all unwanted data in the custom Close implementation.

var _ io.ReadCloser = (*Reader)(nil)

type Reader struct {
	reader io.Reader
}

func (r *Reader) Read(data []byte) (n int, err error) {
	return r.reader.Read(data)
}

func (r *Reader) Close() error {
	_, err := io.Copy(io.Discard, r.reader)

	return err
}

Conclusion

Now let’s put all the parsing logic blocks together and test it out.

func (d *Decoder) DecodeDataItem() (*DataItem, error) {
	dataItemInfo := d.dataInfos[d.cursor]

	d.buffer = io.LimitReader(d.reader, int64(dataItemInfo.Size))

	var dataItem DataItem

	if err := d.decodeDataItemSignature(&dataItem); err != nil {
		return nil, fmt.Errorf("decode signature: %w", err)
	}

	if err := d.decodeDataItemOwner(&dataItem); err != nil {
		return nil, fmt.Errorf("decode owner: %w", err)
	}

	if err := d.decodeDataItemTarget(&dataItem); err != nil {
		return nil, fmt.Errorf("decode target: %w", err)
	}

	if err := d.decodeDataItemAnchor(&dataItem); err != nil {
		return nil, fmt.Errorf("decode anchor: %w", err)
	}

	if err := d.decodeDataItemTags(&dataItem); err != nil {
		return nil, fmt.Errorf("decode tags: %w", err)
	}

	dataItem.Reader = Reader{
		reader: d.buffer,
	}

	d.cursor++

	return &dataItem, nil
}

io.ReadCloser ensures that we can invoke the Close function to discard unwanted data without reading it.

func TestDecoder(t *testing.T) {
	t.Parallel()

	response, err := http.Get(fmt.Sprintf("https://arweave.net/rmJZGbi9_UY2yvQPnnwaEGMoLqIVmD5-XJi_aH6FvR0", testcase.arguments.transactionID))
	require.NoError(t, err)

	defer response.Body.Close()

	decoder := bundle.NewDecoder(response.Body)

	dataInfos, err := decoder.DecodeDataInfos()
	require.NoError(t, err)

	t.Logf("Data Infos: %d", len(dataInfos))

	for _, dataInfo := range dataInfos {
		t.Logf("%+v", dataInfo)
	}

	t.Logf("Data Items")

	for decoder.Next() {
		dataItem, err := decoder.DecodeDataItem()
		require.NoError(t, err)

		require.NoError(t, dataItem.Reader.Close())

		t.Logf("%+v", dataItem)
	}
}

=== RUN   TestDecoder
=== PAUSE TestDecoder
=== CONT  TestDecoder
--- PASS: TestDecoder (14.57s)
=== RUN   TestDecoder/rmJZGbi9_UY2yvQPnnwaEGMoLqIVmD5-XJi_aH6FvR0
=== PAUSE TestDecoder/rmJZGbi9_UY2yvQPnnwaEGMoLqIVmD5-XJi_aH6FvR0
=== CONT  TestDecoder/rmJZGbi9_UY2yvQPnnwaEGMoLqIVmD5-XJi_aH6FvR0
    decoder_test.go:51: Data Infos: 537
    decoder_test.go:54: {Size:1090 ID:V_DfH1RexW2-sx_EAeT7rDqmG98prvHyLHxughywKFk}
    decoder_test.go:54: {Size:162 ID:DHKTfsFOYGHbtH8HL1LK1C-QuRhd9708KcnPgWQhsPs}
    decoder_test.go:54: {Size:1090 ID:FWf-0EuT4ZLRRiz-8wqXBGYjkxSuAmRUejoxSukDItg}
    decoder_test.go:54: {Size:162 ID:zG3ZVjCtrvfk2A6b5uMrr2hrxAySVRop4j7VytCJEqA}
    decoder_test.go:54: {Size:265 ID:lDYGt-ROTfZ6IRJQALOVg8uJx9qgI7maegobZ7sLQXA}

...
    decoder_test.go:68: &{SignatureType:ed25519 Signature:[171 228 249 89 209 224 77 82 218 31 190 32 109 148 155 229 115 25 90 169 57 169 140 69 124 39 77 149 39 250 153 43 248 35 205 241 183 101 141 212 153 223 165 179 135 222 160 1 212 175 134 95 9 12 229 228 215 78 189 34 168 130 79 14] Owner:[226 241 253 235 8 217 10 81 201 63 195 101 165 114 118 193 22 92 206 10 155 130 187 114 132 32 65 38 212 208 82 5] Target:[] Anchor:[50 115 53 56 115 104 103 80 103 99 70 83 84 77 116 110 106 48 111 99 55 111 89 78 54 109 69 81 80 105 87 111] Tags:[] Reader:{reader:0x14000607e90}}
    decoder_test.go:68: &{SignatureType:arweave Signature:[32 180 134 38 164 34 29 199 137 46 214 129 87 104 224 149 177 56 99 65 86 65 128 60 148 5 66 156 79 88 19 121 103 49 148 5 119 112 230 210 125 4 10 150 237 244 24 72 143 86 182 167 1 195 112 44 24 84 133 104 103 62 37 66 121 45 137 102 30 173 30 57 96 190 246 212 120 36 200 91 81 156 234 169 225 127 61 5 136 193 199 145 116 218 225 40 202 166 71 204 123 149 52 33 217 158 238 249 237 162 206 86 97 243 85 27 198 215 163 69 184 49 21 245 225 189 179 107 221 21 239 213 232 130 36 153 47 88 191 191 8 244 181 101 226 5 212 202 101 18 174 213 219 83 129 71 189 152 197 38 184 154 0 210 217 92 52 197 27 95 94 60 42 219 51 158 203 235 236 54 253 244 128 250 150 34 63 43 129 50 72 244 221 35 3 120 33 130 150 88 60 253 182 106 235 75 240 189 65 160 210 108 128 209 181 111 81 110 97 161 129 196 191 24 138 184 84 226 7 161 54 138 157 155 175 89 102 167 211 121 77 126 65 64 34 58 121 158 135 197 254 104 162 157 138 164 15 84 65 174 4 180 191 205 112 231 212 165 182 141 220 221 248 107 180 146 195 165 227 136 129 130 18 23 47 45 80 71 192 131 192 78 177 24 110 111 190 2 95 229 248 237 219 223 150 149 28 88 179 139 6 132 249 136 36 87 126 106 243 105 202 138 27 85 124 128 237 8 70 227 205 191 233 213 159 198 45 26 236 67 35 12 253 232 87 240 151 114 232 4 233 173 66 23 93 109 49 231 27 137 219 116 88 240 95 224 56 133 92 116 93 58 89 44 58 124 141 13 227 216 226 75 109 56 9 174 236 137 48 164 185 190 148 224 79 0 211 227 153 135 202 85 115 67 115 84 214 143 166 41 245 114 111 128 101 119 127 188 247 229 219 132 125 251 3 203 106 15 13 36 140 171 75 3 37 204 254 117 252 21 87 5 213 94 214 44 114 205 35 142 20 33 16 88 38 70 247 159 47 84 163 186 154 15 187 0 155 37 227 177 188 232 82 195 21 112 76 208 21 122 152 245 188 49 43 33 148 210 86 190 88 61 143 189 196 207 34 168 18 122 85 252 67 75 178 8 53 7 170 85 89 41] Owner:[147 114 74 21 99 128 156 233 153 32 87 240 231 230 106 210 162 85 171 153 66 224 51 232 9 167 230 151 38 109 12 65 47 223 195 198 141 146 241 47 22 206 79 100 120 21 226 17 71 3 170 132 133 96 7 69 119 192 116 149 61 9 224 20 16 54 224 55 70 153 228 27 78 176 128 251 150 104 32 241 149 235 14 19 195 70 130 173 207 76 16 105 18 20 112 29 99 183 185 137 129 189 55 172 218 207 243 141 75 32 84 99 232 152 113 120 157 244 52 204 207 105 40 167 107 129 154 77 47 23 146 62 73 173 41 242 41 36 205 138 199 126 15 113 179 30 163 99 144 155 201 204 231 243 16 96 191 64 28 183 64 22 137 199 76 187 137 193 48 3 216 252 220 109 206 190 159 31 73 59 214 210 53 84 65 230 133 32 177 133 239 12 97 85 146 56 2 130 177 148 30 94 200 34 59 85 254 127 93 19 36 57 200 24 142 22 73 194 232 108 127 208 74 220 45 211 70 22 78 47 175 8 139 234 156 24 252 121 41 112 244 82 46 144 41 113 231 155 122 182 229 119 6 73 193 14 121 186 213 91 71 250 240 9 79 83 206 205 57 97 11 183 193 106 163 109 6 224 254 9 247 209 183 35 30 26 172 161 155 240 23 71 146 129 43 127 174 8 48 192 215 26 232 37 234 29 211 171 130 173 99 225 128 180 19 168 201 155 193 235 126 91 130 83 4 9 213 51 142 209 214 217 115 10 157 40 195 28 101 72 206 254 105 244 121 231 162 107 32 39 222 172 71 255 151 171 166 209 142 79 252 139 24 103 233 189 234 33 192 249 71 0 127 122 117 219 203 164 151 230 76 244 52 10 156 228 194 140 165 7 13 49 148 135 12 156 14 122 96 182 104 133 42 243 139 156 204 167 200 89 180 49 234 44 116 6 122 208 232 178 135 188 183 96 86 52 230 186 120 175 107 14 243 172 61 133 110 69 199 237 233 24 200 223 153 5 223 23 52 95 241 184 202 24 152 163 66 157 175 211 176 177 79 215 140 201 19 21 39 43 121 87 6 53 186 136 243 6 197 197 90 125 163 111 74 178 128 134 173 228 49 108 103 206 123 228 208 147 2 204 61 90 108 222 154 116 81 171 236 72 121 65] Target:[] Anchor:[85 112 116 115 84 53 112 87 77 57 67 74 73 121 108 76 55 73 90 77 51 83 81 72 56 70 120 122 113 104 79 110] Tags:[] Reader:{reader:0x14000607f20}}
    decoder_test.go:68: &{SignatureType:ed25519 Signature:[125 165 116 37 159 249 25 253 65 146 31 217 145 244 161 142 99 231 94 25 23 137 125 240 197 163 197 22 120 159 97 80 83 155 233 27 210 19 36 202 101 125 31 109 174 42 106 202 53 52 92 47 234 19 11 191 120 153 200 112 207 219 184 9] Owner:[226 241 253 235 8 217 10 81 201 63 195 101 165 114 118 193 22 92 206 10 155 130 187 114 132 32 65 38 212 208 82 5] Target:[] Anchor:[98 79 76 80 85 72 88 117 105 53 87 114 86 99 71 107 76 90 117 109 89 102 113 71 70 69 114 57 102 110 104 79] Tags:[] Reader:{reader:0x14000607fe0}}
    decoder_test.go:68: &{SignatureType:arweave Signature:[77 255 214 45 96 117 125 96 117 58 225 68 96 138 249 5 128 233 181 151 67 198 172 146 250 244 162 177 233 225 49 53 55 107 39 189 123 208 136 172 139 102 93 27 27 197 231 41 8 94 246 168 239 229 58 176 196 210 123 106 57 106 240 228 237 12 245 46 42 6 1 209 186 200 33 63 87 9 97 5 130 81 129 27 124 199 79 244 125 48 217 2 143 214 102 174 254 210 120 128 127 66 126 151 167 10 130 190 235 139 38 18 19 21 40 155 186 153 28 234 57 28 146 185 220 16 61 226 27 110 234 217 18 205 240 15 54 143 111 164 159 189 103 176 30 202 13 86 61 171 43 48 200 195 91 179 178 251 46 229 127 134 97 189 37 14 210 66 116 241 194 229 227 48 206 81 107 137 190 2 104 148 251 54 132 248 250 153 30 205 22 168 213 42 118 201 7 28 13 21 133 177 19 9 167 92 197 71 122 35 35 115 255 44 199 75 210 175 108 170 37 66 199 112 151 77 82 105 198 113 100 13 76 198 80 20 188 2 17 156 227 230 22 124 160 109 101 172 177 89 151 1 128 156 24 90 236 145 78 98 166 137 97 41 5 192 24 157 167 3 181 90 49 247 153 43 212 255 195 79 106 224 41 41 107 139 252 104 97 3 4 110 13 84 66 253 40 194 63 131 103 232 217 50 144 157 72 119 125 58 61 123 62 74 217 251 230 84 36 48 88 92 197 209 57 117 248 229 108 83 101 200 85 70 91 237 84 84 217 29 209 5 73 210 103 211 22 206 243 229 224 164 94 4 130 149 155 132 39 215 207 35 247 63 93 53 64 225 87 250 213 191 31 131 82 251 231 65 125 235 48 23 57 38 200 97 216 81 130 151 21 98 198 163 204 170 38 23 159 119 138 103 204 214 161 192 104 196 74 2 87 177 235 34 129 53 26 153 169 199 164 19 42 223 214 230 183 37 237 221 206 76 110 4 183 191 137 191 98 246 237 158 15 255 139 64 105 117 87 61 153 220 250 130 113 127 202 189 26 46 212 247 128 104 200 85 6 30 128 17 182 87 131 96 53 7 147 199 145 180 24 149 223 226 203 105 123 227 85 5 207 14 142 219 1 138 212 40 214 88 118 76 221 163 17 171 164 87 155 98 41 24] Owner:[147 114 74 21 99 128 156 233 153 32 87 240 231 230 106 210 162 85 171 153 66 224 51 232 9 167 230 151 38 109 12 65 47 223 195 198 141 146 241 47 22 206 79 100 120 21 226 17 71 3 170 132 133 96 7 69 119 192 116 149 61 9 224 20 16 54 224 55 70 153 228 27 78 176 128 251 150 104 32 241 149 235 14 19 195 70 130 173 207 76 16 105 18 20 112 29 99 183 185 137 129 189 55 172 218 207 243 141 75 32 84 99 232 152 113 120 157 244 52 204 207 105 40 167 107 129 154 77 47 23 146 62 73 173 41 242 41 36 205 138 199 126 15 113 179 30 163 99 144 155 201 204 231 243 16 96 191 64 28 183 64 22 137 199 76 187 137 193 48 3 216 252 220 109 206 190 159 31 73 59 214 210 53 84 65 230 133 32 177 133 239 12 97 85 146 56 2 130 177 148 30 94 200 34 59 85 254 127 93 19 36 57 200 24 142 22 73 194 232 108 127 208 74 220 45 211 70 22 78 47 175 8 139 234 156 24 252 121 41 112 244 82 46 144 41 113 231 155 122 182 229 119 6 73 193 14 121 186 213 91 71 250 240 9 79 83 206 205 57 97 11 183 193 106 163 109 6 224 254 9 247 209 183 35 30 26 172 161 155 240 23 71 146 129 43 127 174 8 48 192 215 26 232 37 234 29 211 171 130 173 99 225 128 180 19 168 201 155 193 235 126 91 130 83 4 9 213 51 142 209 214 217 115 10 157 40 195 28 101 72 206 254 105 244 121 231 162 107 32 39 222 172 71 255 151 171 166 209 142 79 252 139 24 103 233 189 234 33 192 249 71 0 127 122 117 219 203 164 151 230 76 244 52 10 156 228 194 140 165 7 13 49 148 135 12 156 14 122 96 182 104 133 42 243 139 156 204 167 200 89 180 49 234 44 116 6 122 208 232 178 135 188 183 96 86 52 230 186 120 175 107 14 243 172 61 133 110 69 199 237 233 24 200 223 153 5 223 23 52 95 241 184 202 24 152 163 66 157 175 211 176 177 79 215 140 201 19 21 39 43 121 87 6 53 186 136 243 6 197 197 90 125 163 111 74 178 128 134 173 228 49 108 103 206 123 228 208 147 2 204 61 90 108 222 154 116 81 171 236 72 121 65] Target:[] Anchor:[74 119 97 67 82 67 115 47 98 50 54 47 111 66 108 85 79 109 86 82 88 75 76 69 75 89 68 97 43 115 83 68] Tags:[] Reader:{reader:0x14000554078}}
    decoder_test.go:68: &{SignatureType:ed25519 Signature:[244 82 210 223 68 216 26 188 8 111 9 153 28 70 183 131 189 75 189 247 245 199 249 145 255 136 19 79 49 40 112 147 115 204 159 90 29 188 193 172 3 236 129 90 181 190 60 108 247 152 52 26 50 131 125 222 112 65 211 192 209 160 52 6] Owner:[226 241 253 235 8 217 10 81 201 63 195 101 165 114 118 193 22 92 206 10 155 130 187 114 132 32 65 38 212 208 82 5] Target:[] Anchor:[113 50 104 117 56 72 110 78 115 113 119 112 110 68 110 106 76 74 80 103 104 48 76 102 108 47 49 122 71 111 86 78] Tags:[] Reader:{reader:0x14000554108}}
    --- PASS: TestDecoder/rmJZGbi9_UY2yvQPnnwaEGMoLqIVmD5-XJi_aH6FvR0 (14.46s)
PASS

Process finished with the exit code 0

The parser only consumes 0.5 MB of memory⚡️! Which in practice, significantly reduced the amount of resources needed when the RSS3 Network is indexing data from bundled transactions.

The source code for the parser implemented in this article can be found on GitHub: RSS3-Arweave-SDK. We plan to continue moving more features over to the SDK.

Happy building!