~ knotless Search/
↑↓ move↵ open⇧↵ all results
0
0
148 lines · 6.4 KB Raw History

STAR — STreaming ARchives

This document includes material copied from the DASL Project, Copyright © 2026 IPFS Foundation, https://tangled.org/dasl.ing/dasl.ing/blob/main/LICENSE.md (the full notice is in LICENSE-DASL.md beside this file). It is https://dasl.ing/star.html as last updated 2026-11-03, converted from HTML to Markdown: the text is unchanged, the site's navigation is left out, and the links are kept as written.

last updated2026-11-03
statusHot New Stuff
editorsFig Schleihauf fig@microcosm.blue
abstractSTAR serializes key-value data pairs into a flat stream, with header metadata. Application profiles can define additional data validations and restrictions.

Introduction

The STAR format (STreaming ARchive) stores sequences of key-value data pairs. The format is simple and flexible: a single required $type key in the header metadata lets applications associate additional data restrictions and validations for their own use-cases.

Compared to CAR ([car]): STAR is not content addressed and has no format-level CIDs ([cid]), contains data pairs instead of individual blocks, and makes the order of pairs explicitly significant. STAR can losslessly encode any CAR with minimal per-block overhead, see Application Profiles below for an example.

Format

A STAR file begins with four magic bytes, followed by a length-prefixed metadata map with a $type key, and then zero or more data pairs. Each pair starts with a literal 0x01 byte, followed by two length-prefixed byte sequences, "key" and "value". A final 0x00 byte ends the archive.

|------ header ------| |--------- data pairs ---------| | end |
[ magic | len | meta ] [ 1 | len | key | len | val ] *  [  0  ]

Magic

The magic bytes are 0x2A 0x6C 0x01 0x00: ascii *l ("star lite"), a fixed version 1, and a null byte to make four. STAR's magic bytes are always distinct from the first bytes of a CAR ([car]) file.

Length-prefixing

Length prefixes in STAR are encoded as unsigned variable-length integers ([varint]). The integer specifies the number of following bytes to read, not including the integer itself.

Metadata

The header metadata must be a DRISL-encoded ([drisl]) map with the exact size of its length prefix. The map's only reserved and required key is "$type", whose value must be a text string. The meaning of $type is described below under Application Profiles. Arbitrary DRISL metadata can be included at other keys in the map.

Data pairs & terminal byte

A control byte must follow the header metadata and each data pair, with one of two values:

  • 0x01: a data pair must follow
  • 0x00: marks the end of the archive

Each data pair following 0x01 is a length-prefixed key of bytes followed by a length-prefixed value of bytes. Exactly one key and one value must be present in each data pair. Duplicate keys and empty keys and values are permitted.

The terminating 0x00 byte must not be followed by any trailing padding or other bytes.

Additional considerations

Any parser error must be treated as fatal for the entire archive.

There are no data-pair-level integrity checks in STAR: See Application Profiles below for common approaches to assuring archive integrity.

Data pair ordering within an archive is significant: serializers and parsers must emit data pairs in the order they are provided/encoded.

STAR encoding is deterministic where archive contents and data ordering are deterministic.

Length prefixes must comply with the [varint] restrictions: encoded varints must not be longer than 9 bytes (63 integer bits), and must be minimally encoded: the only number that can end in 0x00 is 0. Note that many common LEB128-style varint parsers are not compliant with these restrictions. Non-strict varint parsing can lead to security issues where deterministic archive encoding is depended upon.

Header metadata must comply with DRISL's type restrictions and deterministic encoding requirements. Note that generic CBOR parsers may not be compliant with these restrictions.

Like CAR ([car]), no format-level size limits are defined for header metadata or data-pair keys and values. However, parsers may impose reasonable maximums and are encouraged to let applications configure stricter limits.

Application profiles

TODO

Not stable yet!

STAR is a new format, and while it has applications headed for production soon, be prepared for breaking spec changes while its DASL maturity remains Hot New Stuff. Additional background, resources, and open questions:

Appendix: Media Type

The media type for STAR is application/vnd.dasl.star.

The conventional file extension for STAR is .star.

References