chadtmiller.com/monolith
An atproto PDS
STAR — STreaming ARchives
This document includes material copied from the DASL Project, Copyright ©
2026 IPFS Foundation, https://tangled.org/dasl.ing/dasl.ing/blob/main/LICENSE.md
(the full notice is in LICENSE-DASL.md beside this file). It is
https://dasl.ing/star.html as last updated 2026-11-03, converted from HTML to
Markdown: the text is unchanged, the site's navigation is left out, and the
links are kept as written.
| last updated | 2026-11-03 |
| status | Hot New Stuff |
| editors | Fig Schleihauf fig@microcosm.blue |
| abstract | STAR serializes key-value data pairs into a flat stream, with header metadata. Application profiles can define additional data validations and restrictions. |
Introduction
The STAR format (STreaming ARchive) stores sequences of key-value data
pairs. The format is simple and flexible: a single required $type key in
the header metadata lets applications associate additional data
restrictions and validations for their own use-cases.
Compared to CAR ([car]): STAR is not content addressed and has no format-level CIDs ([cid]), contains data pairs instead of individual blocks, and makes the order of pairs explicitly significant. STAR can losslessly encode any CAR with minimal per-block overhead, see Application Profiles below for an example.
Format
A STAR file begins with four magic bytes, followed by a length-prefixed
metadata map with a $type key, and then zero or more data pairs. Each pair
starts with a literal 0x01 byte, followed by two length-prefixed byte
sequences, "key" and "value". A final 0x00 byte ends the archive.
|------ header ------| |--------- data pairs ---------| | end |
[ magic | len | meta ] [ 1 | len | key | len | val ] * [ 0 ]
Magic
The magic bytes are 0x2A 0x6C 0x01 0x00: ascii *l ("star lite"), a
fixed version 1, and a null byte to make four. STAR's magic bytes are
always distinct from the first bytes of a CAR ([car]) file.
Length-prefixing
Length prefixes in STAR are encoded as unsigned variable-length integers ([varint]). The integer specifies the number of following bytes to read, not including the integer itself.
Metadata
The header metadata must be a DRISL-encoded ([drisl]) map with the exact
size of its length prefix. The map's only reserved and required key is
"$type", whose value must be a text string. The meaning of $type is
described below under Application Profiles. Arbitrary DRISL metadata can be
included at other keys in the map.
Data pairs & terminal byte
A control byte must follow the header metadata and each data pair, with one of two values:
0x01: a data pair must follow0x00: marks the end of the archive
Each data pair following 0x01 is a length-prefixed key of bytes followed
by a length-prefixed value of bytes. Exactly one key and one value must be
present in each data pair. Duplicate keys and empty keys and values are
permitted.
The terminating 0x00 byte must not be followed by any trailing padding or
other bytes.
Additional considerations
Any parser error must be treated as fatal for the entire archive.
There are no data-pair-level integrity checks in STAR: See Application Profiles below for common approaches to assuring archive integrity.
Data pair ordering within an archive is significant: serializers and parsers must emit data pairs in the order they are provided/encoded.
STAR encoding is deterministic where archive contents and data ordering are deterministic.
Length prefixes must comply with the [varint] restrictions: encoded varints
must not be longer than 9 bytes (63 integer bits), and must be minimally
encoded: the only number that can end in 0x00 is 0. Note that many
common LEB128-style varint parsers are not compliant with these
restrictions. Non-strict varint parsing can lead to security
issues
where deterministic archive encoding is depended upon.
Header metadata must comply with DRISL's type restrictions and deterministic encoding requirements. Note that generic CBOR parsers may not be compliant with these restrictions.
Like CAR ([car]), no format-level size limits are defined for header metadata or data-pair keys and values. However, parsers may impose reasonable maximums and are encouraged to let applications configure stricter limits.
Application profiles
TODO
Not stable yet!
STAR is a new format, and while it has applications headed for production soon, be prepared for breaking spec changes while its DASL maturity remains Hot New Stuff. Additional background, resources, and open questions:
- Interop test vectors for implementors.
- STAR-lite, an earlier version of this format specifically for atproto public data repository exports. It is not part of the atproto specs but is actively used on the network, to be superseded by STAR.
- Initial public STAR feedback of the first attempt to adapt STAR-lite for atproto spaces, leading to its current flexible shape.
- Proposal for STAR, tangled issue introducing STAR to DASL.
- Presentation slides introducing STAR at the October 2026 CID Congress.
- Settling format details, a summary of format decisions to settle.
- cobalt/star, a golang implementation of STAR
Appendix: Media Type
The media type for STAR is application/vnd.dasl.star.
The conventional file extension for STAR is .star.
References
- [car] Robin Berjon & Juan Caballero. CAR — Content-Addressable aRchives. 2026-11-03.
- [cid] Robin Berjon & Juan Caballero. CID — Content Identifiers. 2026-11-03.
- [drisl] Robin Berjon & Juan Caballero. DRISL — Deterministic Representation for Interoperable Structures & Links. 2026-11-03.
- [varint] unsigned varint.