Crdt.Model_codec.Binary

crdt · API reference

Binary structural encoding using CBOR-like format.

Format:

  • Header: u32 (offset to clock table)

  • Content: nodes encoded recursively

  • Clock table: vu57(count), then (vu57(sid), vu57(time)) pairs Node encoding (after id):

  • Major 0 (0x00-0x1F): con node

  • Major 1 (0x20-0x3F): val node

  • Major 2 (0x40-0x5F): obj node (minor = key count)

  • Major 3 (0x60-0x7F): vec node (minor = slot count)

  • Major 4 (0x80-0x9F): str node (minor = chunk count)

  • Major 5 (0xA0-0xBF): bin node (minor = chunk count)

  • Major 6 (0xC0-0xDF): arr node (minor = chunk count) ID encoding: b1vu56(session_index) + vu57(time_diff)

  • Compact form when session_index <= 7 and time_diff <= 15: single byte

Encoder - using Cbor module

type encoder = Model_codec_cbor.encoder
val create_encoder : ?capacity:int -> unit -> Model_codec_cbor.encoder
val encoder_contents : Model_codec_cbor.encoder -> bytes
val write_u8 : Model_codec_cbor.encoder -> int -> unit
val write_u32 : Model_codec_cbor.encoder -> int -> unit
val set_u32 : Model_codec_cbor.encoder -> int -> int -> unit
val write_bytes : Model_codec_cbor.encoder -> bytes -> unit
val write_vu57 : Model_codec_cbor.encoder -> int -> unit
val write_b1vu56 : Model_codec_cbor.encoder -> bool -> int -> unit
val write_id : Model_codec_cbor.encoder -> int -> int -> unit
val write_cbor_uint : Model_codec_cbor.encoder -> int -> unit
val write_cbor_string : Model_codec_cbor.encoder -> string -> unit
val write_cbor_value : Model_codec_cbor.encoder -> Value.t -> unit

Clock Table Management

json-joy uses delta encoding for timestamps:

  • The clock table stores (sid, max_time) for each session
  • When encoding an ID, we store (session_index, time_diff) where time_diff = max_time - actual_time
  • This means most time_diffs are small (0, 1, 2...) since we usually reference recent timestamps, allowing 1-byte encoding The clock encoder is initialized from the model's clock vector which already knows max times for all sessions.
type clock_encoder = {
  mutable sids : int list;
  sid_to_idx : (int, int) Hashtbl.t;
  times : (int, int) Hashtbl.t;
}
val create_clock_encoder_from_model : Model.t -> clock_encoder

Initialize clock encoder from model's clock vector. This sets up the clock table with known max times BEFORE encoding.

val clock_append : clock_encoder -> Clock.timestamp -> int * int

Append a timestamp and return (session_index, time_diff). time_diff = max_time - actual_time

Node Encoding

val write_tl : Model_codec_cbor.encoder -> int -> int -> unit
val encode_node : 
  Model_codec_cbor.encoder ->
  clock_encoder ->
  Model.t ->
  Node.t ->
  unit
val encode : Model.t -> bytes

Encode a complete model to binary

val encode_bytes : Model.t -> bytes

Encode to bytes and return as string for convenience

Decoder - using Cbor module

type decoder = Model_codec_cbor.decoder
val create_decoder : bytes -> Model_codec_cbor.decoder
val read_u8 : Model_codec_cbor.decoder -> int
val peek_u8 : Model_codec_cbor.decoder -> int
val read_u32 : Model_codec_cbor.decoder -> int
val read_bytes : Model_codec_cbor.decoder -> int -> bytes
val read_vu57 : Model_codec_cbor.decoder -> int
val read_b1vu56 : Model_codec_cbor.decoder -> bool * int
val read_id : Model_codec_cbor.decoder -> int * int
val read_cbor_length : Model_codec_cbor.decoder -> int -> int
val read_cbor_value : Model_codec_cbor.decoder -> Value.t
val read_key : Model_codec_cbor.decoder -> string
val decoder_pos : Model_codec_cbor.decoder -> int
val set_decoder_pos : Model_codec_cbor.decoder -> int -> unit

Clock Table Decoding

type clock_decoder = {
  sids : int array;
  times : int array;
}
val decode_clock_table : Model_codec_cbor.decoder -> int -> clock_decoder
val decode_ts : clock_decoder -> (int * int) -> Clock.timestamp

Decode a timestamp from (session_index, time_diff). json-joy uses delta encoding: actual_time = max_time - time_diff Note: session indices are 1-based (0 is reserved for system session)

Node Decoding

val decode_node : 
  Model_codec_cbor.decoder ->
  clock_decoder ->
  Model.t ->
  Clock.timestamp option
val decode : bytes -> Model.t option

Decode a complete model from binary

val decode_bytes : bytes -> Model.t option

Decode from bytes