Gossip Sync and Public History Reconciliation

Relevant source files

• bitchat/Models/RequestSyncPacket.swift
• bitchat/Services/BLE/BLEOutboundPacketPolicy.swift
• bitchat/Services/BLE/BLEReceivePipeline.swift
• bitchat/Services/BLE/BLERouteForwardingPolicy.swift
• bitchat/Sync/GCSFilter.swift
• bitchat/Sync/GossipSyncManager.swift
• bitchat/Sync/RequestSyncManager.swift
• bitchat/Sync/SyncResponseRateLimiter.swift
• bitchat/Sync/SyncTypeFlags.swift
• bitchatTests/GCSFilterTests.swift
• bitchatTests/GossipSyncManagerTests.swift
• bitchatTests/Services/BLERouteForwardingPolicyTests.swift
• bitchatTests/Sync/GossipSyncBoardTests.swift
• bitchatTests/Sync/RequestSyncManagerTests.swift
• bitchatTests/Sync/SyncTypeFlagsBoardTests.swift
• docs/REQUEST_SYNC_MANAGER.md
• localPackages/BitFoundation/Sources/BitFoundation/MessageType.swift

BitChat employs a gossip-based anti-entropy protocol to ensure that peers in a mesh network eventually reach consistency regarding public messages, announcements, and shared metadata. This system is designed to handle the transient connectivity inherent in Bluetooth mesh networking, allowing devices to backfill missed history when they come into range of a peer or reconnect to the network.

1. GossipSyncManager Overview

The GossipSyncManager is the central orchestrator for set reconciliation. It maintains a local buffer of recent public packets and periodically initiates synchronization rounds with connected peers using Golomb-Coded Set (GCS) filters <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L6-L12" min=6 max=12 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>.

Core Responsibilities

• Packet Retention: Buffers public messages, fragments, file transfers, and prekey bundles within specific age and capacity limits <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L63-L92" min=63 max=92 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>.
• Periodic Reconciliation: Triggers sync requests at varying intervals (e.g., 15 seconds for messages) <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L79-L80" min=79 max=80 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>.
• Set Reconciliation: Generates GCS filters representing the local state to identify missing data from peers <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GCSFilter.swift#L41-L42" min=41 max=42 file-path="bitchat/Sync/GCSFilter.swift">Hii</FileRef>.
• Rate Limiting: Prevents peers from flooding the mesh with sync responses via SyncResponseRateLimiter <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L130-L133" min=130 max=133 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>.

System Architecture

The following diagram illustrates the relationship between the sync manager and the protocol entities.

Gossip Sync Entity Relationship

Sources: <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L6-L133" min=6 max=133 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>, <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Models/RequestSyncPacket.swift#L13-L24" min=13 max=24 file-path="bitchat/Models/RequestSyncPacket.swift">Hii</FileRef>, <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/SyncTypeFlags.swift#L6-L66" min=6 max=66 file-path="bitchat/Sync/SyncTypeFlags.swift">Hii</FileRef>.

2. The Reconciliation Protocol

BitChat uses a "solicited pull" model for anti-entropy. Instead of blindly broadcasting history, a peer sends a REQUEST_SYNC packet containing a compact representation of what it already knows.

Sync Types and Intervals

Sync rounds are divided by SyncTypeFlags to prevent large binary transfers (like files) from starving small, urgent updates (like public messages) <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/SyncTypeFlags.swift#L89-L97" min=89 max=97 file-path="bitchat/Sync/SyncTypeFlags.swift">Hii</FileRef>. The flags are encoded as a variable-length (1-8 byte) little-endian bitfield <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/SyncTypeFlags.swift#L142-L155" min=142 max=155 file-path="bitchat/Sync/SyncTypeFlags.swift">Hii</FileRef>.

Sync Type	Interval	Capacity	Purpose
publicMessages	15s	1000	Announcements and public chat <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L79-L80" min=79 max=80 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>
fragment	30s	600	Partial large messages/media <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L77-L78" min=77 max=78 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>
fileTransfer	60s	200	Metadata for binary files <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L78-L79" min=78 max=79 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>
board	60s	200	Geohash bulletin board posts <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L83-L84" min=83 max=84 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>
prekeyBundle	60s	200	One-time prekeys for offline DMs <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L89-L90" min=89 max=90 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>

Data Flow: Sync Request and Response

When a sync round triggers, the GossipSyncManager constructs a RequestSyncPacket for a specific peer <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L234-L245" min=234 max=245 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>.

Sync Reconciliation Flow

Sources: <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L234-L280" min=234 max=280 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>, <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/docs/REQUEST_SYNC_MANAGER.md?plain=1#L5-L13" min=5 max=13 file-path="docs/REQUEST_SYNC_MANAGER.md">Hii</FileRef>, <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Models/RequestSyncPacket.swift#L87-L137" min=87 max=137 file-path="bitchat/Models/RequestSyncPacket.swift">Hii</FileRef>.

3. Golomb-Coded Set (GCS) Filters

To minimize bandwidth, BitChat uses GCS filters rather than Bloom filters for set reconciliation. GCS provides a more compact representation for a fixed false-positive rate (FPR) <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GCSFilter.swift#L4-L12" min=4 max=12 file-path="bitchat/Sync/GCSFilter.swift">Hii</FileRef>.

Implementation Details

• Hashing: Packet IDs (16 bytes) are hashed using the first 8 bytes of SHA-256 <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GCSFilter.swift#L116-L125" min=116 max=125 file-path="bitchat/Sync/GCSFilter.swift">Hii</FileRef>.
• Encoding: Values are mapped to a range <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/1, M), sorted, and the deltas are encoded using Golomb-Rice coding with parameter P [bitchat/Sync/GCSFilter.swift#L8-L11" min=8 max=11 file-path="1, M), sorted, and the deltas are encoded using Golomb-Rice coding with parameter P [bitchat/Sync/GCSFilter.swift">Hii</FileRef>.
• Budgeting: The buildFilter function automatically trims the tail of the ID list to fit within a gcsMaxBytes budget (default 400 bytes) <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GCSFilter.swift#L68-L71" min=68 max=71 file-path="bitchat/Sync/GCSFilter.swift">Hii</FileRef>.
• Cursor Sync: The sinceTimestamp TLV in the RequestSyncPacket allows the requester to specify a starting point, preventing the responder from scanning the entire history <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Models/RequestSyncPacket.swift#L9-L10" min=9 max=10 file-path="bitchat/Models/RequestSyncPacket.swift">Hii</FileRef>.

Sources: <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GCSFilter.swift#L41-L78" min=41 max=78 file-path="bitchat/Sync/GCSFilter.swift">Hii</FileRef>, <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Models/RequestSyncPacket.swift#L5-L12" min=5 max=12 file-path="bitchat/Models/RequestSyncPacket.swift">Hii</FileRef>.

4. Security and Attribution

The RequestSyncManager ensures that sync responses (RSR) are only accepted if they were explicitly requested.

The RSR Flag

Packets sent as part of a sync response are marked with the IS_RSR flag (0x10) in the binary header <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/docs/REQUEST_SYNC_MANAGER.md?plain=1#L17-L20" min=17 max=20 file-path="docs/REQUEST_SYNC_MANAGER.md">Hii</FileRef>. These packets are also forced to ttl = 0 to ensure they do not propagate beyond the immediate link <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/docs/REQUEST_SYNC_MANAGER.md?plain=1#L40-L41" min=40 max=41 file-path="docs/REQUEST_SYNC_MANAGER.md">Hii</FileRef>.

Timestamp Validation Exemption

Normal BitChat traffic enforces a strict 2-minute timestamp window to prevent replay attacks. However, sync responses often carry historical data.

• If packet.isRSR is true, the BLEService queries RequestSyncManager.isValidResponse(from:senderID, isRSR: true) <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/docs/REQUEST_SYNC_MANAGER.md?plain=1#L42-L47" min=42 max=47 file-path="docs/REQUEST_SYNC_MANAGER.md">Hii</FileRef>.
• If a matching pending request exists (sent within the last 30 seconds), the timestamp check is bypassed <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/docs/REQUEST_SYNC_MANAGER.md?plain=1#L31-L35" min=31 max=35 file-path="docs/REQUEST_SYNC_MANAGER.md">Hii</FileRef>.

Sources: <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/docs/REQUEST_SYNC_MANAGER.md?plain=1#L5-L13" min=5 max=13 file-path="docs/REQUEST_SYNC_MANAGER.md">Hii</FileRef>, <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Services/BLE/BLEReceivePipeline.swift#L23-L24" min=23 max=24 file-path="bitchat/Services/BLE/BLEReceivePipeline.swift">Hii</FileRef>.

5. Specialized Reconciliation

Fragment Sync

When a large message reassembly stalls (no new fragments for 5 seconds), the system can perform a targeted resync. The RequestSyncPacket includes a fragmentIdFilter (TLV 0x06), which contains a comma-separated list of 8-byte fragment stream IDs <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Models/RequestSyncPacket.swift#L10-L12" min=10 max=12 file-path="bitchat/Models/RequestSyncPacket.swift">Hii</FileRef>. The responder then restricts its diff to only those specific streams <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/docs/REQUEST_SYNC_MANAGER.md?plain=1#L26-L27" min=26 max=27 file-path="docs/REQUEST_SYNC_MANAGER.md">Hii</FileRef>.

Board Provider

Unlike messages which are stored in an internal PacketStore, boardPost packets are served via a boardPacketsProvider closure <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L104-L105" min=104 max=105 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>. This allows the BoardStore to maintain its own logic for expiry, tombstones, and geohash-based retention while still participating in the global gossip sync <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchatTests/Sync/GossipSyncBoardTests.swift#L14-L16" min=14 max=16 file-path="bitchatTests/Sync/GossipSyncBoardTests.swift">Hii</FileRef>.

Prekey Bundle Sync

Prekey bundles (MessageType 0x24) are gossiped similarly to board posts but have a much longer maxAgeSeconds (24 hours) <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L91-L92" min=91 max=92 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>. This ensures that one-time prekeys for offline recipients remain available throughout the mesh even if the owner is not currently connected.

Sources: <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Models/RequestSyncPacket.swift#L28-L47" min=28 max=47 file-path="bitchat/Models/RequestSyncPacket.swift">Hii</FileRef>, <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/GossipSyncManager.swift#L100-L104" min=100 max=104 file-path="bitchat/Sync/GossipSyncManager.swift">Hii</FileRef>, <FileRef file-url="https://github.com/permissionlesstech/bitchat/blob/733098bb/bitchat/Sync/SyncTypeFlags.swift#L64-L65" min=64 max=65 file-path="bitchat/Sync/SyncTypeFlags.swift">Hii</FileRef>.