Message Deduplication and Relay Logic

Relevant source files

• bitchat/Services/BLE/BLEFragmentHandler.swift
• bitchat/Services/BLE/BLERecentPeripheralCache.swift
• bitchat/Services/BLE/BLEService.swift
• bitchat/Services/MessageDeduplicationService.swift
• bitchat/Services/NostrProcessedEventStore.swift
• bitchat/Services/TransportConfig.swift
• bitchat/Utils/MessageDeduplicator.swift
• bitchatTests/MessageDeduplicationServiceTests.swift
• bitchatTests/Services/BLEFragmentHandlerTests.swift
• bitchatTests/Services/BLERecentPeripheralCacheTests.swift

Purpose and Scope

This page documents the mechanisms that prevent duplicate message processing and control how packets propagate through the BitChat mesh network. The system combines high-performance LRU (Least Recently Used) caches for immediate deduplication, persistent event tracking for Nostr, and strict relay policies to manage the mesh topology. It also covers fragment assembly logic and the Gateway service for bridging mesh traffic to global Nostr relays.

Sources: bitchat/Services/MessageDeduplicationService.swift1-10 bitchat/Services/BLE/BLEService.swift60-62

---

Deduplication System Architecture

BitChat employs deduplication at two distinct layers: the Network Layer (Bluetooth/Mesh) to prevent relay loops and the Application Layer (UI/Nostr) to ensure a clean user experience across multiple transports.

Deduplication Components

Diagram: Message Deduplication Data Flow

Sources: bitchat/Utils/MessageDeduplicator.swift7-15 bitchat/Services/MessageDeduplicationService.swift160-186 bitchat/Services/BLE/BLEService.swift60-62

MessageDeduplicator (Network Layer)

The MessageDeduplicator provides a thread-safe cache used by the Bluetooth stack to prevent broadcast storms.

• LRU Eviction: When the cache exceeds maxCount (defined in TransportConfig), it trims entries down to 75% of capacity to optimize memory bitchat/Utils/MessageDeduplicator.swift74-92
• Time Expiry: Entries older than maxAge are purged during periodic cleanup() calls bitchat/Utils/MessageDeduplicator.swift105-127
• Announce Tracking: The markProcessed(_:) function allows adding IDs (like peer announcements) to the seen list without a check-and-add conditional bitchat/Utils/MessageDeduplicator.swift56-65

Sources: bitchat/Utils/MessageDeduplicator.swift7-53 bitchat/Utils/MessageDeduplicator.swift93-127 bitchat/Services/TransportConfig.swift23-24

MessageDeduplicationService (UI Layer)

The MessageDeduplicationService manages application-level filtering on the @MainActor bitchat/Services/MessageDeduplicationService.swift16-164

• Content Normalization: ContentNormalizer strips URL query strings, collapses whitespace, and lowercases text to detect "near-duplicates" before generating a djb2 hash bitchat/Services/MessageDeduplicationService.swift104-156
• Nostr Persistence: NostrProcessedEventStore provides disk persistence for processed gift-wrap event IDs. This is critical for NIP-59 DMs which have randomized timestamps, preventing "delivered" bursts on every app relaunch bitchat/Services/NostrProcessedEventStore.swift12-21
• Nostr ACK Cache: Tracks acknowledgments using a messageId:ackType:senderPubkey format bitchat/Services/MessageDeduplicationService.swift172-173

Sources: bitchat/Services/MessageDeduplicationService.swift104-186 bitchat/Services/NostrProcessedEventStore.swift22-55

---

Relay and Mesh Logic

TTL and Loop Prevention

Messages in the mesh are propagated using a flooding-based gossip protocol with specific constraints:

• TTL Decrementing: Every packet starts with a default TTL (typically 7). Relays decrement this value; when it reaches 0, the packet is no longer forwarded bitchat/Services/BLE/BLEService.swift30 bitchat/Services/TransportConfig.swift8
• Last-Hop Suppression: If a packet's TTL is already 0 when received, it is processed locally but never re-broadcast bitchat/Services/BLE/BLEFragmentHandler.swift70-71
• Adaptive TTL: In dense graphs (where the number of neighbors exceeds bleHighDegreeThreshold), the system may clamp TTL more aggressively to contain floods bitchat/Services/TransportConfig.swift10-17

Gateway Logic

The GatewayService bridges mesh traffic to global relays with loop prevention:

• Uplink: Mesh packets directed to a gateway are verified and published to Nostr relays.
• Downlink: Events from Nostr are rebroadcast to the mesh. To prevent loops, events learned from a gateway broadcast are never re-published to relays, and a gateway never downlinks its own previously uplinked events.

Sources: bitchat/Services/TransportConfig.swift8-17 bitchat/Services/BLE/BLEService.swift30-34

---

BLE Fragment Handling

Large payloads (e.g., media or long text) are fragmented to fit BLE MTU limits. The BLEFragmentHandler orchestrates reconstruction.

Diagram: Fragment Assembly Pipeline

Assembly Rules

• Self-Suppression: Fragments authored by the local peer are tracked for gossip sync (to prevent redundant requests) but skipped for assembly bitchat/Services/BLE/BLEFragmentHandler.swift44-49
• Reassembly: Fragments are appended to BLEFragmentAssemblyBuffer. Once complete, the original packet is decoded, its ttl is zeroed, and it re-enters the receive pipeline bitchat/Services/BLE/BLEFragmentHandler.swift59-72
• Resource Management: The system caps concurrent assemblies (bleMaxInFlightAssemblies) and evicts assemblies that exceed size limits to prevent memory exhaustion bitchat/Services/BLE/BLEFragmentHandler.swift94-100 bitchat/Services/TransportConfig.swift9

Sources: bitchat/Services/BLE/BLEFragmentHandler.swift27-76 bitchat/Services/TransportConfig.swift7-17 bitchatTests/Services/BLEFragmentHandlerTests.swift115-146

---

Summary Table: Relay and Sync Logic

Component	Responsibility	Implementation Detail
MessageDeduplicator	Network Dedup	Thread-safe LRU cache with 75% compaction threshold bitchat/Utils/MessageDeduplicator.swift78-80
LRUDeduplicationCache	Generic Dedup	O(1) lookup with periodic backing storage compaction bitchat/Services/MessageDeduplicationService.swift17-99
BLEFragmentHandler	Reassembly	Buffers fragments and re-injects reassembled packets with TTL=0 bitchat/Services/BLE/BLEFragmentHandler.swift27-33
ContentNormalizer	Near-Dup Detection	DJB2 hashing of collapsed, lowercase, and URL-stripped prefixes bitchat/Services/MessageDeduplicationService.swift153
NostrProcessedEventStore	Persistence	Atomic JSON storage with .completeFileProtectionUntilFirstUserAuthentication bitchat/Services/NostrProcessedEventStore.swift73-89
BLERecentPeripheralCache	Reconnection	Remembers recently seen peripherals for background wake-on-proximity bitchat/Services/BLE/BLERecentPeripheralCache.swift7-23

Sources: bitchat/Utils/MessageDeduplicator.swift7-128 bitchat/Services/MessageDeduplicationService.swift17-186 bitchat/Services/BLE/BLEFragmentHandler.swift27-102 bitchat/Services/NostrProcessedEventStore.swift12-106