DDIA Ch 4 — Notes & Questions

Chapter 4: Encoding and Evolution — Study Notes

Overview

Whenever you send data to another process you don’t share memory with (over the
network, or to a file/disk), you must turn in-memory data structures (objects, lists, arrays,
hash tables, trees) into a self-contained sequence of bytes. This chapter is about the
formats for doing that and, crucially, how those formats let your application evolve over time.

Core terms

Encoding / serialization / marshalling — converting in-memory representation to a

byte sequence.
Decoding / parsing / deserialization / unmarshalling — the reverse.
Evolvability — the ability to make changes to an application easily, upgrading parts
independently rather than changing everything at once.
Rolling upgrade — deploying a new version to a few nodes at a time (no downtime,
less risky). Implies old and new code run simultaneously.

Two directions of compatibility (memorize these)

Backward compatibility — newer code can read data written by older code. (Usually
easy: you know the old format.)
Forward compatibility — older code can read data written by newer code. (Trickier:
old code must ignore additions made by future code.)

Compatibility is a relationship between a process that encodes the data and a process that
decodes it.

Formats for Encoding Data

Language-specific encodings (e.g., Java Serializable, Ruby Marshal, Python

pickle)
 Convenient but come with serious problems:
Tied to one programming language — hard to read from other languages.
Security risk — decoding must instantiate arbitrary classes (remote code
execution vector).
Versioning is an afterthought — often neglect forward/backward compatibility.
Poor efficiency (e.g., Java’s built-in serialization is notoriously slow and
bloated).
Rule of thumb: avoid language-specific formats for anything beyond transient, in-
process use.

Textual formats: JSON, XML, CSV

Widespread, human-readable, language-independent. But have shortcomings:

Ambiguity around numbers: XML/CSV can’t distinguish a number from a string
of digits; JSON distinguishes strings/numbers but not integers vs floats and gives
no precision. Large numbers (> 2^53) lose precision in JavaScript-style parsers
— Twitter sends tweet IDs as both a number and a decimal string to work around
this.
No native support for binary strings (must Base64-encode, increasing size
~33%).
Optional schema support (XML Schema, JSON Schema) — powerful but
complex and not universally used.
CSV has no schema and weak/ad-hoc escaping rules.
Good enough as an interchange format where everyone agrees informally; the
vagueness is the price of human-readability.

Binary encodings

For internal, large-volume data, compactness and speed matter enough to justify
binary.
“Binary JSON” variants (MessagePack, BSON, etc.) keep field names in the data, so
savings are modest. The bigger win comes from schema-driven binary formats that
omit field names.

Schema-Driven Binary Formats

All three below use a schema to describe the structure, enabling compact encoding (no field
names in the data) and clear compatibility rules.
Thrift and Protocol Buffers

Both require a schema (IDL) and use code generation to produce classes.
The encoded record is just a concatenation of encoded fields. Each field carries a
field tag (a number from the schema) and a datatype annotation — not the field
name.
Field tags act as compact aliases for fields. They are critical to meaning.
Thrift CompactProtocol packs field type + tag into one byte and uses variable-
length integers (top bit signals “more bytes follow”). Numbers −64..63 fit in 1 byte;
−8192..8191 in 2 bytes; etc. Example record: 34 bytes.
Protocol Buffers does bit-packing slightly differently; same example record: 33 bytes.
Protobuf has only one binary format.
required vs optional markers make no difference to the encoding — required just

enables a runtime presence check (useful for catching bugs).

Schema evolution rules (Thrift / Protobuf)

You can rename a field freely (encoded data refers to tags, not names).
You cannot change a field’s tag — it would invalidate all existing encoded data.
Adding a field: give it a new tag number.
Forward compatibility: old code sees an unknown tag and skips it (datatype
annotation tells it how many bytes to skip).
Backward compatibility: a newly added field must be optional or have a default
value — it cannot be required (old data wouldn’t contain it, so the presence
check would fail).
Removing a field: mirror of adding. You can only remove an optional field, and you
can never reuse its tag number (old data may still reference it).

Datatype changes

Possible but risky — values may lose precision or be truncated. E.g., widening int32
→ int64: new code reading old data fills with zeros (fine), but old code reading new 64-
bit data truncates to 32 bits if the value doesn’t fit.
Protobuf has no list/array type — instead a repeated marker (a third option besides
required/optional). A repeated field is just the same tag appearing multiple times.
Nice consequence: you can change optional (single-valued) → repeated (multi-
valued). New code reading old data sees a 0/1-element list; old code reading
new data sees only the last element.
 Thrift has a dedicated list type (parameterized by element type). Doesn’t allow the
single→multi evolution, but supports nested lists.

Avro (Apache Avro)

Started 2009 as a Hadoop subproject (Thrift didn’t fit Hadoop’s needs).

Two schema languages: Avro IDL (human-editable) and a JSON-based one
(machine-readable).
No field tags and no datatype annotations in the encoded data. Encoding is just
values concatenated. The most compact format seen (example record: 32 bytes).
To parse, you must walk the fields in schema order and use the schema to know
each field’s type. Therefore decoding requires the reader to use a schema compatible
with the writer’s.

Writer’s schema vs reader’s schema (the key idea)

Writer’s schema — the schema version the data was encoded with.
Reader’s schema — the schema the reading code expects.
They need not be identical, only compatible. The Avro library performs schema
resolution, matching fields by name and translating writer’s data into the reader’s
schema.
Fields in a different order: fine (matched by name).
Field in writer’s schema but not reader’s: ignored.
Field expected by reader but absent in writer’s: filled with the reader’s declared
default value.

Avro schema evolution rules

Forward compatibility = new schema as writer, old schema as reader.

Backward compatibility = new schema as reader, old schema as writer.
You may only add or remove a field that has a default value.
Adding a field without a default breaks backward compatibility (new readers
can’t read old data).
Removing a field without a default breaks forward compatibility (old readers
can’t read new data).
null is not a default for everything — you must use a union type (e.g., union {
null, long, string } ). null can only be a default if it’s a branch of the union (and Avro

requires the default to match the first branch). This explicit nullability helps prevent
bugs.
Avro has no required/optional markers — it uses union types + defaults instead.
 Changing a field’s datatype is allowed if Avro can convert it.
Changing a field’s name: use aliases in the reader’s schema → backward
compatible but not forward compatible. (Adding a branch to a union is likewise
backward- but not forward-compatible.)

How does the reader know the writer’s schema?

Depends on context: - Large file, many records (Hadoop): include the writer’s schema
once at the start — Avro object container files do this. - Database, individually written
records: store a version number in each record + keep a list of schema versions in a
registry. (LinkedIn’s Espresso works this way.) - Network connection: processes
negotiate the schema version on connection setup. (Avro RPC works this way.) - A schema
registry is valuable anyway — acts as documentation and lets you check compatibility
before deploying.

Dynamically generated schemas

Avro’s killer advantage: no tag numbers, so schemas can be generated

automatically (e.g., from a relational DB dump — each table → record, each column
→ field by name).
When the DB schema changes, just regenerate the Avro schema; readers match fields
by name, so old readers still work.
With Thrift/Protobuf you’d have to manually assign tag numbers and carefully avoid
reusing old ones — dynamic schema generation wasn’t a design goal.

Code generation & dynamic languages

Thrift/Protobuf rely on code generation — great for statically typed languages

(efficient structures, type checking, IDE autocomplete).
In dynamically typed languages (JS, Ruby, Python) code generation adds little value
and is often unwanted.
Avro offers optional code generation but works fine without it — object container files
are self-describing (embed the writer’s schema), so you can read them like a JSON
file (great for tools like Apache Pig).

The Merits of Schemas

Thrift/Protobuf/Avro schema languages are simpler than XML Schema / JSON

Schema (which support richer validation like regex/range checks).
 Ideas aren’t new — similar to ASN.1 (standardized 1984, still used for DER-encoded
X.509 SSL certs); but ASN.1 is complex and badly documented — not recommended
for new apps.
Many databases use proprietary binary protocols + drivers (ODBC/JDBC).
Benefits of schema-based binary encodings:
More compact than binary-JSON variants (field names omitted).
Schema is valuable, always-up-to-date documentation (it’s required for
decoding, so it can’t drift).
A schema registry lets you check forward/backward compatibility before
deploying.
Code generation enables compile-time type checking for statically typed
languages.
Net: schema evolution gives the flexibility of schema-on-read JSON databases plus
better data guarantees and tooling.

Modes of Dataflow

Three common ways encoded data flows between processes.

1. Dataflow Through Databases

Writer encodes; reader decodes. The reader may simply be a later version of the
same process (“sending a message to your future self”) — so backward
compatibility is essential.
Multiple processes (different services, or many instances during a rolling upgrade)
commonly hit the DB at once → some run new code, some old → forward
compatibility is also needed.
Preserving unknown fields: if old code reads a record written by new code (with a
field it doesn’t understand), updates it, and writes it back, the desired behavior is to
keep the unknown field intact. The encoding formats support this, but watch out:
decoding into application model objects and re-encoding can silently drop unknown
fields. Be aware of it (Figure 4-7).
Data outlives code: a deployed app version replaces the old one in minutes, but DB
data written years ago is still there in its original encoding unless explicitly rewritten.
Migrating/rewriting large datasets is expensive, so most DBs avoid it — e.g., adding a
column with a null default doesn’t rewrite existing rows; the DB fills in null on read.
(Exception: MySQL often rewrites the whole table unnecessarily.)
 Schema evolution makes the whole DB appear as if encoded with one schema,
even though storage holds many historical versions.
Archival storage: snapshots/backups/warehouse loads are written once, immutably,
typically in the latest schema — a good fit for Avro object container files, and an
opportunity to use a column-oriented analytics format like Parquet.

2. Dataflow Through Services: REST and RPC

Clients call servers that expose an API (a service). A server can itself be a client to
other services → service-oriented architecture (SOA), refined/rebranded as
microservices.
Unlike databases (arbitrary queries), services expose an application-specific API
restricted by business logic → encapsulation.
Design goal: services independently deployable and evolvable, each owned by one
team releasing frequently → old & new versions coexist → encoding must be
compatible across versions.

Web services (HTTP-based)

Used in 3 contexts: client app → service over public internet; service → service within
an org (sometimes called middleware); service → another org’s service (public APIs,
e.g., credit-card processing, OAuth).
REST — not a protocol, a design philosophy built on HTTP: simple formats, URLs
identify resources, uses HTTP for cache/auth/content negotiation. APIs following it are
RESTful. Described by OpenAPI / Swagger. Dominant for public APIs.
SOAP — XML-based protocol, aims to be HTTP-independent; huge WS-* standards
family; API described in WSDL (not human-readable, relies heavily on code
generation/IDEs). Interoperability is painful; fallen out of favor outside large
enterprises. (Note: SOAP ≠ SOA.)

Problems with RPC (remote procedure calls)

RPC (since the 1970s) tries to make a network call look like a local function call —
called location transparency. This is fundamentally flawed because a network
request differs from a local call:
Unpredictable — requests/responses can be lost; remote machine may be
slow/down. Must anticipate (e.g., retries).
Extra outcome — timeout — you may get no result and not know whether the
request executed.
 Retries risk duplicates — if only the response was lost, retrying performs the
action twice unless you build idempotence/deduplication.
Variable latency — network calls are much slower and wildly variable vs. local
calls.
Parameters must be encoded to bytes — fine for primitives, awkward for large
objects (can’t pass pointers).
Cross-language type translation — RPC frameworks must map types across
languages (ugly; e.g., JS numbers > 2^53).
Older RPC tech was limited/complex: EJB/RMI (Java only), DCOM (Microsoft only),
CORBA (complex, no fwd/bwd compatibility).

Current directions for RPC

RPC isn’t going away; new frameworks are explicit that remote ≠ local:
gRPC (Protocol Buffers), Thrift & Avro RPC, Finagle (Thrift), [Link] (JSON
over HTTP).
Use futures/promises for async failures; gRPC supports streams (many
requests/responses).
Some provide service discovery (find a service’s IP/port).
Trade-off: custom binary RPC can be faster, but REST wins for
experimentation/debugging (curl, browser), broad language/tooling support (caches,
load balancers, proxies, etc.). REST predominates for public APIs; RPC is mostly for
intra-organization, same-datacenter calls.

Encoding & evolution for RPC

Simplifying assumption: servers are upgraded first, clients second. So you need
backward compatibility on requests and forward compatibility on responses.
Compatibility properties are inherited from the underlying encoding
(Thrift/gRPC/Avro per their rules; SOAP via XML schemas with subtle pitfalls;
REST/JSON: adding optional request params and new response fields is usually
compatible).
Across organizational boundaries the provider can’t force clients to upgrade →
compatibility must hold for a long time, maybe indefinitely; breaking changes often
mean running multiple API versions side by side.
API versioning has no standard: version in the URL, in an HTTP Accept header, or
per-client version stored server-side.

3. Message-Passing Dataflow
 Asynchronous message-passing sits between RPC and databases: low-latency
delivery (like RPC) but via an intermediary that stores the message (like a DB).
A message broker (a.k.a. message queue / message-oriented middleware) holds the
message temporarily.
Advantages over direct RPC:
Buffers when recipient is unavailable/overloaded → improves reliability.
Can redelivery to crashed consumers → prevents message loss.
Sender needn’t know recipient’s IP/port (good for ephemeral cloud VMs).
One message can go to multiple recipients.
Decouples sender from recipient (publish without caring who consumes).
One-way / asynchronous: sender doesn’t wait for or usually expect a reply (a
response, if any, goes on a separate channel).

Message brokers

Past: commercial (TIBCO, IBM WebSphere, webMethods). Now: open source —

RabbitMQ, ActiveMQ, HornetQ, NATS, Apache Kafka.
Model: a process sends to a named queue/topic; broker delivers to one or more
consumers/subscribers. Many producers and consumers per topic. A topic is one-way,
but a consumer can republish to another topic or a reply queue (enabling
request/response).
Brokers don’t enforce a data model — a message is just bytes + metadata, so any
encoding works. With forward/backward compatible encoding you can change
publishers and consumers independently and deploy in any order.
If a consumer republishes, preserve unknown fields (same caution as databases /
Figure 4-7).

Distributed actor frameworks

Actor model — concurrency via actors (encapsulated logic + local non-shared state)
communicating by async messages, avoiding threads/locks/races. Message delivery
not guaranteed (may be lost on error).
Distributed actor frameworks scale this across nodes; the same message-passing
works whether local or remote (transparently encoded over the network).
Location transparency works better here than in RPC, because the actor model
already assumes messages may be lost.
A distributed actor framework ≈ message broker + actor model in one. Rolling
upgrades still require fwd/bwd compatibility (messages flow between new- and
old-version nodes).
 Examples:
Akka — Java serialization by default (no compatibility); swap in Protobuf to
enable rolling upgrades.
Orleans — custom format by default, no rolling upgrades (deploy via new cluster
+ traffic shift); custom serialization plug-ins possible.
Erlang OTP — hard to change record schemas; rolling upgrades possible but
need careful planning (the maps type may help).

Key Takeaways

Forward + backward compatibility are the central goal: in any evolving system
(especially during rolling upgrades), old and new code run at the same time, so data
must be readable both ways — new code reads old data (backward) and old code
reads new data (forward).
Avoid language-specific serialization (Java/pickle/etc.) for anything durable or
cross-process — poor compatibility, security, efficiency.
Textual formats (JSON/XML/CSV) are universal but vague about datatypes
(numbers, binary) — fine as an informal interchange format, watch the edge cases.
Schema-driven binary formats (Thrift, Protobuf, Avro) give compact encoding,
precise compatibility rules, self-documenting schemas, and code-gen for typed
languages — at the cost of needing the schema to decode.
Thrift/Protobuf use field tags: rename freely, never change/reuse a tag; new fields
must be optional/defaulted; you can only remove optional fields.
Avro uses writer’s-vs-reader’s schema resolution by field name (no tags):
add/remove only fields with defaults; use union types for nullability; it’s uniquely
friendly to dynamically generated schemas.
Data outlives code — databases hold data in many historical schema versions;
schema evolution lets the whole DB appear as one schema. Beware silently dropping
unknown fields on read-modify-write.
RPC ≠ local calls — network requests are unpredictable, can time out, may need
idempotence for retries; modern frameworks (gRPC, Finagle) embrace this with
futures/streams. For RPC, assume servers upgrade first → need backward-compatible
requests, forward-compatible responses.
Message brokers decouple sender/receiver, buffer, redeliver, and fan out — any
encoding works as long as it’s compatible; preserve unknown fields when republishing.
Net result: with a little care, backward/forward compatibility and zero-downtime rolling
upgrades are very achievable — enabling frequent, low-risk deployments and an
 evolvable architecture.

(Bridge) Part II: Distributed Data — preview

The chapter ends as the book transitions into Part II. Reasons to distribute a DB across
machines: scalability (spread load), fault tolerance / high availability (redundancy),
latency (serve users from nearby datacenters).

Scaling up (vertical / shared-memory): one big machine, fast interconnect; cost

grows faster than linearly, limited fault tolerance, single location.
Shared-disk: independent CPUs/RAM, shared disk array over fast network; limited by
contention/locking.
Shared-nothing (horizontal / scaling out): independent nodes coordinating in
software over a conventional network; commodity hardware, multi-region, can survive
datacenter loss — the book’s focus (requires the most caution from developers).
Two ways data is distributed: Replication (copies on several nodes — redundancy +
performance, Ch. 5) and Partitioning / sharding (split into subsets across nodes, Ch.
6). They often combine.
Chapter 4: Encoding and Evolution - Assessment
Questions

Section 1: Conceptual/Reasoning Questions

Question 1

Why are field tags (rather than field names) critical to the encoding in Protocol Buffers and
Thrift? What specific property of field tags enables both forward and backward compatibility,
and what constraint does this impose on schema evolution?

Question 2

Avro does not use field tags. Instead, it relies on matching fields by name between the
writer’s schema and the reader’s schema. Explain why this design choice makes Avro
particularly well-suited for dynamically generated schemas (e.g., dumping a relational
database to a binary format). Why would Protocol Buffers or Thrift be more awkward for this
use case?

Question 3

The chapter states that in Protocol Buffers and Thrift, the required vs optional annotation
“makes no difference to how the field is encoded.” If this is true, what purpose does the
required marker actually serve, and why does the book argue that you can never make a

newly added field required if you want backward compatibility?

Question 4

Explain the concept of “data outlives code” in the context of database schema evolution.
How does this observation affect the strategy for managing encoding formats in a long-lived
production database, and how does Avro’s schema resolution mechanism address it
elegantly?
Question 5

The chapter identifies a subtle data loss scenario when an older version of application code
reads a record containing unknown fields, updates it, and writes it back. Explain the
mechanism by which data loss occurs and what principle must be upheld at the application
level to prevent it.

Question 6

Why does location transparency work better in the actor model than in traditional RPC?
What fundamental assumption in the actor programming model makes the mismatch
between local and remote communication less problematic?

Section 2: Scenario-Based Problems

Question 7

Your team is performing a rolling upgrade of a microservice. The new version adds a field
loyaltyTier (an enum with values GOLD, SILVER, BRONZE) to the user profile schema

encoded in Protocol Buffers.

(a) You make this field required . Describe exactly what happens when an instance running
the old code tries to read a record written by the new code, and vice versa.

(b) Propose a schema change design that maintains both forward and backward
compatibility during the rolling upgrade.

Question 8

A data engineering team uses Avro to serialize records from a PostgreSQL database into
files for a data warehouse. The database schema changes: column email is renamed to
contact_email .

(a) Without any special handling, what happens when the data warehouse (still using the old
reader’s schema with field name email ) tries to read new files?

(b) How can Avro’s schema features be used to handle this rename without breaking
existing readers?
Question 9

You are designing a system where Service A sends requests to Service B. During a
deployment, Service B is upgraded first (servers before clients). Service B’s new version
adds an optional response field estimatedDeliveryDate .

(a) Which compatibility direction (forward or backward) must the response encoding support
in this scenario? Justify your answer.

(b) If instead, clients are upgraded before servers, which compatibility direction must the
request encoding support?

Question 10

A message broker sits between producers and consumers. Producer v2 starts publishing
messages with a new field priority (default value: “NORMAL”). Consumer v1 reads these
messages, processes them, and republishes enriched versions to a downstream topic.

(a) If the encoding format supports forward compatibility, will Consumer v1 crash? Why or
why not?

(b) Identify a subtle correctness issue that can occur with the republished messages even
though no crash happens. What must Consumer v1 preserve?

Question 11

You have a Thrift schema where a field is defined as optional i32 quantity (field tag 4). A
new version changes this to a list<i32> quantities (same field tag 4). Explain whether this
evolution is safe or dangerous. How would the same evolution work differently in Protocol
Buffers using its repeated mechanism?

Section 3: Compare & Contrast

Question 12

Compare JSON, Protocol Buffers, and Avro across the following dimensions. Provide
specific details from the chapter for each cell:
 Dimension JSON Protocol Buffers Avro

Schema evolution mechanism

Binary encoding compactness

Human readability

Self-describing data

Suitability for dynamically generated schemas

Question 13

Compare REST and RPC (e.g., gRPC) on the following criteria:

Transparency of network semantics

Tooling and debugging ecosystem
Performance characteristics
Typical deployment context (public API vs internal services)
Handling of API versioning and evolution

Under what circumstances would you choose one over the other?

Question 14

Compare the three modes of dataflow discussed in the chapter (databases, service
calls/RPC, asynchronous message passing) in terms of:

Who encodes and who decodes

Temporal coupling between sender and receiver
Need for forward vs backward compatibility
How schema evolution is typically managed

Question 15

Compare how Akka, Orleans, and Erlang OTP handle message encoding and rolling
upgrades in their distributed actor frameworks. What is the common risk they all share, and
how does each framework’s default behavior address (or fail to address) it?
Section 4: True/False with Justification

Question 16

True or False: In Protocol Buffers, changing a field from optional to repeated is a safe,
backward-compatible schema evolution.

Provide a detailed justification citing the specific encoding mechanism that makes this
possible (or impossible) and describe what old code sees when reading data written by new
code.

Question 17

True or False: REST is always a better choice than RPC for service-to-service
communication because REST is simpler and more widely supported.

Justify your answer with at least three specific trade-offs discussed in the chapter.

Question 18

True or False: Avro’s binary encoding is self-describing – a reader can decode an Avro
binary record without access to the schema that was used to write it.

Justify your answer, and explain how Avro object container files reconcile the apparent
contradiction.

Section 5: Matching

Question 19

Match each encoding format/protocol (left) with its correct set of properties (right):

Formats: 1. Thrift BinaryProtocol 2. Thrift CompactProtocol 3. Protocol Buffers 4. Avro 5.

JSON (without schema) 6. SOAP/XML

Properties (each set applies to exactly one format):

A. Uses variable-length integers, packs field type and tag into a single byte, 34 bytes for the
example record
B. No field tags, fields matched by name between writer and reader schemas, 32 bytes for
example record
C. Uses field tags with full type bytes, no variable-length integer packing, most verbose
binary format among the three binary-schema formats
D. Human-readable text, self-describing, no formal schema required, ambiguous number
types
E. Uses variable-length integers, field tags, 33 bytes for example record, supports repeated

keyword for lists

F. XML-based, uses WSDL for API description, designed to be transport-protocol
independent, relies heavily on code generation and tooling

Question 20

Match each dataflow mode (left) with its compatibility requirement (right):

Dataflow Modes: 1. Database storage (single application, rolling upgrades) 2. RPC/Service

calls (servers upgraded before clients) 3. RPC/Service calls (clients upgraded before
servers) 4. Asynchronous message passing (independent producer/consumer deployment)

Compatibility Requirements:

A. Backward compatibility on requests (new servers read old client requests), forward
compatibility on responses (old clients read new server responses)
B. Both forward and backward compatibility needed, plus preservation of unknown fields
during read-modify-write cycles
C. Forward compatibility on requests (old servers read new client requests), backward
compatibility on responses (new clients read old server responses)
D. Both forward and backward compatibility needed; encoding format must allow publishers
and consumers to be deployed independently in any order

Answer Key
Question 1 - Answer

Field tags are numeric identifiers assigned to each field in the schema. They are critical
because the encoded binary data uses tag numbers (not field names) to identify fields. This
enables: - Forward compatibility: Old code encountering an unrecognized tag number can
skip over the field using the datatype annotation to determine how many bytes to skip. -
Backward compatibility: New code reading old data still recognizes existing tag numbers
with their original meanings.

The constraint: you can never change or reuse a field’s tag number after initial
deployment, because doing so would invalidate all previously encoded data.

Question 2 - Answer

Avro matches fields by name rather than by numeric tag. When generating a schema from a
relational database, column names map directly to Avro field names. If the database schema
changes (columns added/removed), a new Avro schema is generated automatically – no
manual tag assignment is needed.

With Protocol Buffers or Thrift, an administrator would need to manually assign and track tag
numbers for each column. The schema generator would have to ensure it never reuses
previously assigned tags (even for deleted columns), making automated schema generation
fragile and error-prone. This kind of dynamic schema generation simply was not a design
goal of Thrift or Protocol Buffers, whereas it was explicitly a design goal of Avro.

Question 3 - Answer

The required marker serves as a runtime validation check – if a field marked required is
not set, the encoding/decoding library raises an error. It has zero impact on the binary wire
format (nothing in the encoded bytes indicates required vs optional).

You cannot add a new required field because: when new code (which expects the required
field) reads old data (which was written without that field), the runtime check will fail since
old writers never included that field. This breaks backward compatibility. Therefore, every
field added after initial deployment must be optional or have a default value.
Question 4 - Answer

“Data outlives code” means that a database may contain values written years ago using old
schema versions, while application code gets replaced within minutes during deployment.
Unlike code (which is fully replaced), old data persists in its original encoding.

Strategy implications: - You cannot easily rewrite all data to match the latest schema
(expensive for large datasets) - The encoding format must support reading data written with
any historical schema version - Most relational databases handle this by filling in nulls for
new columns when reading old rows

Avro addresses this elegantly: the reader’s schema and writer’s schema are resolved at
read time. Each record stores (or references) the writer’s schema version. The Avro library
translates between schemas automatically, making the entire database appear as if
encoded with a single schema despite containing records from many schema eras.

Question 5 - Answer

The mechanism: 1. New code writes a record with a new field (e.g., field X) to the database
2. Old code reads the record, deserializes it into application model objects (which don’t know
about field X) 3. Old code modifies some other field and re-serializes the model object back
to the database 4. Field X is silently dropped because the old code’s model object never
captured it

Prevention principle: The application must preserve unknown fields during deserialization
and re-serialization. The encoding formats themselves support this (parsers can skip
unknown tags while keeping the raw bytes), but the application-level model object
translation must also be designed to pass through unrecognized fields unchanged.

Question 6 - Answer

Location transparency works better in the actor model because actors already assume that
messages may be lost, even within a single process. The programming model is inherently
asynchronous with no guaranteed delivery.

In traditional RPC, the abstraction tries to make remote calls look like local function calls, but
local calls are predictable (they either succeed, fail, or loop) while network calls have
fundamentally different failure modes (timeouts, lost responses, variable latency). The
mismatch is jarring.

In the actor model, since message loss and asynchronous behavior are the baseline
assumption regardless of locality, extending communication to remote nodes introduces
higher latency but not a fundamentally different failure model. The programmer has already
designed for unreliable message delivery.

Question 7 - Answer

(a) - Old code reading new data: Old code will see an unrecognized field tag and should
be able to skip it (forward compatible at the encoding level). However, required does not
affect encoding – the issue is only if old code is also expected to write valid records. - New
code reading old data: New code expects field loyaltyTier to be present (required). Since
old writers never included it, the runtime required-field check fails with an error, breaking
backward compatibility. The service will crash or reject the record.

(b) Make loyaltyTier an optional field with a sensible default (e.g., default to SILVER or an
UNKNOWN sentinel value). This way: - Old readers ignore the unrecognized tag (forward
compatible) - New readers handle missing field gracefully using the default (backward
compatible)

Question 8 - Answer

(a) The reader’s schema has field email but the writer’s new schema has field
contact_email . Avro matches by field name. Since no field named email exists in the writer’s

schema, Avro will use the default value from the reader’s schema for email . If no default is
defined, decoding fails. Either way, the actual data in contact_email is lost to the reader.

(b) Use Avro’s aliases feature. In the new writer’s schema, define the field as contact_email
with an alias of email . When schema resolution occurs, Avro matches the old reader’s
email field against the new writer’s contact_email field via the alias. Note: aliases make the

change backward compatible (new readers with aliases can read old data) but not
forward compatible (old readers without aliases cannot read new data). To maintain
forward compatibility, keep the old field name and add the alias in the reader’s schema.
Question 9 - Answer

(a) Since servers are upgraded first, the new server sends responses with the new field
estimatedDeliveryDate to old clients. Old clients must be able to read (and ignore) unknown

fields in responses. This requires forward compatibility on responses – old code can read
data written by new code.

(b) If clients upgrade first, new clients send requests with potentially new fields to old
servers. Old servers must handle requests containing unknown fields. This requires forward
compatibility on requests – old server code can read data written by new client code.
(Equivalently stated: backward compatibility on requests from the server’s perspective – but
the key direction is that old code reads new data, which is forward compatibility.)

Question 10 - Answer

(a) No, Consumer v1 will not crash. Forward-compatible encoding formats allow old code to
recognize that an unknown field tag is present and skip over it using the datatype annotation
to determine byte length. The priority field is simply ignored during processing.

(b) The subtle issue is loss of unknown fields during republishing. When Consumer v1
deserializes the message, processes it, and re-serializes it to the downstream topic, the
priority field may be silently dropped if the consumer’s application-level code does not

preserve unknown fields. Downstream Consumer v2 instances that expect priority will find
it missing. Consumer v1 must preserve unknown fields in messages that it republishes –
analogous to the database read-modify-write problem (Figure 4-7).

Question 11 - Answer

In Thrift, this is dangerous and not safe. Thrift has a dedicated list datatype that is
structurally different from a scalar field. Changing from i32 to list<i32> under the same tag
fundamentally changes the encoding structure. Old readers expecting a single i32 would
encounter list encoding bytes and misinterpret the data.

In Protocol Buffers, this evolution is safe because Protocol Buffers uses the repeated
marker instead of a dedicated list type. A repeated field is simply the same field tag
appearing multiple times. Changing optional to repeated : - New code reading old data:
sees a single value, interprets it as a list with one element - Old code reading new data:
sees multiple occurrences of the tag, keeps only the last element

This is possible because Protocol Buffers’ repeated encoding is just multiple instances of the
same tagged field, not a distinct container structure.

Question 12 - Answer

Dimension JSON Protocol Buffers Avro

No formal
Writer’s
mechanism;
schema +
adding fields
reader’s
to JSON
schema
objects is Field tags with
resolved at
generally required/optional/repeated
Schema evolution read time;
backward markers; old code skips
mechanism fields
compatible by unknown tags; new fields must
matched by
convention; no be optional
name; new
guarantees
fields must
without
have
external
defaults
tooling

Verbose (field
names
repeated as
Most
strings in
compact (32
every record;
Compact (33 bytes for bytes for
numbers as
Binary encoding example); variable-length example); no
ASCII text);
compactness integers; field tags instead of field
binary JSON
names identifiers or
variants
type markers
(MessagePack
in data at all
etc.) still
include field
names
 Dimension JSON Protocol Buffers Avro

Not human-
readable
Human-
Not human-readable (binary); (binary);
Human readability readable text
requires schema to decode requires
format
schema to
decode

No –
requires both
writer’s and
reader’s
Yes – field
schemas for
names
decoding;
Self-describing present in No – requires schema (field
but Avro
data data; can be tags are meaningless without it)
container
parsed without
files embed
schema
the schema,
making files
self-
describing

Excellent –
designed for
this; field
Good
Suitability for Poor – requires manual tag names map
(schema-free);
dynamically number assignment; automated directly from
easy to
generated generation must carefully avoid source
generate from
schemas tag reuse schema; no
any source
tag
management
needed

Question 13 - Answer
 Criterion REST RPC (e.g., gRPC)

Embraces
network New frameworks (gRPC, Finagle)
nature; does acknowledge network differences via
Network transparency
not hide that futures/streams, but RPC historically tried
operations go to hide network behind local-call semantics
over HTTP

Excellent –
browser, curl,
proxies, Requires specialized tooling; cannot easily
Tooling/debugging caches, load debug with curl; needs code generation
balancers, and specific client libraries
monitoring
tools all work

Higher
overhead (text-
Better performance with binary encoding
Performance based JSON,
and efficient serialization
HTTP
headers)

Predominant
for public APIs;
Primarily for internal service-to-service
Deployment context cross-
calls within same organization/datacenter
organizational
communication

URL
versioning,
Accept Inherits evolution rules of underlying
API versioning headers, API encoding (Protobuf tags, Thrift tags, Avro
keys with schemas)
version
mapping

Choice guidance: Use REST for public APIs, cross-organization integration, and when
debuggability matters. Use RPC for internal microservice communication where
performance is critical and you control both ends.
Question 14 - Answer

Service Calls Async Message

Aspect Databases
(RPC/REST) Passing

Writer
process
encodes;
reader Client encodes request, Sender/producer
Who process server decodes; server encodes;
encodes/decodes decodes encodes response, recipient/consumer
(possibly client decodes decodes
same
process at a
later time)

None –
writer and
reader may Low – sender fires and
be Synchronous – client forgets; broker buffers
Temporal
separated waits for response; tight messages; consumer
coupling
by years temporal coupling processes
(“data asynchronously
outlives
code”)

Compatibility Both Backward on requests, Both forward and

needs forward forward on responses backward (producers
AND (assuming servers and consumers deploy
backward upgrade first) independently in any
(rolling order)
upgrades
mean
old+new
code
coexist);
plus
unknown
field
preservation
 Service Calls Async Message
Aspect Databases
(RPC/REST) Passing
in read-
modify-write

Schema
versions in
API versioning (URL, Message broker is
database;
Schema headers, API keys); encoding-agnostic (just
fill nulls for
evolution encoding-specific rules bytes); schema registry
missing
management (Protobuf tags, JSON or encoding-level
columns;
optional fields) compatibility rules apply
Avro-style
resolution

Question 15 - Answer

Framework Default Encoding Rolling Upgrade Support

Does NOT provide forward/backward

compatibility by default. Must be
Akka Java’s built-in serialization
replaced with something like Protocol
Buffers to enable rolling upgrades.

Does NOT support rolling upgrades by

default. Requires setting up a new
Custom data encoding
Orleans cluster, migrating traffic, and shutting
format
down the old cluster. Custom
serialization plug-ins can be added.

Surprisingly hard to change record

schemas despite the system being
designed for high availability. Rolling
Erlang OTP Native term encoding
upgrades are possible but require
careful planning. Experimental maps

datatype may improve this.

Common risk: All distributed actor frameworks send messages between nodes that may be
running different code versions. If the message encoding does not support forward and
backward compatibility, rolling upgrades are impossible – you must stop the world or migrate
clusters. All three frameworks have this problem with their default serialization.

Question 16 - Answer

True.

In Protocol Buffers, there is no dedicated list/array type. Instead, repeated means the same
field tag appears multiple times in the encoded record. An optional field is simply a field tag
that appears zero or one times. Changing to repeated means it can appear zero or more
times.

Old code reading new data (multiple occurrences): Old code sees multiple entries for
the same tag and keeps only the last element of the list. It does not crash.
New code reading old data (zero or one occurrence): New code interprets a single
occurrence as a list with one element, or missing field as an empty list.

This is safe and backward compatible, though old readers will silently lose all but the last list
element.

Question 17 - Answer

False.

The chapter discusses several trade-offs that make this blanket statement incorrect:

1. Performance: Custom RPC protocols with binary encoding achieve better

performance than JSON over REST. For high-throughput internal service
communication, this matters significantly.

2. Use case fit: REST is predominant for public APIs, but RPC frameworks are the
better choice for internal service-to-service communication within the same
organization/datacenter, where both ends are controlled by the same team.

3. Modern RPC advantages: Newer RPC frameworks (gRPC, Finagle) provide features
like streaming (series of requests/responses over time), futures/promises for async
operations, and service discovery – capabilities not native to REST.
 4. The real trade-off: REST’s advantages (debuggability, universal language support,
vast ecosystem of tools) make it superior for public/external APIs. But RPC’s
advantages (performance, streaming, strong typing) make it superior for internal
service meshes. Neither is “always better.”

Question 18 - Answer

False.

Avro binary encoding is explicitly not self-describing. The encoded bytes contain only
values concatenated together – no field names, no field tags, no type annotations. A reader
MUST have access to the exact writer’s schema to decode the bytes correctly. Any
mismatch between the actual writer’s schema and what the reader assumes leads to
completely garbled data (unlike Protobuf/Thrift where at minimum field tags and type
annotations are present in the data).

Reconciliation via container files: Avro object container files embed the writer’s schema
once at the beginning of the file. This makes the file self-describing (any reader can
extract the embedded schema and use it for decoding), but the individual binary records
within the file are still not self-describing on their own. Other contexts (database records,
network communication) use schema version numbers or connection-level schema
negotiation to communicate the writer’s schema to readers.

Question 19 - Answer

1. Thrift BinaryProtocol –> C (field tags with full type bytes, no variable-length packing,
most verbose)
2. Thrift CompactProtocol –> A (variable-length integers, packs type+tag in one byte, 34
bytes)
3. Protocol Buffers –> E (variable-length integers, field tags, 33 bytes, repeated keyword)
4. Avro –> B (no field tags, matched by name, 32 bytes – most compact)
5. JSON (without schema) –> D (human-readable, self-describing, no schema required,
ambiguous numbers)
6. SOAP/XML –> F (XML-based, WSDL, transport-independent, code generation
dependent)
Question 20 - Answer

1. Database storage (single application, rolling upgrades) –> B (Both directions needed +
unknown field preservation during read-modify-write)
2. RPC/Service calls (servers upgraded before clients) –> A (Backward compat on
requests [new servers read old requests]; forward compat on responses [old clients
read new responses])
3. RPC/Service calls (clients upgraded before servers) –> C (Forward compat on
requests [old servers read new requests]; backward compat on responses [new clients
read old responses])
4. Asynchronous message passing (independent deployment) –> D (Both directions
needed; independent deployment order requires full bidirectional compatibility)