Kafka topics are long-lived. A schema decision made when a topic is created persists for years. Designing Kafka schemas with evolution in mind from the start prevents painful compatibility breaks later.
Schema format selection
Avro, Protocol Buffers, and JSON are the three practical schema formats for Kafka. Avro is the standard in the Confluent ecosystem: compact binary encoding, schema embedded in the Confluent Schema Registry, and mature compatibility checking. Protobuf provides a better developer experience (code generation, nested types, clear field numbering) and is increasingly supported by Schema Registry. JSON is schema-optional: convenient for development, hazardous for production (no type enforcement, no schema evolution guarantees).
Schema evolution compatibility rules
Schema Registry enforces compatibility rules. Backward compatible: new schema can read data written with old schema (consumers can upgrade before producers). Forward compatible: old schema can read data written with new schema (producers can upgrade before consumers). Full compatible: both backward and forward compatible. Adding optional fields is backward compatible. Removing fields is forward compatible. Changing field types or making optional fields required are breaking changes regardless of compatibility mode.
Namespace and naming conventions
Kafka schemas should use reverse-domain namespace conventions (com.company.domain.EventName) to avoid naming conflicts across teams. Event names should be past-tense domain events (OrderPlaced, PaymentReceived, InventoryReserved) rather than commands or internal state names. The schema name and namespace are part of the schema fingerprint in Schema Registry, renaming a schema creates a new schema, not a version of the old one.
The schema registry operational model
Schema Registry is a critical dependency for Kafka producers and consumers that use registered schemas. Schema Registry availability requirements are similar to Kafka's: producers cannot produce if they cannot register or validate schemas. The operational setup: deploy Schema Registry in high-availability mode (multiple instances behind a load balancer), include Schema Registry in disaster recovery plans, and configure producers to cache schemas locally to tolerate brief Schema Registry outages.