When you split a monolith into microservices, communication becomes one of your hardest problems. Services need to talk to each other reliably. Get this wrong and you end up with slow, fragile systems that are harder to debug than the monolith you replaced.
Service discovery solves the issue of hardcoded service addresses in static deployments. Services register themselves on startup and look up others by name. Kubernetes has this built in through its DNS and Service abstraction. For non-Kubernetes environments, Consul or Netflix Eureka are common choices.
An API gateway is the single entry point for clients. It handles routing, load balancing, authentication, and protocol translation. Kong, Ambassador, and Azure API Management are all solid options. The gateway also gives you one place to enforce rate limiting and collect traffic metrics.
For synchronous communication between services, HTTP/HTTPS is the default. Simple and works with most tooling. gRPC is a better choice when you need high throughput or are calling across language boundaries. For async communication, use a message broker. Apache Kafka handles high-volume event streams well. RabbitMQ is simpler and works for most workloads.
Synchronous communication is simple but creates tight coupling. If Service B is slow or down, Service A blocks. Use it when you need an immediate response. Async messaging decouples services. Service A publishes an event, Service B processes it when ready.
The circuit breaker pattern stops cascading failures. When a service repeatedly fails, the circuit 'opens' and calls fail fast instead of waiting for a timeout. After a cooldown, it lets a few calls through to check recovery. Netflix Hystrix and resilience4j implement this for JVM services. Polly is the .NET option.
Distributed systems are hard to debug without good tooling. Set up distributed tracing with Jaeger or Azure Application Insights so you can follow a request across service boundaries. Use Prometheus and Grafana for metrics. Structure logs with correlation IDs so you can tie entries from different services to the same request.
Never trust inter-service communication by default. Use mutual TLS (mTLS) so services verify each other's identity. OAuth 2.0 with JWT tokens for authorization. RBAC to limit what each service can do. Encrypt everything in transit. In Kubernetes, network policies let you restrict which pods can talk to which.