Kafka

Inspect Kafka topics and groups with the Kafka MCP server

Connect an agent to a Kafka cluster through the Kafka MCP server and work through its four tools: list_topics for inventory, describe_topic for a single topic, describe_consumer_group for lag, and consume_messages for sampling — stream debugging from an agent, with the sampling risks called out up front.

Troubleshoot a lagging group with the Kafka Troubleshooter

Hand the Kafka Troubleshooter agent the consumer-group describe output, per-partition offsets, and consumer logs, then follow its four-step workflow from lag trend to the least destructive fix.

Debug a Kafka stream with the Kafka Console plugin

Drive the Kafka Console plugin's three commands across a lag incident: /topic-info for inventory, /lag-triage for per-partition lag and rebalance detection, and /sample-topic for a read-only message sample with a PII redaction pass.

Own the consumer side — offsets, rebalances, and the KIP-848 protocol

Consumer reliability is where streams quietly break: a rebalance loop looks like slowness, consumers past the partition count buy nothing, and stop-the-world rebalances pause consumption.

Production gotchas — the configs and capacities that break streams

The five causes of consumer lag on one card, the two destructive shortcuts that must never happen, and the capacity trap of scaling past the partition count.

Triage Kafka consumer lag from per-partition offsets

Combine the lag-triage skill's three questions with the lagging-consumer runbook: read per-partition lag to separate capacity from key skew, check the group state history for a rebalance loop, find the one stuck offset that marks a poison message, then apply the matching fix — and never reset offsets.

Kafka Consumer Lag Triage

• Read per-partition lag to separate skew from throughput
• Detect a rebalance loop from consumer-group state

Kafka MCP Server

Community server for Apache Kafka that lists topics, reads consumer-group lag, and samples messages, for stream debugging from an agent.

Kafka Console

Topic inventory, per-partition lag triage, rebalance detection and a read-only message sampler with a PII redaction pass.

Kafka Troubleshooter

Triage consumer lag, rebalance loops and poison messages in minutes.

Kafka consumer lagging

A consumer group is falling behind the topic and downstream tables are going stale.

Apache Kafka Documentation

Kafka's official documentation — architecture and design, delivery semantics, the protocol, consumer groups and offsets, operations, and security. The design section explains why ordering and partitions behave the way they do; the operations section companions our lag-triage skill and consumer-lag runbook when a group falls behind.

Optimize consumer rebalancing on Amazon MSK with next generation protocol

The KIP-848 consumer protocol in Apache Kafka 4.0 redesigns consumer group rebalancing to eliminate stop-the-world pauses.