Skip to content
Yassir Halaoui
FR
Lire en français

Competence

Kafka, Debezium and event-driven systems where money moves

I design event pipelines for systems that can't lose a message or apply one twice: change data capture from PostgreSQL, the transactional outbox, and Avro schemas that evolve across teams.

In short

Yassir Halaoui designs event-driven systems with Apache Kafka, Kafka Connect and Debezium change data capture, using the transactional outbox pattern and Avro with Confluent Schema Registry. On a multi-country payments platform he built a CDC pipeline on PostgreSQL's write-ahead log, including a heartbeat that keeps the replication slot moving so the WAL cannot fill the disk.

Where I've used it

  1. Pluxee · 2025 to now

    Designed the change-data-capture pipeline of a multi-country beneficiary platform: transactional outbox, Debezium 2.4 reading PostgreSQL through pgoutput, Kafka, and Avro with Confluent Schema Registry for schema evolution.

  2. Pluxee · 2025 to now

    Added a heartbeat that advances the replication slot so quiet tables can't pin the write-ahead log, and provisioned Kafka topics and their roles with Terraform.

  3. CIH Bank · 2017 to 2022

    Worked with IBM MQ messaging in production banking services.

  4. RevealTogether

    Real-time events over WebSockets backed by Redis, pushing votes and the reveal moment to every connected guest.

Everything in this area

Streaming
  • Apache Kafka
  • Kafka Connect
  • Debezium (CDC)
  • IBM MQ
Contracts
  • Avro
  • Confluent Schema Registry
  • OpenAPI
Patterns
  • Transactional outbox
  • Idempotent consumers
  • Dead letter topics
  • Event-driven architecture
  • Domain-driven design
Real time
  • WebSockets
  • Redis

In depth

CDC on a database you don't own

The failure that takes Debezium down is rarely the connector. It's the replication slot pinning PostgreSQL's write-ahead log until the disk fills, on a busy database whose captured tables happen to be quiet. A heartbeat that writes to a captured table every few seconds keeps the slot moving, and it's the first thing I add.

Exactly once is a design, not a setting

Kafka delivers at least once. Making that safe means the deduplication record and the state change commit in the same transaction, side effects go through an outbox, and the key is a business key rather than a UUID the producer made up.

Schemas that outlive a sprint

Across team boundaries I default to FULL_TRANSITIVE compatibility, give every new field a default, and retire fields over quarters rather than sprints. The registry protects the wire format. Talking to the consuming team protects everything else.

Questions

Can you help us set up Kafka with Debezium CDC?

Yes. I've run Debezium on PostgreSQL with pgoutput in production. Setup is the easy part; the heartbeat, slot monitoring and the agreement with whoever owns the database are what keep it running.

Transactional outbox or change data capture?

If you own the application that writes the data, an outbox table drained by a poller is usually simpler to run. CDC earns its cost when you can't change the writer. Both need idempotent consumers downstream.

Why is our PostgreSQL WAL growing with Debezium?

Almost always an inactive or lagging replication slot. Debezium only advances the slot when it commits an offset, so quiet captured tables on a busy database pin the WAL. Add a heartbeat action that writes to a captured table, and alert on retained WAL per slot.

Is exactly-once processing possible with Kafka?

End to end, in practice you get at-least-once delivery plus an idempotent consumer. That's achievable and reliable: commit the deduplication marker with the state change, and push side effects through an outbox.

Field notes

Shipped work

Related competences