confluentinc/bottledwater-pg: Change data capture from PostgreSQL into Kafka

confluentinc/bottledwater-pg: Change data capture from PostgreSQL into Kafka: "Bottled Water uses the logical decoding feature (introduced in PostgreSQL 9.4) to extract a consistent snapshot and a continuous stream of change events from a database. The data is extracted at a row level, and encoded using Avro. A client program connects to your database, extracts this data, and relays it to Kafka (you could also integrate it with other systems if you wish, but Kafka is pretty awesome). Key features of Bottled Water are: Works with any PostgreSQL database (version 9.4 or later). There are no restrictions on your database schema. No schema changes are required, no triggers or additional tables. (However, you do need to be able to install a PostgreSQL extension on the database server. More on this below.) Negligible impact on database performance. Transactionally consistent output. That means: writes appear only when they are committed to the database (writes by aborted transactions are discarded), writes appear in the same order as they were committed (no race conditions). Fault-tolerant: does not lose data, even if processes crash, machines die, the network is interrupted, etc."



'via Blog this'