OLake 9th Community Meetup

## **OLake Community Call #9** ​**Introducing Kafka-Powered CDC Pipelines and Smarter Ingestion Controls Across the Open Lakehouse** ​In our previous community call, we explored real-world CDC challenges, showcased **Oracle support**, **incremental syncs**, **ingestion filters**, and **Helm-based deployments**, demonstrating how OLake simplifies open lakehouse operations end to end. ​For our **9th community meetup**, we’re introducing the next wave of advancements expanding OLake’s CDC ecosystem and refining user control, performance, and reliability. ​**1\. Expanding with Kafka Support** ​This update brings **Kafka support**, enabling data ingestion from Kafka topics directly into Iceberg. It supports batch data ingestion ideal for modern architectures and will be demonstrated live during the community call. ​**2\. Smarter Sync Management** * ​**Clear Destination:** Erases all data from the destination for a particular job, simplifying reconfiguration and cleanup. * ​**Cancel Job:** Safely stop running syncs while preserving checkpoints for consistent recovery. * ​**Flexible Ingestion Modes:** Choose between *Append* for ingesting all records or *Upsert* for keeping only the latest updates. ​**3\. Simplified Iceberg Destination Handling** * ​**Table/Column Normalization:** Table and column names are normalized to ensure compatibility with tools like AWS Glue and others that don’t support uppercase letters or special characters. * ​**Destination Database & Namespace Options:** When a job is created and streams are discovered, Olake automatically creates a destination database to store synced tables. You can choose between **per-namespace** or a **unified database** setup ensuring seamless compatibility across **Trino, Athena, and Iceberg**. ​**4\. Secure Connectivity** * ​**IAM Integration for MongoDB:** Passwordless AWS IAM-based authentication, reducing credential management and improving compliance. ​**5\. Documentation & Learning** ​We’ve **revamped the documentation** to make contributing and experimenting easier than ever. A new set of **blogs around Apache Iceberg** and tutorials with **Polaris and Bauplan** highlight adoption patterns and practical workflows across open lakehouse stacks. ​**6\. Community Spotlight** ​We’ll wrap up with a **community spotlight**, celebrating contributions from our **Hacktoberfest participants** and ongoing open-source efforts. From PRs to discussions, our contributors continue to drive the wave toward a more open, collaborative, and high-performance data ecosystem.