Link Search Menu Expand Document

Overview

Table of contents

  1. What is Kio?
  2. Modules
  3. Motivation
  4. Contribute

What is Kio?

Kio is a set of Kotlin extensions for Apache Beam implementing fluent-like API for Java SDK.

In general, Kio just adds some methods to the classes from Apache Beam. So, if you are a novice in distributed data processing, please check out the Beam’s programming guide first for a detailed explanation of the Beam programming model and concepts.

Thanks to the wonderful interact between Kotlin and Java you can use all methods from Kio and usual Beam Java SDK in the same program, but it can lead to poor readability. Therefore, we recommend you keep one style where possible.

Modules

The Kio project contains the following main modules:

  • core - common classes and methods (see Developers Guide)
  • connectors - modules to connect with various third-party systems (see Connectors)
  • test - a module to make testing of pipelines easier (see Testing)
  • cep - a module with methods for complex event processing (see Complex Event Processing)

Motivation

The main reason we founded the project is that the Beam Java SDK is very different to APIs of other tools like Spark, Flink, Scalding, Pandas, etc. The reasons for the Beam community chose this approach are well described in this blog post.

Kotlin allowed extending the API without losing full compatibility. Now you don’t need to look through the documentation to find PTransform class name what you should use in one place or another. Your IDE does everything for you:

With Kio Without Kio

Also, this project is a good place for experiments like the CEP module.

Contribute

Kio is an open-source project, so let’s expand the community together, making it more and more open and friendly. We’re waiting for everyone to join the community and contribute Kio. Feel free to do it, we are always happy to see you!

Kio is at the stage of active development and growth now, so any contribution will help us a lot.

Here is a list what you can to do for the project:

🐞 Find and report a bug
🌱 Suggest a new feature
We are tracking all bugs and suggestions on GitHub, please go to the Issues section and submit a new one.
🔣 Contribute code
🔍 Help with review
📄 Improve the      documentation
Feel free to submit a new pull request with any fixes and improvements that you consider appropriate. Also, you can take part in a code review of currently existing requests.
👥 Support users Check the latest issues which are labeled as a question or reply to possible questions on StackOverflow.
📢 Put in a good      word for Kio You can organize or attend a meetup or submit a new post to our blog.