Reactive: Akka Streams, RxJava, Ratpack

How Akka Streams can be used to process the Wikidata dump in parallel and using constant memory with just your laptop.

Here at Intent HQ we use Wikipedia and Wikidata as sources of data. They are very important to us because they both encode an enormous amount of information in several languages that we use to build our Topic Graph.
Although the current process we have to process these dumps works well enough, we are always interested in finding new and better ways of doing our work. It’s because of that that we were very excited when we saw the Reactive Streams initiative ¹. We thought it could be used to process the largest encyclopædia in the world ².