Integration akka-http with Spark (Scala example)
In this topic, I will explain one example of integration akka-http (server-side) with Spark (spark session). Why it can be necessary sometimes? Imagine that: 1) You have source data (from Cassandra, Oracle, Parquet, etc) 2) And you have a machine learning model that uses this data to make classification, for example, Multilayer perceptron classifier (MLPC). MLPC is a classifier based on the feedforward artificial neural network. 3) You have written code on Scala for Spark that load source data, train MLPC model and can be used to predict output value (label) by input value(features). You want to run it all on Spark with standalone jar application and communicate with application from external, it can be RPC or any. I will use the easiest way - simple HTTP and HTML. 1) Source data description: I have Cassandra NoSQL database with previously calculated Column Family (like a table in RBDMS). Screenshot made from DataStax DevCenter 1.5.0 Here we have different Forex tickers inf