Bigdata ideas

 Scala Worksheet import java.util.concurrent.TimeUnit import zio.clock.Clock import zio.console.putStrLn import zio.console.Console import zio.duration.durationInt import zio.{ExitCode, Runtime, ZEnv, ZIO} val prg: ZIO[Console with Clock, Throwable, Int] = for { c <- ZIO.access[Clock](_.get) zio_begin <- c.currentTime(TimeUnit.MILLISECONDS) zio_beginM <- ZIO.accessM[Clock](_.get.currentTime(TimeUnit.MILLISECONDS)) sys_begin <- ZIO.succeed(System.currentTimeMillis) _ <- putStrLn(s" zio_begin = $zio_begin") _ <- putStrLn(s" zio_beginM = $zio_beginM") _ <- putStrLn(s" sys_begin = $sys_begin") _ <- c.sleep(1.second) zio_end <- c.currentTime(TimeUnit.MILLISECONDS) zio_endM <- ZIO.accessM[Clock](_.get.currentTime(TimeUnit.MILLISECONDS)) sys_end <- ZIO.succeed(System.currentTimeMillis) _ <- putStrLn(s" zio_end = $zio_end") _ <- putStrLn(s" zio_endM = $zio_beginM") _ <- putStrL

A good example, the book "Programming Scala" by Dean Wampler & Alex Payne.  Page 283 trait Function2[-T1, -T2, +R] extends AnyRef The last type parameter, +R, the is the return type. It is covariant. The leading two type parameters are for the first and second function arguments, respectively. They are contravariant.   class CSuper { def msuper() = println("CSuper") } class C extends CSuper { def m() = println("C") } class CSub extends C { def msub() = println("CSub") } var f: C => C = (c: C) => new C // f = (c: CSuper) => new CSub // c: CSuper is valid, because the argument // C is contravariant, so CSuper is a valid substitution // while the return value is covariant, so CSub is a valid replacement for C. f = (c: CSuper) => new C // Similar to the previous case, but we simply return a C. f = (c: C) => new CSub // Similar to the previous cases, but we sim

1) andThen - composes 2 functions from left to right, "execute first and send result into input of seconds function and etc"  Example: we have 4 functions: String => Int  Int => Int Int => Double Double => String we can compose all this function in chain and produce function  String  =>  String , look at val func val StrToInt :String => Int = param => param.toInt val mult2: Int => Int = param => param*2 val addSmall: Int => Double = param => param+0.1 val DblToString :Double => String = param => param.toString val initVal:String = "123" (StrToInt andThen mult2 andThen addSmall andThen DblToString)(initVal) DblToString(addSmall(mult2(StrToInt(initVal)))) val func : String => String = StrToInt andThen mult2 andThen addSmall andThen DblToString func(initVal) Output StrToInt: String => Int = function mult2: Int => Int = function addSmall: Int => Double = function DblToString: Double => String = fun

When we use Futures in scala sometimes we need to combine sequentially and parallel execution of Futures. My research in this way is beginning from a task - generate a lot of Users (100K) and insert it all in different databases (Postgres and Cassandra) to make common score of write speed. case class User(id: Long, name: String, email: String, edomain: String) For Postgres, I should have used  Slick library and for Cassandra  Phantom  library (Schema safe, type-safe, reactive Scala driver for Cassandra/Datastax Enterprise). Steps ware easy, generate Seq of Users and in loop insert there into a database. With Postgres driver and with this count of items it was ok, every row was inserted without error. But Phantom is reactive and insert command return Future. (somewhere about) A snippet of code: abstract class CassUsers extends Table[CassUsers, User] with RootConnector { override def tableName: String = "users"

#1. Why method generates an error at compile time : Error here because the expression is expanding into SOMETHING match { case xs :Seq[Int] if xs.isEmpty => 0 case xs :Seq[Int] if xs.nonEmpty => xs.head + recursiveSum(xs.tail: _*) } and scala compiler doesn't know what is SOMETHING and his type. You think that it is xs : Int* parameter, but why. Functions definition can be like this def recursiveSum(a :Int,s :String, xs :Int*) :Int = { We can solve this by explicitly adding the parameter in the patter match, like this ( inside {} we use xs but it can be any name, it's not referred to internal parameter xs. ) def recursiveSum(xs :Int*) :Int = { xs match { case xs if xs.isEmpty => 0 case xs if xs.nonEmpty => xs.head + recursiveSum(xs.tail: _*) } } recursiveSum((1 to 5): _ *) result : 15 Also, we can do it without pattern matching, with IF val g: (Seq[Int]) => Int = si => { if (si.isEmpty) 0 else si.head + g(si.tail

In this topic, I will collect different questions and small tasks of Scala. Solutions or explanations will be on any page. All related by # - numbers. #1. Why method generates an error at compile time: error: missing parameter type for expanded function <console>:11: error: missing parameter type for expanded function The argument types of an anonymous function must be fully known. (SLS 8.5) Expected type was: Int        def recursiveSum(xs :Int*) :Int = {                                                               ^ For example, you write a method like this, to calculate sum recursively. def recursiveSum(xs :Int*) :Int = { case xs :Seq[Int] if xs.isEmpty => 0 case xs :Seq[Int] if xs.nonEmpty => xs.head + recursiveSum(xs.tail: _*) } #2. How to make a chain of PartialFunctions, manually and with fold, and use it to filter data.      And the short explanation about P.F. and their methods. val resultSeqPers :Seq[Person] = sourceSeqPers collect Co