SystemFw/Free conversation.md

Balaji Sivaraman @balajisivaraman_twitter

Hi all, I need some help understanding a piece of Doobie code from the examples. It is the StreamingCopy one: (https://github.com/tpolecat/doobie/blob/series/0.4.x/yax/example/src/main/scala/example/StreamingCopy.scala). I am using a modified version of the fuseMap2 example from that file. Here’s how I’ve modified it for my requirements:

  def fuseMap[F[_]: Catchable: Monad, A, B](
      source: Process[ConnectionIO, A],
      sink: Vector[A] => ConnectionIO[B],
      delete: ConnectionIO[Unit]
  )(
      sourceXA: Transactor[F],
      sinkXA: Transactor[F]
  ): F[Unit] =
    sinkXA.exec.apply(
      source
        .transact(sourceXA)
        .chunk(10000)
        .translate(λ[F ~> Kleisli[F, Connection, ?]](a => Kleisli(_ => a)))
        .evalMap(sink(_).foldMap(sinkXA.interpret))
        .append(Process.eval_(delete.foldMap(sinkXA.interpret)))
        .run
    )

Basically, I needed to use PostgresCopyManager to do Copy, instead of insert as in the example. That’s the reason for the chunking. Also I had to run a delete query as part of the same transaction. You can see the changes for this in the above code. I was able to make modifications purely by looking at the types and functions and using a bit of intuition. It works perfectly. The only problem is I have no idea why.

My knowledge of typed FP is basic. I can grok HKTs, understand why we do Catchable: Monad etc? But type lambdas and kind projector is where my brain becomes mush. I tried splitting up the above piece of code, but that didn’t help much, because I didn’t know how to break down the entire block that is passed to apply.

I'm still having difficulty wrapping my head around the types:

val exec: ~>[Kleisli[F, Connection, ?], F] = sinkXA.exec ->

This is the type of the exec expression. So my understanding is that we need to pass something of type Kleisli[F, Connection, A] to apply or something like that. Is that the type of the expression passed in to apply? For me, it shows the type is F[Unit]. My IntelliJ is also screwing up with the types. I also don't understand this line: translate(λ[F ~> Kleisli[F, Connection, ?]](a => Kleisli(_ => a))). Would appreciate help on that?

I read this post (http://underscore.io/blog/posts/2016/12/05/type-lambdas.html) to get a basic understanding of type lambdas, but it's not of much help here.

Fabio Labella @SystemFw

are you familiar with ~>?

Balaji Sivaraman @balajisivaraman

@SystemFw Apart from following the code and being taken to the natural transformation source code, not really. I also read this post (http://eed3si9n.com/learning-scalaz/Natural-Transformation.html), and got a very rudimentary idea, but the last part of that post with the Cat Theory was lost on me.

Fabio Labella @SystemFw so, basically you have an ADT describing the instructions of your language

trait KVS[A]
case class Get(k: String) extends KVS[Option[String]]
case class Put(k: String, v: String) extends KVS[Unit]

Fabio Labella @SystemFw

when we interpret a Free program, we get something of type A ( No, that's incorrect a Free monad is always translated into another Monad the other Monad then does what's needed to produce an A the only exception is when some examples use Id, with side effects

Fabio Labella @SystemFw cool, now let's see how the interpretation function looks in code yeah, but without getting too technical, that sentence in quotes above is actually what being free means anyway, the most common way to interpret a Free program (there are others) is through foldMap

 trait Free[F[_], A] {
 //...
 def foldMap[G](translator: F ~> G)(implicit gmonad: Monad[G]): G[A] = ???
}

Fabio Labella @SystemFw

if you give me a way to translate each instruction in Instr[] into an instruction in Target[] that's F ~> G, I'll explain more about it in a second I will translate each program in Free[Instr, A] that's why foldMap is on Free (implicit gmonad: Monad[G]) as long as the target is a monad yes

Fabio Labella @SystemFw wow that's the correct answer! we want to pass in a polymorphic function so when we pass translator, we want to keep its polymorphism because we will only specialise once we have a specific instruction this difference can be made specific in types, but scala doesn't give you a nice notation for it I'm going to write it in Haskell for a second, but it should be ok:

theTranslatorYouHave :: forall f g b. (f b -> g b) -- f b = Instr[B], g b = Task[B]

theTranslatorYouWant: forall f g. (forall b. f b -> g b)

which basically means

Fabio Labella @SystemFw yes, it's roughly right the ability to pass polymorphic functions around, without losing their polymorphism is called higher-rank polymorphism scala does not have such a feature however, we can work our way around it let's create a trait

 trait Translator[F[_], G[_]] {
  def apply[B](in: F[B]): G[B]
}

now we can write

def translator: Translator[Instr, Task] = new Translator[Instr, Task] {
  def apply[B](in: Instr[B]): Task[B] = in match {
      case Put(k, v) => Task.delay (DB.put(k, v))
      case Get(k) => Task.delay(DB.get(k))
}

Fabio Labella @SystemFw exactly, A is the return type of the whole program B is the return type of each individual instructions, that's why we need the polymorphism for Category Theory reasons that are (as I hope I've proved) not necessary to understand how Free works to an usable extent, Translator is the natural transformation:

trait ~>[F[_], G[_]] {
  def apply[A](f: F[A]): F[B]
}

Fabio Labella @SystemFw Yes, the important thing is to grasp the key concepts:

• A Free monad is always translated to another monad, which does the actual work
• The process of interpretation takes a translation between instructions, and gives a translation between programs
• You need higher-rank polymorphism (the trait trick)
• everything follows by having things fit into the right shape

Fabio Labella @SystemFw @ZizhengTai Thank you! :-) I will put this in a gist for future consultation, I think conversations are a great way to learn