Abroskin Alexander Arkweid

Dagger Demos

Summary

Dagger is a tool for modeling your workflows as data that can be composed together at runtime.

Dagger constructs can be integrated into a Dagger.Workflow and evaluated lazily in concurrent contexts.

Getting Lazy with Dataflow Graphs in Elixir

Intro

What do Tensorflow, Apache Airflow, Rule Engines, and Excel have in common?

Under the hood they all use DAGs to model data-flow dependencies of the program. Using graphs to model programs is great because you can modify the program at runtime. Lets talk about doing this in Elixir for great good.

Since it has come up a few times, I thought I’d write up some of the basic ideas around domain modeling in Clojure, and how they relate to keyword names and Specs. Firmly grasping these concepts will help us all write code that is simpler, cleaner, and easier to understand.

Clojure is a data-oriented language: we’re all familiar with maps, vectors, sets, keywords, etc. However, while data is good, not all data is equally good. It’s still possible to write “bad” data in Clojure.

“Good” data is well defined and easy to read; there is never any ambiguity about what a given data structure represents. Messy data has inconsistent structure, and overloaded keys that can mean different things in different contexts. Good data represents domain entities and a logical model; bad data represents whatever was convenient for the programmer at a given moment. Good data stands on its own, and can be reasoned about without any other knowledge of the codebase; bad data is deeply and tightly coupled to specific generating and

Profiling performance

Using perf:

$ perf record -g binary
$ perf script | stackcollapse-perf.pl | rust-unmangle | flamegraph.pl > flame.svg

NOTE: See @GabrielMajeri's comments below about the -g option.

-# delete local tag '12345'
-git tag -d 12345
-# delete remote tag '12345' (eg, GitHub version too)
-git push origin :refs/tags/12345
-# alternative approach
-git push --delete origin tagName
-git tag -d tagName

	Latency Comparison Numbers (~2012)
	----------------------------------
	L1 cache reference 0.5 ns
	Branch mispredict 5 ns
	L2 cache reference 7 ns 14x L1 cache
	Mutex lock/unlock 25 ns
	Main memory reference 100 ns 20x L2 cache, 200x L1 cache
	Compress 1K bytes with Zippy 3,000 ns 3 us
	Send 1K bytes over 1 Gbps network 10,000 ns 10 us
	Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD