- 
      
 - 
        
Save dogenpunk/6da9f9e7a5b621039f15 to your computer and use it in GitHub Desktop.  
Revisions
- 
        
kachayev revised this gist
Sep 1, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -213,7 +213,7 @@ Lets try to build something more complicated and interesting: (return (str "major: " major "; minor: " minor)))) (parse-all clojure-version "clojure 1.7") ;; => "major: 1; minor: 7" ``` ## 5. CSS  - 
        
kachayev revised this gist
Sep 1, 2014 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -153,8 +153,8 @@ Useful combinators that will help us: (do* (r1 <- p1) (r2 <- p2) ;; xxx: note, that it's dirty hack to use STR to concat outputs ;; Full functional implementation should use MonadPlus protocol (return (str r1 r2)))) ;; (a|b)  - 
        
kachayev revised this gist
Sep 1, 2014 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -73,11 +73,11 @@ We also will use few helpers: I'm not going to pay much attention to what ```monads``` are. Everything that you need to know is that we will use it to compose small parsers into bigger one. Also we're not going to implement entire "monad infrastructure". ```clojure ;; builds parser that always returns given element without consuming (changing) input (defn return [v] (fn [input] (list [v input]))) ;; takes parser and function that builds new parsers from (each) result of applying first one (defn >>= [m f] (fn [input] (->> input  - 
        
kachayev revised this gist
Aug 31, 2014 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -80,7 +80,9 @@ I'm not going to pay much attention to what ```monads``` are. Everything that yo ;; take parser and function that builds new parsers from result of applying first one (defn >>= [m f] (fn [input] (->> input (parse m) (mapcat (fn [[v tail]] (parse (f v) tail))))) ``` Next part is a simple macro that provides haskell-like ```do``` notation. If you don't know how ```do``` block works in Haskell, just skip this code and return to it when necessary.  - 
        
kachayev revised this gist
Aug 30, 2014 . 1 changed file with 4 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -62,7 +62,10 @@ We also will use few helpers: (parser input)) (defn parse-all [parser input] (->> input (parse parser) (filter #(= "" (second %))) ffirst)) ``` ## 2. Monads  - 
        
kachayev revised this gist
Aug 30, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -40,7 +40,7 @@ From theory: type Parser a = String -> [(a, String)] ``` So, we're going to represent ```parser``` as simple function that takes input and returns seq of possible results. Simplest parsers: ```clojure ;; takes any input and "consume" first char from it  - 
        
kachayev revised this gist
Aug 30, 2014 . 1 changed file with 29 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -143,6 +143,15 @@ Ok, so far so good. Moving forward... Useful combinators that will help us: ```clojure ;; (ab) (defn and-then [p1 p2] (do* (r1 <- p1) (r2 <- p2) ;; xxx: note, that it's dirty hack to use ;; full functional implementation should use MonadPlus protocol (return (str r1 r2)))) ;; (a|b) (defn or-else [p1 p2] (fn [input] @@ -177,9 +186,29 @@ Pay special attention to ```plus``` combinator implementation using ```do*``` me Example of combinators usage: ```clojure ;; recognizes space (or newline) (def space (or-else (match " ") (match "\n"))) ;; recognizes empty string or arbitrary number of spaces (def spaces (many space)) ;; recognizes given string, i.e. "clojure" (defn string [s] (reduce and-then (map #(match (str %)) s))) ``` Lets try to build something more complicated and interesting: ```clojure (def clojure-version (do* (string "clojure") (match " ") (major <- digit) (match ".") (minor <- digit) (return (str "major: " major "; minor: " minor)))) (parse-all clojure-version "clojure 1.7") ;; => "major: 1 minor: 7" ``` ## 5. CSS  - 
        
kachayev revised this gist
Aug 30, 2014 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -107,10 +107,10 @@ Let's build basic parsers that can recognize chars, strings, letters, spaces, et (fn [c] (sat (partial f (first c))))) ;; recognizes given char (def match (char-cmp =)) ;; rejects given char (def noneOf (char-cmp not=)) ;; just a helper (defn from-re [re]  - 
        
kachayev revised this gist
Aug 30, 2014 . 1 changed file with 8 additions and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -102,14 +102,19 @@ Let's build basic parsers that can recognize chars, strings, letters, spaces, et (defn sat [pred] (>>= any (fn [v] (if (pred v) (return v) failure)))) ;; just a helper (defn char-cmp [f] (fn [c] (sat (partial f (first c))))) ;; recognizes given char (defn match (char-cmp =)) ;; rejects given char (defn noneOf (char-cmp not=)) ;; just a helper (defn from-re [re] (sat (fn [v] (not (nil? (re-find re (str v))))))) ;; recognizes any digit (def digit (from-re #"[0-9]"))  - 
        
kachayev revised this gist
Aug 30, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -49,7 +49,7 @@ So, we're going to represent ```parser``` as simple function that takes input an (list [(first input) (apply str (rest input))]))) ;; this one doesn't accept any input (defn failure [_] '()) (any "clojure-1.7") ;; => ([c lojure-1.7])  - 
        
kachayev revised this gist
Aug 30, 2014 . 1 changed file with 4 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,7 +1,11 @@ # Parsing CSS file with monadic parser in Clojure Inspired by ["Parsing CSS with Parsec"](http://blog.jakubarnold.cz/2014/08/10/parsing-css-with-parsec.html). Just quick notes and code that you can play with in REPL. By [@kachayev](https://twitter.com/kachayev) ## 0. Intro What do we have?  - 
        
kachayev revised this gist
Aug 30, 2014 . 1 changed file with 9 additions and 7 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -80,7 +80,9 @@ Next part is a simple macro that provides haskell-like ```do``` notation. If you ```clojure (defn merge-bind [body bind] (if (and (not= clojure.lang.Symbol (type bind)) (= 3 (count bind)) (= '<- (second bind))) `(>>= ~(last bind) (fn [~(first bind)] ~body)) `(>>= ~bind (fn [~'_] ~body)))) @@ -194,22 +196,22 @@ Now lets define ```rule``` and ```ruleset``` parsers: (def letter+ (or-else letter (match "-"))) (def rule (do* (p <- (many (noneOf ":"))) (match ":") spaces (v <- (many (noneOf ";"))) (match ";") spaces (return (Rule. (apply str p) (apply str v))))) (def ruleset (do* (s <- (plus (noneOf "{"))) (match "{") spaces (r <- (plus rule)) spaces (match "}") spaces (return (Ruleset. (trim (apply str s)) r)))) ```  - 
        
kachayev revised this gist
Aug 30, 2014 . 1 changed file with 1 addition and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -79,12 +79,10 @@ I'm not going to pay much attention to what ```monads``` are. Everything that yo Next part is a simple macro that provides haskell-like ```do``` notation. If you don't know how ```do``` block works in Haskell, just skip this code and return to it when necessary. ```clojure (defn merge-bind [body bind] (if (and (= 3 (count bind)) (= '<- (second bind))) `(>>= ~(last bind) (fn [~(first bind)] ~body)) `(>>= ~bind (fn [~'_] ~body)))) (defmacro do* [& forms] (reduce merge-bind (last forms) (reverse (butlast forms))))  - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 26 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -98,13 +98,37 @@ Let's build basic parsers that can recognize chars, strings, letters, spaces, et (defn sat [pred] (>>= any (fn [v] (if (pred v) (return v) failure)))) ;; recognizes given char (defn match [c] (sat (partial = (first c)))) ;; rejects given char (defn noneOf [c] (sat (partial not= (first c)))) ;; just a helper (defn from-re [re] (sat (fn [v] (not (nil? (re-find re (str v))))))) ;; recognizes any digit (def digit (from-re #"[0-9]")) ;; recognizes any letter (def letter (from-re #"[a-zA-Z]")) ``` Lets try: ```clojure (parse (match "c") "clojure1.7") ;; => ([\c "lojure1.7"]) (parse letter "clojure1.7") ;; => ([\c "lojure1.7"]) (parse digit "1.7clojure") ;; => ([\1 ".7clojure"]) ``` Ok, so far so good. Moving forward... ## 4. Combinators Useful combinators that will help us: @@ -146,6 +170,7 @@ Example of combinators usage: ```clojure (def space (or-else (match " ") (match "\n"))) (def spaces (many space)) ``` ## 5. CSS  - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -121,7 +121,7 @@ Useful combinators that will help us: ;; (a*) (defn many [parser] (optional (plus parser))) ;; (a+) equals to (aa*) (defn plus [parser] (do* (a <- parser)  - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -113,7 +113,7 @@ Useful combinators that will help us: ;; (a|b) (defn or-else [p1 p2] (fn [input] (lazy-cat (parse p1 input) (parse p2 input)))) (declare plus) (declare optional)  - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -107,7 +107,7 @@ Let's build basic parsers that can recognize chars, strings, letters, spaces, et ## 4. Combinators Useful combinators that will help us: ```clojure ;; (a|b)  - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 2 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,5 +1,7 @@ Inspired by ["Parsing CSS with Parsec"](http://blog.jakubarnold.cz/2014/08/10/parsing-css-with-parsec.html). Just quick notes and code that you can play with in REPL. ## 0. Intro What do we have?  - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -217,7 +217,7 @@ Looks nice. ## 6. Notes - it's easier to work both with monads and parsers in dynamically typed language - it's not that convenient to work with ```string```s and ```list``` of ```char```s (you can see few "irrational" ```apply str _``` and ```first _```)  - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 15 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -171,7 +171,7 @@ Now lets define ```rule``` and ```ruleset``` parsers: (def rule (do* (p <- (many letter+)) (match ":") (_ <- spaces) ;; xxx: fix do* to avoid unnecessary "_" binding (v <- (many (noneOf ";"))) (match ";") (_ <- spaces) @@ -199,6 +199,12 @@ Play a bit: ;; :rules ;; ({:key "background", :value "#fafafa"} {:key "color", :value "red"})} (def css ".container h1 { color: rgba(255, 0, 0, 0.9); font-size: 24px; font-family: Monaco; }") (parse-all ruleset css) ;; {:selector ".container h1", ;; :rules @@ -211,3 +217,11 @@ Looks nice. ## 6. Notes - it's easier to work both with monads and parser in dynamically typed language - it's not that convenient to work with ```string```s and ```list``` of ```char```s (you can see few "irrational" ```apply str _``` and ```first _```) - there is still room for improvement, i.e. more general ```return```, ```bind``` and ```do*``` - you can try to rewrite parsers using applicative style 
 - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 77 additions and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,4 +1,8 @@ Inspired by ["Parsing CSS with Parsec"](http://blog.jakubarnold.cz/2014/08/10/parsing-css-with-parsec.html). ## 0. Intro What do we have? ``` .container h1 { @@ -8,7 +12,7 @@ What we have? } ``` What do we want to get? ```clojure {:selector ".container h1", @@ -126,7 +130,14 @@ Useful combinators that we will use: (defn optional [parser] (or-else parser (return ""))) ``` Pay special attention to ```plus``` combinator implementation using ```do*``` mentioned above. ```do*``` is only syntax sugar for ```bind``` operations, in reality ```plus``` looks like: ```clojure (defn plus [parser] (>>= parser (fn [a] (>>= (many parser) (fn [as] (return (cons a as))))))) ``` Example of combinators usage: @@ -137,3 +148,66 @@ Example of combinators usage: ## 5. CSS From CSS 2.1 grammar definition: ```haskell type Selector = String data Rule = Rule String String deriving Show data Ruleset = Ruleset Selector [Rule] deriving Show ``` In Clojure we can use records to represent data types: ```clojure (defrecord Rule [key value]) (defrecord Ruleset [selector rules]) ``` Now lets define ```rule``` and ```ruleset``` parsers: ```clojure (def letter+ (or-else letter (match "-"))) (def rule (do* (p <- (many letter+)) (match ":") (_ <- spaces) (v <- (many (noneOf ";"))) (match ";") (_ <- spaces) (return (Rule. (apply str p) (apply str v))))) (def ruleset (do* (s <- (plus (noneOf "{"))) (match "{") (_ <- spaces) (r <- (plus rule)) (_ <- spaces) (match "}") (_ <- spaces) (return (Ruleset. (trim (apply str s)) r)))) ``` Play a bit: ```clojure (parse-all rule "background: #fafafa; ") ;; {:key "background", :value "#fafafa"} (parse-all ruleset "p { background: #fafafa; color: red; }") ;; {:selector "p", ;; :rules ;; ({:key "background", :value "#fafafa"} {:key "color", :value "red"})} (parse-all ruleset css) ;; {:selector ".container h1", ;; :rules ;; ({:key "color", :value "rgba(255, 0, 0, 0.9)"} ;; {:key "font-size", :value "24px"} ;; {:key "font-family", :value "Monaco"})} ``` Looks nice. ## 6. Notes  - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 81 additions and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -30,7 +30,7 @@ From theory: type Parser a = String -> [(a, String)] ``` So, we're going to represent ```parser``` as simple function that takes input and return seq of possible results. Simplest parsers: ```clojure ;; takes any input and "consume" first char from it @@ -39,10 +39,13 @@ So, we're going to represent ```parser``` as simple function that takes input an (list [(first input) (apply str (rest input))]))) ;; don't accept any input (defn failure [_] '()) (any "clojure-1.7") ;; => ([c lojure-1.7]) ``` We also will use few helpers: ```clojure (defn parse [parser input] @@ -54,8 +57,83 @@ Will use few helpers: ## 2. Monads I'm not going to pay much attention to what ```monads``` are. Everything that you need to know is that we will use it to compose small parsers into bigger one. Also we're not going to implement entire "monad infrastructure". ```clojure ;; build parser that always returns given element without consuming (changing) input (defn return [v] (fn [input] (list [v input]))) ;; take parser and function that builds new parsers from result of applying first one (defn >>= [m f] (fn [input] (mapcat (fn [[v tail]] (parse (f v) tail)) (parse m input)))) ``` Next part is a simple macro that provides haskell-like ```do``` notation. If you don't know how ```do``` block works in Haskell, just skip this code and return to it when necessary. ```clojure (defn skip-arg [f] (fn [_] (f))) (defn merge-bind [body bind] (if (and (= 3 (count bind)) (= '<- (second bind))) `(>>= ~(last bind) (fn [~(first bind)] ~body)) `(>>= ~bind (skip-arg (fn [] ~body))))) (defmacro do* [& forms] (reduce merge-bind (last forms) (reverse (butlast forms)))) ``` ## 3. Basic parsers Let's build basic parsers that can recognize chars, strings, letters, spaces, etc. ```clojure (defn sat [pred] (>>= any (fn [v] (if (pred v) (return v) failure)))) (defn match [c] (sat (partial = (first c)))) (defn noneOf [c] (sat (partial not= (first c)))) (def letter (sat (fn [v] (not (nil? (re-find #"[a-zA-Z]" (str v))))))) ``` ## 4. Combinators Useful combinators that we will use: ```clojure ;; (a|b) (defn or-else [p1 p2] (fn [input] (concat (parse p1 input) (parse p2 input)))) (declare plus) (declare optional) ;; (a*) (defn many [parser] (optional (plus parser))) ;; (a+) (defn plus [parser] (do* (a <- parser) (as <- (many parser)) (return (cons a as)))) ;; (a?) (defn optional [parser] (or-else parser (return ""))) ``` Pay special attention to ```plus``` combinator implementation with using ```do*`` mentioned above. Example of combinators usage: ```clojure (def space (or-else (match " ") (match "\n"))) (def spaces (many space)) ``` ## 5. CSS  - 
        
kachayev revised this gist
Aug 29, 2014 . 1 changed file with 43 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -16,4 +16,46 @@ What we want to get? ({:key "color", :value "rgba(255, 0, 0, 0.9)"} {:key "font-size", :value "24px"} {:key "font-family", :value "Monaco"})} ``` Let's do this with [Monadic parsing](http://eprints.nottingham.ac.uk/223/1/pearl.pdf) approach. Step-by-step explanation of how it works one can find in [Monadic Parsing in Python](http://goo.gl/X7klJ8). What about Clojure? ## 1. Parser abstraction From theory: ```haskell type Parser a = String -> [(a, String)] ``` So, we're going to represent ```parser``` as simple function that takes input and return seq of possible results. Simplest parser: ```clojure ;; takes any input and "consume" first char from it (defn any [input] (if (empty? input) '() (list [(first input) (apply str (rest input))]))) (any "clojure-1.7") ;; => ([c lojure-1.7]) ``` Will use few helpers: ```clojure (defn parse [parser input] (parser input)) (defn parse-all [parser input] (ffirst (filter #(= "" (second %)) (parse parser input)))) ``` ## 2. Monads ## 3. Basic parsers ## 4. Combinators ## 5. CSS  - 
        
kachayev created this gist
Aug 29, 2014 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,19 @@ What we have? ``` .container h1 { color: rgba(255, 0, 0, 0.9); font-size: 24px; font-family: Monaco; } ``` What we want to get? ```clojure {:selector ".container h1", :rules ({:key "color", :value "rgba(255, 0, 0, 0.9)"} {:key "font-size", :value "24px"} {:key "font-family", :value "Monaco"})} ```