Skip to content

Instantly share code, notes, and snippets.

@alexanderdean

alexanderdean/pipeline.scala

Last active August 29, 2015 14:10
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save alexanderdean/1c2dfad1328093b08855 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save alexanderdean/1c2dfad1328093b08855 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. alexanderdean revised this gist Dec 7, 2014. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion pipeline.scala
    View file
    Original file line number Diff line number Diff line change
    @@ -3,4 +3,4 @@ private val PayloadDataSchema =
    SchemaCriterion("com.snowplowanalytics.snowplow", "payload_data", "jsonschema", 1, 0)

    // Check JSON is a payload_data version 1-0-*, and verify it against the schema
    val node: ValidatedNel[JsonNode] = rawNode.verifySchemaAndValidate(schemaCriterion)
    val body: ValidatedNel[JsonNode] = bodyNode.verifySchemaAndValidate(schemaCriterion)
  2. alexanderdean revised this gist Dec 7, 2014. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion pipeline.scala
    View file
    Original file line number Diff line number Diff line change
    @@ -2,5 +2,5 @@
    private val PayloadDataSchema =
    SchemaCriterion("com.snowplowanalytics.snowplow", "payload_data", "jsonschema", 1, 0)

    // Check JSON is a payload_data, and validate it against that schema version 1-0-*
    // Check JSON is a payload_data version 1-0-*, and verify it against the schema
    val node: ValidatedNel[JsonNode] = rawNode.verifySchemaAndValidate(schemaCriterion)
  3. alexanderdean revised this gist Dec 7, 2014. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion pipeline.scala
    View file
    Original file line number Diff line number Diff line change
    @@ -2,5 +2,5 @@
    private val PayloadDataSchema =
    SchemaCriterion("com.snowplowanalytics.snowplow", "payload_data", "jsonschema", 1, 0)

    // Check JSON is a payload_data, and validate it against that schema
    // Check JSON is a payload_data, and validate it against that schema version 1-0-*
    val node: ValidatedNel[JsonNode] = rawNode.verifySchemaAndValidate(schemaCriterion)
  4. alexanderdean revised this gist Dec 7, 2014. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions pipeline.scala
    View file
    Original file line number Diff line number Diff line change
    @@ -2,4 +2,5 @@
    private val PayloadDataSchema =
    SchemaCriterion("com.snowplowanalytics.snowplow", "payload_data", "jsonschema", 1, 0)

    // Check JSON is a payload_data, and validate it against that schema
    val node: ValidatedNel[JsonNode] = rawNode.verifySchemaAndValidate(schemaCriterion)
  5. alexanderdean revised this gist Dec 7, 2014. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion pipeline.scala
    View file
    Original file line number Diff line number Diff line change
    @@ -2,4 +2,4 @@
    private val PayloadDataSchema =
    SchemaCriterion("com.snowplowanalytics.snowplow", "payload_data", "jsonschema", 1, 0)

    val node: ValidatedNel[JsonNode] = extractAndValidateJson(PayloadDataSchema, rawNode)
    val node: ValidatedNel[JsonNode] = rawNode.verifySchemaAndValidate(schemaCriterion)
  6. alexanderdean revised this gist Dec 7, 2014. 1 changed file with 4 additions and 5 deletions.
    9 changes: 4 additions & 5 deletions pipeline.scala
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,5 @@
    type Validated[A] = ValidationNel[String, A]
    // Request body expected to validate against this JSON Schema
    private val PayloadDataSchema =
    SchemaCriterion("com.snowplowanalytics.snowplow", "payload_data", "jsonschema", 1, 0)

    def processEvents(registry: EnrichmentRegistry, etlVersion: String,
    etlTstamp: String, input: Validated[Option[CollectorPayload]])
    (implicit resolver: iglu.client.Resolver)
    : List[Validated[EnrichedEvent]] = {
    val node: ValidatedNel[JsonNode] = extractAndValidateJson(PayloadDataSchema, rawNode)
  7. alexanderdean created this gist Dec 7, 2014.
    6 changes: 6 additions & 0 deletions pipeline.scala
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,6 @@
    type Validated[A] = ValidationNel[String, A]

    def processEvents(registry: EnrichmentRegistry, etlVersion: String,
    etlTstamp: String, input: Validated[Option[CollectorPayload]])
    (implicit resolver: iglu.client.Resolver)
    : List[Validated[EnrichedEvent]] = {