Skip to content

Instantly share code, notes, and snippets.

View bartosz25's full-sized avatar

Bartosz Konieczny bartosz25

360 followers · 2 following
View GitHub Profile
@bartosz25
bartosz25 / ByteBufferIOOps.scala
Created November 30, 2019 14:15
selector-nio-example
package com.waitingforcode
import java.nio.ByteBuffer
import java.nio.channels.SocketChannel
import java.nio.charset.StandardCharsets
object ByteBufferIOOps {
def writeToChannel(channel: SocketChannel, textToWrite: String, buffer: ByteBuffer) = {
buffer.clear()
buffer.put(textToWrite.getBytes())
@bartosz25
bartosz25 / docker-compose.yaml
Created July 12, 2019 04:43
StackOverflow https://stackoverflow.com/questions/56985354/why-tasks-are-stuck-in-none-state-in-airflow-1-10-2-after-a-trigger-dag/56987794
version: '3.1'
services:
webserver:
image: puckel/docker-airflow:1.9.0-3
environment:
- LOAD_EX=n
volumes:
- ./assets/dags:/usr/local/airflow/dags
ports:
- "8080:8080"
@bartosz25
bartosz25 / CdcCommitLogReadHandler.java
Created May 28, 2019 12:58
Cassandra CDC idempotent consumer
package com.waitingforcode.cassandra_cdc;
import org.apache.cassandra.config.ColumnDefinition;
import org.apache.cassandra.db.Clustering;
import org.apache.cassandra.db.Mutation;
import org.apache.cassandra.db.commitlog.CommitLogDescriptor;
import org.apache.cassandra.db.commitlog.CommitLogReadHandler;
import org.apache.cassandra.db.partitions.PartitionUpdate;
import org.apache.cassandra.db.rows.Cell;
import org.apache.cassandra.db.rows.Row;
@bartosz25
bartosz25 / Airflow external triggers
Last active August 18, 2024 04:11
Apache Airflow external trigger example
curl -X POST http://localhost:8081/api/experimental/dags/hello_world_a/dag_runs -H 'Cache-Control: no-cache' -H 'Content-Type: application/json' -d '{"conf":"{\"task_payload\":\"payload1\"}"}'
curl -X POST http://localhost:8081/api/experimental/dags/hello_world_a/dag_runs -H 'Cache-Control: no-cache' -H 'Content-Type: application/json' -d '{"conf":"{\"task_payload\":\"payload2\"}"}'
curl -X POST http://localhost:8081/api/experimental/dags/hello_world_a/dag_runs -H 'Cache-Control: no-cache' -H 'Content-Type: application/json' -d '{"conf":"{\"task_payload\":\"payload3\"}"}'
@bartosz25
bartosz25 / ArrayElementsCastExample.scala
Created March 28, 2019 06:15
renaming Array struct columns
case class ElementCaseClass(contrast_ratio: Double, error_text: String, error_x_position: Long, error_x_size: Long)
val dataset = Seq(
(Array(
ElementCaseClass(1.0d, "error1", 1L, 2L),
ElementCaseClass(2.0d, "error2", 100L, 101L),
ElementCaseClass(3.0d, "error3", 1000L, 1001L)
))
).toDF("elements")
@bartosz25
bartosz25 / CassandraSession.java
Created October 9, 2018 11:50
Apache Cassandra wide rows vs skinny rows micro-benchmark
package com.waitingforcode;
import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.Session;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;