Roberto Fernandez ctwoprod

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

Moved to git repository: https://github.com/denji/nginx-tuning

NGINX Tuning For Best Performance

For this configuration you can use web server you like, i decided, because i work mostly with it to use nginx.

Generally, properly configured nginx can handle up to 400K to 500K requests per second (clustered), most what i saw is 50K to 80K (non-clustered) requests per second and 30% CPU load, course, this was 2 x Intel Xeon with HyperThreading enabled, but it can work without problem on slower machines.

You must understand that this config is used in testing environment and not in production so you will need to find a way to implement most of those features best possible for your servers.

Squashing Git Commits

The easy and flexible way

This method avoids merge conflicts if you have periodically pulled master into your branch. It also gives you the opportunity to squash into more than 1 commit, or to re-arrange your code into completely different commits (e.g. if you ended up working on three different features but the commits were not consecutive).

Note: You cannot use this method if you intend to open a pull request to merge your feature branch. This method requires committing directly to master.

Switch to the master branch and make sure you are up to date:

Python3 Virtualenv Setup

Requirements

Python 3
Pip 3

$ brew install python3

	Cloud VM: Vercell, Heroku
	DB: https://www.db-fiddle.com/
	Dashboard data / visualization: https://app.flourish.studio/
	Java: https://dev.java/playground/
	Nodejs / JS: https://stackblitz.com/edit/node-99aw1w?file=index.js https://playcode.io/new
	Python: https://python-playground.netlify.app
	Go: https://go.dev/play/
	Custom API Mock (json, WS etc.): https://dummyjson.com
	Kafka: https://gauravsalvi.com/projects/kafka-online-test/
	Serverless data platform: https://upstash.com/ (ex. redis)

	pipeline {
	// run on jenkins nodes tha has java 8 label
	agent { label 'java8' }
	// global env variables
	environment {
	EMAIL_RECIPIENTS = '[email protected]'
	}
	stages {

	stage('Build with unit testing') {

	#!/usr/bin/python

	import boto3

	# Create SQS client
	sqs = boto3.client('sqs',region_name='ap-southeast-1')

	queue_url = 'https://sqs.ap-southeast-1.amazonaws.com/862498237182/test-paywalletuser-sqs'

	# Send message to SQS queue

	======
	Videos
	======

	DevOps

	What is DevOps? by Rackspace - Really great introduction to DevOps
	https://www.youtube.com/watch?v=_I94-tJlovg

	Sanjeev Sharma series on DevOps (great repetition to really get the DevOps concept)

	## Install Jenkins

	$ sudo yum update
	$ sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
	$ sudo rpm --import http://pkg.jenkins-ci.org/redhat-stable/jenkins-ci.org.key
	$ sudo yum install jenkins
	$ sudo service jenkins start
	$ sudo chkconfig jenkins on

	## Install Docker (Amazon EC2 Container Service)

	input {
	jdbc {
	jdbc_connection_string => "jdbc:mysql://localhost:3306/sample"
	jdbc_user => "root"
	jdbc_password => "adminmysql"
	# The path to our downloaded jdbc driver
	jdbc_driver_library => "/Users/robertofernandez/libs/mysql-connector-java-5.1.25.jar"
	jdbc_driver_class => "com.mysql.jdbc.Driver"
	# our query
	schedule => "/2 * * *"