akhitev

Problem Statement

A service layer (Java API in this case) is backed by dozens of services (over the network in an SOA) and used by dozens of clients, each using different subsets of the overall API. In our case all client interaction happens within a single JVM so we can know which clients are accessing what services, fields and keys (versus a RESTful API over HTTP where this would be unknown).

Principally the API is functionality focused rather than system focused. A client should request data to serve a given use case and not need to concern themselves with the underlying implementation details of what backend system the data is coming from.

Today this is achieved through static typing and Java object modeling. Types from underlying services are decoupled from clients so that backend systems can change their implementations without impacting clients.

This means (assuming Java language environment):

Introduction to Kafka

Kafka acts as a kind of write-ahead log (WAL) that records messages to a persistent store (disk) and allows subscribers to read and apply these changes to their own stores in a system appropriate time-frame.

Terminology:

Producers send messages to brokers
Consumers read messages from brokers
Messages are sent to a topic

	import java.util.ArrayList;
	import java.util.Iterator;
	import java.util.List;
	import java.util.concurrent.Callable;
	import java.util.concurrent.ExecutorService;
	import java.util.concurrent.Future;
	import java.util.concurrent.LinkedBlockingQueue;
	import java.util.concurrent.ThreadPoolExecutor;
	import java.util.concurrent.TimeUnit;

	# config/initializers/will_paginate.rb

	module WillPaginate
	module ActionView
	def will_paginate(collection = nil, options = {})
	options[:renderer] \|\|= BootstrapLinkRenderer
	super.try :html_safe
	end

	class BootstrapLinkRenderer < LinkRenderer

	// includes bindings for fetching/fetched

	PaginatedCollection = Backbone.Collection.extend({
	initialize: function() {
	_.bindAll(this, 'parse', 'url', 'pageInfo', 'nextPage', 'previousPage');
	this.page = 1;
	},
	fetch: function(options) {
	options \|\| (options = {});
	this.trigger("fetching");