Sushim Mukul Dutta void666

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

List the topics to which the group is subscribed

kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

Reset the consumer offset for a topic (preview)


	class Crawler:
	def __init__(self, base_url, writer, predicate):
	"""
	* base_url is the first URL that would be crawled.
	* writer is a class that is defined below. It is in-charge of writing
	to the file system.
	* predicate is a function with signature predicate(url). It
	returns true, false and on the basis of that, the decision on
	whether to download a link or not is taken. """