It worked well, Admin server need to be updated with new topic name else it won;t work

Getting Error while running command : INFO SASL config status: Will not attempt to authenticate using SASL (unknown error). sudo docker run -p 9092:9092 \ -e KAFKA_ZOOKEEPER_CONNECT=192.168.43.70:2181 \ -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.43.70:9092 \ -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \ confluentinc/cp-kafka Please help me on this

Remove \ and use the complete command in one line

docker run -p 9092:9092 -e KAFKA_ZOOKEEPER_CONNECT=192.168.43.70:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.43.70:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 confluentinc/cp-kafka

This worked for me. Thanks 👍 😎

If anyone need docker compose file

version: "3"

services:
    zookeeper:
        image: zookeeper
        container_name: zookeeper
        ports:
            - "2181:2181"

    kafka:
        image: confluentinc/cp-kafka
        depends_on:
            - zookeeper
        ports:
            - "9092:9092"
        expose:
            - "29092"
        environment:
            KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
            KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
            KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
            KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: "1"
            KAFKA_MIN_INSYNC_REPLICAS: "1"

    kafka-ui:
        container_name: kafka-ui
        image: provectuslabs/kafka-ui
        ports:
            - 8080:8080
        environment:
            DYNAMIC_CONFIG_ENABLED: true

Also if you are getting error that port received NaN then type URL like this localhost:9092 instead of http://localhost:9092

Edit: add kafka UI

@bhumit070 good work for creating decompose file

Unable to find image ':2181' locally i got this error

I guess you forgot to use zookeeper at the end of it?
This is full line: docker run -p 2181:2181 zookeeper

@bhumit070

If anyone need docker compose file

version: '3'

services:
  zookeeper:
    image: zookeeper
    ports:
      - "2181:2181"

  kafka:
    image: confluentinc/cp-kafka
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092

Also if you are getting error that port received NaN then type URL like this localhost:9092 instead of http://localhost:9092

This is working like a charm, Thanks bro 😊

docker run -p 9092:9092 -e KAFKA_ZOOKEEPER_CONNECT=192.168.43.70:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.43.70:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 confluentinc/cp-kafka

I tried your solution still getting error, below is what I used:

sudo docker run -p 9092:9092 -e KAFKA_ZOOKEEPER_CONNECT=103.171.98.61:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://103.171.98.61:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 confluentinc/cp-kafka

I'm having linux mint, and used curl ifconfig to get the ip.

Error:

java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289)
[2024-03-20 13:32:49,380] INFO Opening socket connection to server 103.171.98.61/103.171.98.61:2181. (org.apache.zookeeper.ClientCnxn)
[2024-03-20 13:32:49,381] INFO SASL config status: Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2024-03-20 13:32:49,387] WARN Session 0x0 for server 103.171.98.61/103.171.98.61:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)

I am not able to generate and see SOUTH location. Its showing only NORTH

I also had this same issue, what worked for me is

* in admin.js file, update the topic name

* run admin.js again

* run producer and consumer again it worked for me

It worked for me as well

It worked well, Admin server need to be updated with new topic name else it won;t work

Does anyone knows the reason?

I need to run the consumer and producer on different servers. Can someone let me know what changes I should implement to set that up?

I need to run the consumer and producer on different servers. Can someone let me know what changes I should implement to set that up?

Yes. Simply run producer file onto a different server you will have to copy client.js also on to another server

I need to run the consumer and producer on different servers. Can someone let me know what changes I should implement to set that up?

Yes. Simply run producer file onto a different server you will have to copy client.js also on to another server

Thank you for replying but I suspect I will have issues with the docker files and with establishing a connection from the producer to the consumer. Should I be worried about that?

I need to run the consumer and producer on different servers. Can someone let me know what changes I should implement to set that up?

Yes. Simply run producer file onto a different server you will have to copy client.js also on to another server

Thank you for replying but I suspect I will have issues with the docker files and with establishing a connection from the producer to the consumer. Should I be worried about that?

No. I don't think so. Try it and share the git repo I will check and try to help.

I am getting rebalancing error for two consumers. Do someone know how to solve this.

I am getting rebalancing error for two consumers. Do someone know how to solve this.

Please share more details with screenshot

https://github.com/A1996KASH/apache-kafka-nodejs
It worked fine for me

south location is not visible even if i change my topic name in admin.js

I am not able to generate and see SOUTH location. Its showing only NORTH

I also had this same issue, what worked for me is

• in admin.js file, update the topic name
• run admin.js again
• run producer and consumer again it worked for me

its not working for me bro

change the topic to something else like rider-status, I also had the same issue. i think it will keep the whatever code we deployed first so just change the topic. I hope it will work as intended.

this works .....but why......

should we change this in producer.js and consumer.js as well? or only in admin?

I am not able to generate and see SOUTH location. Its showing only NORTH

I also had this same issue, what worked for me is

• in admin.js file, update the topic name
• run admin.js again
• run producer and consumer again it worked for me

Working now! Thanks, Why do you think this was the case?

I am not able to generate and see SOUTH location. Its showing only NORTH

I also had this same issue, what worked for me is

• in admin.js file, update the topic name
• run admin.js again
• run producer and consumer again it worked for me

Working now! Thanks, Why do you think this was the case?

In general, the topic name should remain the same throughout Kafka

1. At the moment of creation
2. When the producer is being produced
3. At the moment of consuming

@Axhat I hope this helps

Instead of <PRIVATE_IP> i want to use localhost, but its not working,(changed in kafka container and client.js)
is there any solution for this?

Instead of <PRIVATE_IP> i want to use localhost, but its not working,(changed in kafka container and client.js) is there any solution for this?

When running Kafka in Docker, you can't use localhost or 127.0.0.1 as the value for KAFKA_ZOOKEEPER_CONNECT and KAFKA_ADVERTISED_LISTENERS. This is because localhost and 127.0.0.1 refer to the container's own loopback interface, not the host machine's IP address. So you have to use your own IPv4 address

Instead of <PRIVATE_IP> i want to use localhost, but its not working,(changed in kafka container and client.js) is there any solution for this?

When running Kafka in Docker, you can't use localhost or 127.0.0.1 as the value for KAFKA_ZOOKEEPER_CONNECT and KAFKA_ADVERTISED_LISTENERS. This is because localhost and 127.0.0.1 refer to the container's own loopback interface, not the host machine's IP address. So you have to use your own IPv4 address.

for this you can use -p and expose your machine 's ports

Thanks this is the best tutorial for Kafka and works fine.

Here is the same kafka implementation using typescript : https://gist.github.com/ssk090/852139926f7157ff61baccd40e900d0c

I am not able to generate and see SOUTH location. Its showing only NORTH

• change the topic from admin.js consumer.js and producer.js
• run admin.js (node admin.js)
  ** Then the partitioner will be form and assign as it should be **

please make a video on the implementation of long polling and push

Paid courses for them, No one will teach everything for free.

[cause]: KafkaJSConnectionError: Connection timeout
at Timeout.onTimeout [as _onTimeout] (C:\Users\Dell\Desktop\kafka-app\node_modules\kafkajs\src\network\connection.js:223:23)
at listOnTimeout (node:internal/timers:594:17)
at process.processTimers (node:internal/timers:529:7) {
retriable: true,
helpUrl: undefined,
broker: '192.168.0.204:9092',
code: undefined,
[cause]: undefined

while running any js file this same error is showing. kindly help

[cause]: KafkaJSConnectionError: Connection timeout at Timeout.onTimeout [as _onTimeout] (C:\Users\Dell\Desktop\kafka-app\node_modules\kafkajs\src\network\connection.js:223:23) at listOnTimeout (node:internal/timers:594:17) at process.processTimers (node:internal/timers:529:7) { retriable: true, helpUrl: undefined, broker: '192.168.0.204:9092', code: undefined, [cause]: undefined

while running any js file this same error is showing. kindly help

@ANKIT3412 Seems like you don't have the correct image for running the kafka in docker , i also faced the same issue and with the new image from bitnami it was resolved

If anyone need docker compose file

version: "3"

services:
    zookeeper:
        image: zookeeper
        container_name: zookeeper
        ports:
            - "2181:2181"

    kafka:
        image: confluentinc/cp-kafka
        depends_on:
            - zookeeper
        ports:
            - "9092:9092"
        expose:
            - "29092"
        environment:
            KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
            KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
            KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
            KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: "1"
            KAFKA_MIN_INSYNC_REPLICAS: "1"

    kafka-ui:
        container_name: kafka-ui
        image: provectuslabs/kafka-ui
        ports:
            - 8080:8080
        environment:
            DYNAMIC_CONFIG_ENABLED: true

Also if you are getting error that port received NaN then type URL like this localhost:9092 instead of http://localhost:9092

Edit: add kafka UI

thanks buddy

Here is the updated docker-compose.yaml file which works for me

version: '3.8'

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.5.0
    container_name: zookeeper
    restart: unless-stopped
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  kafka:
    image: confluentinc/cp-kafka:7.5.0
    container_name: kafka
    restart: unless-stopped
    ports:
      - "9092:9092"
      - "29092:29092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:29092,PLAINTEXT_HOST://0.0.0.0:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_MIN_INSYNC_REPLICAS: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
    depends_on:
      - zookeeper