Apache kafka setup

 

To run Apache Kafka from the binary distribution, you need to follow a series of steps. Below are the general steps for running Kafka on a Unix-like system (Linux or macOS). Make sure you have Java installed on your machine before proceeding.

Prerequisites:

1. Java Installation: Ensure you have Java installed on your system. You can check your Java version using:

   bash

java -version

If Java is not installed, you can install it by following the instructions for your operating system.

Download and Extract Kafka: Download the Kafka binary distribution from the official Apache Kafka website. Once downloaded, extract the contents of the tarball:

bash

2. tar -xzf kafka_2.12-3.7.0.tgz
   cd kafka_2.12-3.7.0
   

Start ZooKeeper:

Kafka uses ZooKeeper, so you need to start it before starting Kafka.

bash

bin/zookeeper-server-start.sh config/zookeeper.properties

Start Kafka Server:

In a new terminal, start the Kafka server.

bash

bin/kafka-server-start.sh config/server.properties

Create a Kafka Topic:

In another terminal, you can create a Kafka topic to test your setup.

bash

bin/kafka-topics.sh --create --topic mytopic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

Produce and Consume Messages:

You can now use the Kafka producer and consumer to test your setup.

Producer:

bash

bin/kafka-console-producer.sh --topic mytopic --bootstrap-server localhost:9092

Consumer:

bash

bin/kafka-console-consumer.sh --topic mytopic --bootstrap-server localhost:9092 --from-beginning

Type messages in the producer terminal, and you should see them being consumed in the consumer terminal.

These are basic steps to run Kafka locally for testing and development. In a production environment, you may need to configure Kafka appropriately for your specific use case and environment. Additionally, consider creating systemd service files or using other deployment tools based on your needs.