Windows平台下kafka环境的搭建以及简单使用

一、Kafka配置： 

参考网址：

http://m.blog.csdn.net/ydc321/article/details/70154278

http://www.2cto.com/net/201701/588235.html

http://www.jianshu.com/p/f7037105db46

http://www.jianshu.com/p/64d25dcf8300

https://my.oschina.net/phoebus789/blog/733670

http://orchome.com/kafka/index

 

准备工作：

1.安装jdk环境

http://www.oracle.com/technetwork/java/javase/downloads/index.html 

 

2.下载kafka的程序安装包，并解压

http://kafka.apache.org/downloads 

 

3.用命令行测试kafka生产和消费

在 kafka 目录，按住shift+鼠标右键->在此处打开命令窗口(W) 

第一个命令窗口->启动zookeeper服务： 

bin\windows\zookeeper-server-start.bat config\zookeeper.properties

 

第二个命令窗口->启动kfaka服务： 

bin\windows\kafka-server-start.bat config\server.properties 

 

第三个命令窗口->启动启produce: 

• 创建一个主题：

bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testtopic

 

• 使用如下命令查看创建的主题列表：

bin\windows\kafka-topics.bat --list --zookeeper localhost:2181

 

• 启动生产者：

bin\windows\kafka-console-producer.bat --broker-list {本机ip}:9092 --topic testtopic

 

第四个命令窗口->启动consumer：

bin\windows\kafka-console-consumer.bat --zookeeper localhost:2181 --topic testtopic --from-beginning 

 

二、测试kafka生产和消费

• 创建两个控制台项目，一个是生产者，一个是消费者

 

//生产者代码
 1 using Confluent.Kafka;
using Confluent.Kafka.Serialization;
using System;
using System.Collections.Generic;
using System.Text;

namespace KafkaProducter
{
    public class Program
    {
        public static void Main(string[] args)
        {
            string brokerList = "127.0.0.1:9092";
            string topicName = "testtopic";

            var config = new Dictionary<string, object> { { "bootstrap.servers", brokerList } };

            using (var producer = new Producer<Null, string>(config, null, new StringSerializer(Encoding.UTF8)))
            {
                Console.WriteLine($"{producer.Name} producing on {topicName}. q to exit.");

                string text;
                while ((text = Console.ReadLine()) != "q")
                {
                    var deliveryReport = producer.ProduceAsync(topicName, null, text);
                    deliveryReport.ContinueWith(task =>
                    {
                        Console.WriteLine($"Partition: {task.Result.Partition}, Offset: {task.Result.Offset}");
                    });
                }

                // Tasks are not waited on synchronously (ContinueWith is not synchronous),
                // so it's possible they may still in progress here.
                producer.Flush(TimeSpan.FromSeconds(10));
            }
        }
    }
}

 

 

//消费者代码
 1 using Confluent.Kafka;
using Confluent.Kafka.Serialization;
using System;
using System.Collections.Generic;
using System.Text;

namespace KafkaConsumer
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var config = new Dictionary<string, object>
            {
                { "group.id", "simple-csharp-consumer" },
                { "bootstrap.servers", "127.0.0.1:9092" }
            };

            using (var consumer = new Consumer<Null, string>(config, null, new StringDeserializer(Encoding.UTF8)))
            {
                consumer.Assign(new List<TopicPartitionOffset> { new TopicPartitionOffset("testtopic", 0, 0) });

                while (true)
                {
                    Message<Null, string> msg;
                    if (consumer.Consume(out msg, TimeSpan.FromSeconds(1)))
                    {
                        Console.WriteLine($"Topic: {msg.Topic} Partition: {msg.Partition} Offset: {msg.Offset} {msg.Value}");
                    }
                }
            }
        }
    }
}