FAQ Database Discussion Community

Apache Kafka is JMS api?

Apache Kafka is not a JMS implementation? I using JMS+AMQ in my application and emigration to Apache Kafka, i have to change all JMS codes?...

kafka log.retention.hours inconsistency in multiple brokers

I am trying to run multiple kafka brokers. There is a file named server.properties and there is a field "log.retention.hours" which is set to be 168 in my server.properties file. Now if i change the kafka brokers properties file and set different values of "log.retention.hours" in each properties file then...

What exactly IS Kafka Rebalancing?

I am a new user to Kafka and have been playing around with it for about 2-3 weeks now. I believe at the moment I have a good understand of how Kafka works for the most part, but after attempting to play around with and design my own Kafka consumer...

how to get kafka to consume http streaming data on spring xd runtime?

Is it possible to have kafka source module work as a processor module in spring xd runtime? Any code samples? I am trying to achieve something like this: http (xd source) | kafka source (xd processor)| kafka consumer (xd sink) I am trying to do this because I have streaming...

Kafka QuickStart, advertised.host.name gives kafka.common.LeaderNotAvailableException

I am able to get a simple one-node Kafka (kafka_2.11- working locally on one linux machine, but when I try to run a producer remotely I'm getting some confusing errors. I'm following the quickstart guide at http://kafka.apache.org/documentation.html#quickstart. I stopped the kafka processes and deleted all the zookeeper & karma files...

Kafka partitions meaning

When we decide about partitions, should we do that on per-topic base, or it is topic-wide decision? If T1 partitioned on 3 partitions, and T2 partitioned on 2 partitions, can they both be consumed by 1 consumer? Or it is better make equal number of partitions if topics must be...

scala + kafka does not send message with id

I'm trying to send a message to Kafka through my actor and it does not work. The following code works new KeyedMessage[String, Array[Byte]]("my-topic", msg.message) This one doesn't... why? new KeyedMessage[String, Array[Byte]]("my-topic", msg.id, msg.message) or even new KeyedMessage[String, Array[Byte]]("my-topic", msg.id, null, msg.message) setting partition to null, forcing it to fill only...

Error while deploying topology on storm cluster

I am trying to deploy a simple word count topology on storm clustre.I am using kafka as the input(kafka Spout).This is the error i am getting.I am very new to storm.Please suggest the changes.Thanks in Advance !! java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.Log4jLoggerFactory at org.apache.log4j.Logger.getLogger(Logger.java:39) at kafka.utils.Logging$class.logger(Logging.scala:24) at kafka.consumer.SimpleConsumer.logger$lzycompute(SimpleConsumer.scala:30) at...

Should Apache Kafka and Hadoop be installed seperatedly (on a diffrent cluster)?

Should Apache Kafka and Hadoop be installed seperatedly (on a diffrent cluster) ?

Can I achieve ordered processing with multiple consumers in Kafka?

In Kafka, I have a producer queuing up work of clients. Each piece of work has a client ID on it. Work of different clients can be processed out of order, but work of one client must be processed in order. To do this, I intend to have (for example)...

Fail to export from voltdb to kafka

VoltDB is enterprise 5.1.2 Kafka is 2.9.2-, also tried 2.10- VoltDB is at Kafka is at Here is my deployment configuration for VoltDB: <deployment> <cluster hostcount="1" sitesperhost="4" kfactor="0" /> <commandlog enabled="true"> <frequency time="400" transactions="1000" /> </commandlog> <export> <configuration enabled="true" type="kafka" stream="archive"> <property name="metadata.broker.list"></property> <property...

Wildfly Kafka clients module javax/management/JMException not found

I'm try to create Kafka clients module for Wildfly. Here is module.xml: <?xml version="1.0" encoding="UTF-8"?> <module xmlns="urn:jboss:module:1.0" name="org.apache.kafka.clients"> <resources> <resource-root path="kafka-clients-"/> <resource-root path="lz4-1.2.0.jar"/> <resource-root path="snappy-java-"/> </resources> <dependencies> <module name="org.slf4j"/> </dependencies> </module> Wildfly (8.2.0-Final) gives this error on boot:...