FAQ Database Discussion Community


Wildfly Kafka clients module javax/management/JMException not found

java-ee,jboss,wildfly,kafka
I'm try to create Kafka clients module for Wildfly. Here is module.xml: <?xml version="1.0" encoding="UTF-8"?> <module xmlns="urn:jboss:module:1.0" name="org.apache.kafka.clients"> <resources> <resource-root path="kafka-clients-0.8.2.0.jar"/> <resource-root path="lz4-1.2.0.jar"/> <resource-root path="snappy-java-1.1.1.6.jar"/> </resources> <dependencies> <module name="org.slf4j"/> </dependencies> </module> Wildfly (8.2.0-Final) gives this error on boot:...

scala + kafka does not send message with id

scala,akka,apache-kafka,kafka
I'm trying to send a message to Kafka through my actor and it does not work. The following code works new KeyedMessage[String, Array[Byte]]("my-topic", msg.message) This one doesn't... why? new KeyedMessage[String, Array[Byte]]("my-topic", msg.id, msg.message) or even new KeyedMessage[String, Array[Byte]]("my-topic", msg.id, null, msg.message) setting partition to null, forcing it to fill only...

Kafka partitions meaning

configuration,partitioning,partition,kafka
When we decide about partitions, should we do that on per-topic base, or it is topic-wide decision? If T1 partitioned on 3 partitions, and T2 partitioned on 2 partitions, can they both be consumed by 1 consumer? Or it is better make equal number of partitions if topics must be...

Can I achieve ordered processing with multiple consumers in Kafka?

design,message-queue,kafka
In Kafka, I have a producer queuing up work of clients. Each piece of work has a client ID on it. Work of different clients can be processed out of order, but work of one client must be processed in order. To do this, I intend to have (for example)...

kafka log.retention.hours inconsistency in multiple brokers

apache-kafka,distributed-system,kafka
I am trying to run multiple kafka brokers. There is a file named server.properties and there is a field "log.retention.hours" which is set to be 168 in my server.properties file. Now if i change the kafka brokers properties file and set different values of "log.retention.hours" in each properties file then...

Kafka QuickStart, advertised.host.name gives kafka.common.LeaderNotAvailableException

apache-kafka,kafka
I am able to get a simple one-node Kafka (kafka_2.11-0.8.2.1) working locally on one linux machine, but when I try to run a producer remotely I'm getting some confusing errors. I'm following the quickstart guide at http://kafka.apache.org/documentation.html#quickstart. I stopped the kafka processes and deleted all the zookeeper & karma files...

Apache Kafka is JMS api?

jms,kafka
Apache Kafka is not a JMS implementation? I using JMS+AMQ in my application and emigration to Apache Kafka, i have to change all JMS codes?...

how to get kafka to consume http streaming data on spring xd runtime?

spring,spring-xd,http-streaming,kafka
Is it possible to have kafka source module work as a processor module in spring xd runtime? Any code samples? I am trying to achieve something like this: http (xd source) | kafka source (xd processor)| kafka consumer (xd sink) I am trying to do this because I have streaming...

Should Apache Kafka and Hadoop be installed seperatedly (on a diffrent cluster)?

hadoop,apache-kafka,kafka
Should Apache Kafka and Hadoop be installed seperatedly (on a diffrent cluster) ?

Error while deploying topology on storm cluster

java,logging,storm,apache-kafka,kafka
I am trying to deploy a simple word count topology on storm clustre.I am using kafka as the input(kafka Spout).This is the error i am getting.I am very new to storm.Please suggest the changes.Thanks in Advance !! java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.Log4jLoggerFactory at org.apache.log4j.Logger.getLogger(Logger.java:39) at kafka.utils.Logging$class.logger(Logging.scala:24) at kafka.consumer.SimpleConsumer.logger$lzycompute(SimpleConsumer.scala:30) at...

Fail to export from voltdb to kafka

export,voltdb,kafka
VoltDB is enterprise 5.1.2 Kafka is 2.9.2-0.8.1.1, also tried 2.10-0.8.2.1 VoltDB is at 192.168.56.101 Kafka is at 192.168.56.102 Here is my deployment configuration for VoltDB: <deployment> <cluster hostcount="1" sitesperhost="4" kfactor="0" /> <commandlog enabled="true"> <frequency time="400" transactions="1000" /> </commandlog> <export> <configuration enabled="true" type="kafka" stream="archive"> <property name="metadata.broker.list">192.168.56.102:9092</property> <property...

What exactly IS Kafka Rebalancing?

apache,apache-kafka,kafka-consumer-api,kafka
I am a new user to Kafka and have been playing around with it for about 2-3 weeks now. I believe at the moment I have a good understand of how Kafka works for the most part, but after attempting to play around with and design my own Kafka consumer...