FAQ Database Discussion Community


Can I use lpop/rpop to create a simple queue system with Redis?

redis,queue,message-queue
I tried several message/job queue systems but they all seem to add unnecessary complexity and I always end up with the queue process dying for no reason and cryptic log messages. So now I want to make my own queue system using Redis. How would you go about doing this?...

Is it a good idea to use a message queue as websocket server in production (instead of Java EE @ServerEndpoint)? [closed]

websocket,message-queue
Is it a good idea to use a message queue as websocket server in production (instead of Java EE @ServerEndpoint)? It would establish a loose coupling towards the JS client and perhaps reduce the load from the application, but are there major disadvantages? Should one really expose a MQ over...

Is message queueing mandatory for Azure web app when sending emails through SendGrid

asp.net,azure,message-queue,sendgrid
I have an Azure web application (Asp.net MVC & WebAPI) that sends emails through SendGrid service. I'm not using SendGrid's API but rather use .net built-in SMTP that I configured in web.config and directed to SendGrid. I'm now wondering whether I also need message queueing application in my solution that...

C# MSMQ Invalid Operation Exception

c#,message-queue,msmq,invalidoperationexception
I create a basic application to manage MSMQ in C#. I create messages in my trasaction message queue with this method: /// <summary> /// Method to send message in message queue /// </summary> /// <param name="mq">message queue (to)</param> /// <param name="messageBody">message body</param> /// <param name="messageLabel">message label</param> public static void sendMessage(MessageQueue...

Multiprocessing Queue Negative Max Size

python,message-queue,python-multiprocessing
There are some examples of code that specify a negative number for the maxsize of a multiprocessing.Queue like so import multiprocessing queue = multiprocessing.Queue(-1) I could not find anything in the documentation that specifies what a negative value means so I looked at the multiprocessing.Queue source code. class Queue(object): def...

Can netcat block on sending side?

bash,message-queue,netcat
After failing to produce a robust messaging system using named pipes I'm trying to use netcat. My receiving side will be a small script that listens to a port using netcat, spawns a new listener and execute a script based on the received message: # jh_listen.sh message=$(netcat -l 5555) nohup...

User request queue in WCF

c#,wcf,message-queue,wcfserviceclient,request-queueing
I have a WCF service which creates a number of files at a server location doing various calculation on seed file depending upon the params given. The problem is that, the when 2 or more clients try to do calculation on same seed file, it is returning error. The cause...

Again on error C2248

c++,boost,compiler-errors,message-queue,boost-interprocess
the error C2248 is not new on stackoverflow. Unfortunately I'm a beginner in using Boost library and I'm not able to fix the error in my code: // .h file using namespace boost::interprocess; using namespace std; class CMsqQueueMngr { public: // constructors & destructors CMsqQueueMngr(); ~CMsqQueueMngr(); int Open(char *queueName, int...

How to Bulk Process Messages from Service Bus Queue Every X Seconds

c#,azure,message-queue,servicebus,azure-servicebus-queues
My service bus queue previously processed one message at a time. I wanted to change it to allow the messages to queue up so I could process them in bulk every X seconds. I initially did this: var messagingFactorySettings = new MessagingFactorySettings { NetMessagingTransportSettings = { BatchFlushInterval = TimeSpan.FromMilliseconds(10000) },...

error installing zmq (ZeroMQ)

node.js,npm,message-queue,zeromq,pyzmq
I am trying to install node module zmq I have referred to this https://github.com/JustinTulloss/zeromq.node/wiki/Installation#installation-on-windows However i get this error while installing the module 0 info it worked if it ends with ok 1 verbose cli [ 'C:\\Program Files (x86)\\nodejs\\\\node.exe', 1 verbose cli 'C:\\Program Files (x86)\\nodejs\\node_modules\\npm\\bin\\npm-cli.js', 1 verbose cli 'install', 1...

Using flume to read IBM MQ data

hadoop,streaming,message-queue,flume
I want to read data from IBM MQ and put it into HDFs. Looked into JMS source of flume, seems it can connect to IBM MQ, but I’m not understanding what does “destinationType” and “destinationName” mean in the list of required properties. Can someone please explain? Also, how I should...

Can I achieve ordered processing with multiple consumers in Kafka?

design,message-queue,kafka
In Kafka, I have a producer queuing up work of clients. Each piece of work has a client ID on it. Work of different clients can be processed out of order, but work of one client must be processed in order. To do this, I intend to have (for example)...

Message queuing protocols: PGM vs UDP

udp,message-queue,multicast
I need to have an intelligent conversation with several co-workers who have not yet been sold on embracing tried and trusted message queuing libraries such as MSMQ, ZeroMQ, RabbitMQ, etc and we need this type of messaging in our solution. But to be able to sell to my colleagues the...

How to use MessageQueue in Crawler?

architecture,web-crawler,message-queue
It seems that MessageQueue should be a good architectural solution for building Web Crawler, but still I can't understand how to do it. Let's consider the first case with shared database, it is pretty clear how to do it the algorithm would be the classical Graph Traversal: There are multiple...

RabbitMQ: How to prevent QueueDeclare to automatically generate a new Queue

c#,rabbitmq,message-queue
With RabbitMQ I am doing something similar to this: channel.QueueDeclare(QueueName, true, false, false, null); By default RabbitMQ creates a new queue if none of the existing matches the name provided. I would like to have an exception thrown instead. Is that possible? Thanks...

RabbitMQ: server side Calculation for every received Message

rabbitmq,message-queue,spring-amqp,spring-rabbit
I'm working with RabbitMQ and I want on the server side to conduct a calculation each time an Exchange receives a message. I have a queue for ratings and when too many bad reviews (let's say more than ten) received, then a consumer should be notified. What options are there...

what are the use cases for persistant, in-order message broker

persistence,message-queue,raft
Using RAFT distributed consensus algorithm one call build a distributed message broker. The broker will also have the additional feature of supporting in-order delivery of messages to subscribers. The messages belonging to a topic will be delivered (or pulled from subscribers, in case of a pull model) in the order...

Using message queuing to decouple transactional updates to two systems?

database,transactions,message-queue,acid,eventual-consistency
In an article on BASE as an ACID alternative, Dan Pritchett presents one option for decoupling a transaction spanning two tables, Transaction table (e.g. Buy/Sell transaction) and User table: Source: http://queue.acm.org/detail.cfm?id=1394128 Dan also suggests that there is a problem with this approach: The message persistence is on the transaction host...

Mule Queued-Asynchronous Flow Queue vs. VM Queue

mule,message-queue
While studying and using Mule, I couldn't figure out if there's a difference between a queued-asynchronous flow queue and a VM queue. My question is, are they the same queues (just with different names along the documentation) or different ones? In a concrete example: <flow name="fooFlow" processingStrategy="queued-asynchronous"> <vm:inbound-endpoint path="foo" exchange-pattern="one-way"/>...

Azure Service Bus or Queue

azure,queue,message-queue
I have a process where I would like to use an Azure Queue or Service Bus to decouple the processing from the UI. A user will press a button, and I would like to place 2 messages in the queue, each with it's own topic. 1 set of competing consumers...

How do I building disque with make? Missing jemalloc/jemalloc.h error

message-queue,disque
When building disque with make getting the following error: Missing jemalloc/jemalloc.h ...

how to clear a queue routingkey binding in rabbitmq (.net)

c#,.net,rabbitmq,message-queue
How can I clear all routing key binding from a queue in rabbitmq? The QueueUnbind method need the routingkey parameter, but I just want to clear all bindings....

CRQS : benefits of queuing commands

php,message-queue,cqrs
I can't really understand the benefit of queuing commands in a CommandBus. Here is my very simple implementation of a CommandBus : class CommandBus implements ICommandBus { private $handlerLocator; public function __construct(ICommandHandlerLocator $handlerLocator) { $this->handlerLocator = $handlerLocator; } public function handle(ICommand $command) { $handler = $this->handlerLocator->getHandlerInChargeFor($command); try{ $handler->handle($command); } catch(\Exception...

Custom Error Queue Name when using EasyNetQ for RabbitMQ?

c#,.net,rabbitmq,message-queue,easynetq
Rather than having my unhandled exceptions go into EasyNetQ_Default_Error_Queue I wondered if there is a way that I can explicitly state the name of an Error Queue that should be used for a given application, so errors don't ALL end up in this one EasyNetQ_Default_Error_Queue? I can see how to...

Single process blocking queue

java,queue,message-queue,priority-queue
I am writing an application that communicates with hardware. While the application can receive and process multiple requests simultaneously in parallel, the hardware cannot! The hardware requires these parallel requests to basically be organised into a linear request chain each one executed one after the other. I also have a...

what is the difference between MQ and MQTT?

message-queue,mqtt,mq
I am beginner to MQTT , but what is confusing me is it's basic definition MQTT is a publish-subscribe based "light weight" messaging protocol'. Based on my previous understanding on MQ (Message Queue), both MQ and MQTT sounds very same to me. Can any one elaborate what the real difference...

How to log/monitor selected WM_USER+n Windows messages?

delphi,delphi-xe2,monitoring,message-queue
Some dialogs and data modules in my code need to delay execution and do this by calling PostMessage(). This is not across modules, i.e. the datamodule will have the handler for the PostMessage issued there, so will a dialog. There is (of course?) interaction between code in the different modules....

RabbitMQ returns same message again and again

c#,rabbitmq,message-queue
I try to unit test a rather basic scenario - worker queue with 2 workers and 1 publisher scenario, but it keeps returning the same message over and over again from the queue. The following code in the test just puts the 1 to 100 messages to the queue and...

Connection management when using kafka producer in high traffic enviornment

java,php,message-queue,apache-kafka
I am going to use kafka in a very high traffic environment of more than a billion requests per day. Every request will make a connection to kafka cluster to send message. So there will be so many connections being made continuously every second. This could cause issues like socket...

Tibco RV: Message Lifetime + No Copy for fanouts

listener,message-queue,ttl,tibco-rv
1, How long does a message live in the listener queue? Until the dispatcher reads the message out of the queue in a "1 publisher 1 consumer" scenario? Listener listener = new Listener(Queue.Default, transport, subject, new object()); listener.MessageReceived += OnMessageReceived; Dispatcher dispatcher = new Dispatcher(listener.Queue); 2, Tibco RV is typically...

Sending messages to MQOutput node

message-queue,websphere-mq,middleware,messagebroker,mqueue
Just want to confirm the correct way the MQ delivers messages to the MQOutput node. Recently came across a situation where i a felt bit confused. Here is the scenario. I have a local queue on Qmgr,say(A) which receives messages from applications and have a local broker associated with this...

Amazon batch job process queue

amazon-web-services,amazon-ec2,queue,message-queue
I am using AWS EC2 instances for bioinformatics work. I have a number (~1000) of large files that should be processed with a script on EC2 instances and results should be uploaded back to an S3 bucket. I want to distribute jobs (files) to a number of EC2 instances, preferentially...

Use Google App Engine's NDB as a message queue?

google-app-engine,message-queue,app-engine-ndb,google-datastore
Has anyone tried to use NDB as a message queue? We have several consumers and producers, which may want to do broadcast, multicast, and publish-subscribe. I've read several documents on why using a RDBMS as a message queue is bad. But in my case, my app can tolerate latency of...

Example mq_timedreceive

c,linux,timeout,message-queue
I can not find how to work properly with mq_timedreceive, can anyone give me an example? ssize_t mq_timedreceive(mqd_t mqdes, char *msg_ptr, size_t msg_len, unsigned *msg_prio, const struct timespec *abs_timeout); I want timereceive to do not spent more than 20 seconds waiting. Thank you very much....

Concurrent message consumption in ActiveMQ

java,jms,activemq,message-queue,messaging
I use a ActiveMQServer as a broker. Server.java import java.util.HashMap; import java.util.HashSet; import java.util.Map; import org.apache.activemq.api.core.TransportConfiguration; import org.apache.activemq.core.config.Configuration; import org.apache.activemq.core.config.impl.ConfigurationImpl; import org.apache.activemq.core.remoting.impl.netty.NettyAcceptorFactory; import org.apache.activemq.core.server.ActiveMQServer; import org.apache.activemq.core.server.ActiveMQServers; public class Server { public static void...

Why is Queue/FIFO ordering important in Message Queue?

design-patterns,data-structures,message-queue
One option for a Messaging Provider is a Message Queue, which provides FIFO ordering, i.e. Queue. Why would the ordering of messages be important? I wonder if is it because of the priority of the messages or anything similar to that. i would appreciate if anyone could explain with example.

Is there a way to specify a different Error Queue when using EasyNetQ.Hosepipe?

c#,.net,rabbitmq,message-queue,easynetq
I recently posted the following question... Custom Error Queue Name when using EasyNetQ for RabbitMQ? ... and was provided with a solution on how to use different, application specific Error Queue names when using EasyNetQ, rather than having all unhandled exceptions go into the default error queue (EasyNetQ_Default_Error_Queue). What I'd...

Message queues and application structure

service,rabbitmq,message-queue,soa,amqp
I've read the documentation and I'm pretty sure I understand the concept of the different components present in an AMQP based message queue. However, I am having a hard time mapping that to an actual service oriented implementation. Could someone give a sort of "textbook" example of two services communicating...