FAQ Database Discussion Community


What is the diameter of a graph with just one noed?

algorithm,math,distributed-computing,distributed-algorithm
I'm trying to find an answer to a problem in my Distributed Algorithms course, and to do so I want to get something clarified. What is the diameter of a graph with one node, with an edge to itself? Is it 1 or 0? If you are interested, the question...

How to execute a method once (multiple processes/instances) per minute utilizing AWS

java,amazon-web-services,distributed,distributed-computing,amazon-sqs
I have a process that sends SQS messages every minute. It's important that the messages go out every minute so I'm planning on running the process on multiple instances so that it's more fault tolerant. Even though it's running on multiple instances I only want the SQS messages to go...

Synchronize actions in a distributed system

distributed,distributed-computing,zookeeper,distributed-system
What techniques/tools can be used to implement a distributed system with these requirements: At a given time, the system can be in one of 3 states: SYNCING, COMPUTING, or IDLE. Each node in the system can receive two instructions: sync() and compute(). A sync() instruction will be sent to all...

Why doesn't LocateRegistry.getRegistry() fail if I specify the wrong port number?

java,rmi,distributed,distributed-computing
I'm trying to distribute a simple room booking system that I implemented, I tried researching and this is what I came up with.. but When I run them and enter different port numbers the client still runs..shouldn't it only run if the port number matches the one I entered for...

Shuffled vs non-shuffled coalesce in Apache Spark

scala,apache-spark,bigdata,distributed-computing
What is the difference between the following transformations when they are executed right before writing RDD to a file? coalesce(1, shuffle = true) coalesce(1, shuffle = false) Code example: val input = sc.textFile(inputFile) val filtered = input.filter(doSomeFiltering) val mapped = filtered.map(doSomeMapping) mapped.coalesce(1, shuffle = true).saveAsTextFile(outputFile) vs mapped.coalesce(1, shuffle = false).saveAsTextFile(outputFile)...

How are distributed queues architectured?

queue,distributed,distributed-computing,distributed-system
What are architectural patterns/solutions that make distributed queues tick? Please share for both ordered and non-ordered types....

Is curator's persistent ephemeral nodes just regular ephemeral with retries?

distributed-computing,zookeeper,distributed-system,curator
I'm new to curator and zk - and wanted to double check my understanding with the rest of the community. It seems that documentation for curator is not that well covered. Are curator's persistent ephemeral nodes basically ephemeral znodes, but have extra mechanisms to re-establish connections once it's disconnected? Are...

Akka clustering conflicts

akka,zeromq,distributed-computing
The Akka doc talks about a variety of seemingly inter-related Akka technologies without distinguishing much between them: Akka Networking Akka Remoting Akka Clustering The Akka ZeroMQ module My understanding is that "Akka Networking" is simply a module/lib that gives Akka the ability to speak to remote actor systems over TCP....

Parallelism and Failover of a Sequential Data

java,distributed-computing,cqrs,failover,xa
Good time guys! We have a pretty straightforward application-adapter: once in 30 seconds it reads records from a database (can't write to it) of one system, converts each of these records into an internal format, performs filtering, encrichment, ..., and, finally, transforms the resulting, let's say, entities into an xml...

why Spark is not distributing jobs to all executors, but to only one executer?

performance,configuration,apache-spark,distributed-computing,spark-streaming
My Spark cluster has 1 master and 3 workers (on 4 separate machines, each machine with 1 core), and other settings are as in the picture below, where spark.cores.max is set to 3, and spark.executor.cores also 3 (in pic-1) But when I submit my job to Spark cluster, from the...

How to scale a trie across multiple servers

architecture,system,scalability,distributed-computing,trie
Does anyone know how I might scale a Trie across multiple machines? Say the first machine runs out of space and I need to add more words from a very large dictionary, what might I do to add more words? (I am a Java thinker, but I believe the answer...

Action Listener doesn't change set variable to other value

java,user-interface,jbutton,distributed-computing
I am writing GUI for a chat, and I have problem I can't seem to find a solution. When button Send is clicked variable OKpressed should change to true and in function getUserInput it should recognize it changed but it doesn't.. It's acting like it still says false.. I tried...

Distribute computing on multiple devices

java,machine-learning,bigdata,distributed-computing
My project takes very long time at running, I made threads and distributed data and processing on my processor cores, But, still takes long time, I tried to optimize the code as i can, How can i distribute computing on multiple laptops?

Cloud Haskell hanging forever when sending messages to ManagedProcess

haskell,cloud,distributed-computing,cloud-haskell
The Problem Hello! I'm writing in Cloud Haskell a simple Server - Worker program. The problem is, that when I try to create ManagedProcess, after the server disovery step, my example hangs forever even while using callTimeout (which should break after 100 ms). The code is very simple, but I...

How to reproduce experiments in a non-adhoc manner

distributed,distributed-computing,experimental-design
For a master thesis in the field of distributed computing (think of hadoop and two-level schedulers like Mesos) I'm setting up various experiments on a university cluster. However I'm already piling up bash scripts which function as a driver for the experiment. I miss composability and reuse between subparts of...

Client-Server architecture: 100% Android (Android as a server) or J2EE+Android?

android,java-ee,architecture,client-server,distributed-computing
Context I am considering going into a client-server architecture with Java. The idea is that several Android tablets (let's say around 15) need to display a content from a server. Content can vary times to times (e.g. day display v/s night display). Furthermore, tablets will also display a Yes /...

Persistent storage for Apache Mesos

postgresql,distributed-computing,mesos,mesosphere
Recently I've discovered such a thing as a Apache Mesos. It all looks amazingly in all that demos and examples. I could easily imagine how one would run for stateless jobs - that fits to the whole idea naturally. Bot how to deal with long running jobs that are stateful?...

How is service discovery not a subset of centralized configuration?

distributed-computing,zookeeper,service-discovery,consul,consensus
I'm looking at consensus-type tools like ZooKeeper, Consul and Eureka and they all seem to market the same set of solutions: Service discovery Dynamic, centralized configuration management Synchronization primitives Consensus algorithms However the more I read about these things, the more I struggle to see how service discovery is really...

Hazelcast 3.3 - EntryProcessor is accessing “non-local” keys

distributed-computing,hazelcast
I'm using Hazelcast 3.3. One member writes entries to an IMap and calls map.executeOnEntries(myEntryProcessor). The task of EntryProcessor is to just print the entries on console. However, the members (3 other and the 1st one = 4 members) seem to print overlapping set of entries. My understanding was that the...

Why are Distributed Systems considered complex?

asynchronous,distributed-computing,middleware
I'm just getting into the concept of a Distributed System and its advantages and disadvantages. In the book I'm reading it discusses the complexity of a Distributed System and that they are inherently complex, it lists the following as potential reasons for complexity; Heterogeneity Asynchronous communication Partial failures What I...

Spark distributed matrix multiply and pseudo-inverse calculating

matrix,apache-spark,distributed-computing
I am very new in Apache Spark Scala. Can you help me with some operations? I have two distributed matrix H and Y in Spark Scala. I want to compute the pseudo-inverse of H and then multiply H and Y. How can I do this? ...

YARN UNHEALTHY nodes

hadoop,distributed-computing,cloudera,yarn,cloudera-cdh
In our YARN cluster which is 80% full, we are seeing some of the yarn nodemanager's are marked as UNHEALTHY. after digging into logs I found its because disk space is 90% full for data dir. With following error 2015-02-21 08:33:51,590 INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Node hdp009.abc.com:8041 reported UNHEALTHY with details: 4/4...

How to classify a failure detector?

distributed-computing,distributed-system,distributed-algorithm
I understand that failure detectors in asynchronous systems are basically classified as (eventually)perfect/(eventually)strong and how those classes are defined, but I kind of struggle to get the intuition behind it. Suppose I have a concrete implementation of a failure detector, which periodically listens for heartbeat messages from each process. If...

IPython MPI with a Machinefile

ipython,mpi,distributed-computing
I want to use IPython's MPI abilities with distributed computing. Namely I would like MPI to be run with a machine file of sorts so I can add multiple machines. EDIT: I forgot to include my configuration. Configuration ~/.ipython/profile_default/ipcluster_config.py # The command line arguments to pass to mpiexec. c.MPILauncher.mpi_args =...

How to store data in and get data out of memory mapping files using CopyMemory in VBA?

vba,excel-vba,distributed-computing,memory-mapping
I am trying to build a distributive computing system that uses memory mapping files to coordinate work among several networked PCs all via VBA. Put another way, I want to get a group of networked computers to do work at the same time in a coordinated way on a single...

Does Data Synchronization between servers count as distributed system? [closed]

database,distributed,distributed-computing,data-synchronization,distributed-system
I'm just a little bit confused of this concept. I heard the words "Distributed system" a lot, but I'm not really sure my stuff is kind of "Distributed system". Basically, we have a master server( a very big one) as the front line production server. Then , in order to...

How to stop mesos from offering resources to a framework?

distributed,distributed-computing,mesos,mesosphere
I have a use case where I have 20-30 frameworks runnings on mesos cluster that has over 200 nodes. A lot of the times mesos is offering resources to frameworks that do not want any offers at all. While doing that, it is offering little resources to frameworks that actually...

Connect from specific ip address in linux

linux,sockets,distributed-computing
I'm writing a distributed system and I want to test it on my machine. I created several ip addresses on the interface lo using ip addr add ip_add dev lo. I have binded all servers to their specific addresses and now I want my servers to connect to each other...

C# How to Generate Random Number Depends on Probabilities

c#,.net,math,random,distributed-computing
I have a situation in which I must generate a random number, this number must be either zero or one So, the code is something like this: randomNumber = new Random().Next(0,1) However, the business requirements state that there is just 10% probability that the generated number is zero and 90%...