FAQ Database Discussion Community


Spark - How to run a standalone cluster locally

apache-spark,cluster-computing
Is there the possibility to run the Spark standalone cluster locally on just one machine (which is basically different from just developing jobs locally (i.e., local[*]))?. So far I am running 2 different VMs to build a cluster, what if I could run a standalone cluster on the very same...

Add Cluster Items onClick over Expandable List View

android-activity,maps,cluster-computing,expandablelistview,onclicklistener
First of all I'm sorry for my bad English :X Iam quite a newbie in Android App development and Iam about to develope an app which gives the user informations like free park-places in town. The app is based on Google Maps. My Problem: My app starts and shows a...

How does CoreOS load balancing work out there in the wild on a cloud service?

cloud,cluster-computing,load-balancing,coreos
Say I have a CoreOS cluster deployed on some cloud service somewhere. Now I have, say, 4 machines running a node.js app which follows all the 12-factor tenets and one machine with Couchbase. How does load balancing work in this scenario? Wouldn't ONE ip eventually run out of juice as...

SGE(Sun Grid Engine) Setup and Tuning

linux,performance,server,cluster-computing,sungridengine
I am just starting setup SGE (Sun Grid Engine) for cluster. I have two Cluster Node1 and Node2, Each Node has 24vCPU and 240GB RAM. Problem is looks like I don't use fully memory (Around 8%) and CPU (around 30%) Even thought I increase Thread of program looks like every...

How to deploy three config server instances for sharding in mongodb?

mongodb,server,cluster-computing,sharding
I am a newbie in MongoDB. Also I have not much knowledge in networking and servers. I am trying to deploy sharded cluster in mongodb using this article. It says I need to deploy three configuration server instances and create data directories for each. I can create a data directory...

Spring JPA transaction in clustered environment

spring,jpa,transactions,cluster-computing,jta
I use Spring-JPA in my application. I use container managed option for EntityManager and use JTA transaction manager. this Java EE application is deployed in JBoss clustered environment, will the isolation of transactions be taken care of? In other words, will two different requests on different JBoss nodes and two...

slurm script gives “command not found”

linux,bash,shell,cluster-computing,slurm
I am trying to submit a script to slurm that runs m4 on an input file. m4 is installed on our cluster, and if I run the script by itself, everything works as expected. But when I submit a run to slurm via a slurm script, I get an error....

sun grid engine qsub to all nodes

cluster-computing,sungridengine,qsub
I have a master and two nodes. They are install with SGN. And I have a shell script ready on all the nodes as well. Now I want to use a qsub to submit the job on all my nodes. I used: qsub -V -b n -cwd /root/remotescript.sh but it...

Invalid job array specification in slurm

cluster-computing,slurm
I am submitting a toy array job in slurm. My command line is $ sbatch -p development -t 0:30:0 -n 1 -a 1-2 j1 where j1 is script: #!/bin/bash echo job id is $SLURM_JOB_ID echo array job id is $SLURM_ARRAY_JOB_ID echo task id id $SLURM_ARRAY_TASK_ID When I submit this, I...

Minimum number of config servers needed for Mongo Cluster

mongodb,cluster-computing,sharding
I am trying to test sharding in MongoDB cluster. I am folllowing MongoDB documentation . They suggest to use 3 config servers. Can I set up a cluster with 1 or 2 servers? Or I need 3 or more config servers?...

How to recover Cassandra node from failed bootstrap

cassandra,bootstrap,cluster-computing
A node when down while bootstrapping a new node, and the bootstrapping failed. The node shut down, leaving the following messages in its log: INFO [main] 2015-02-07 06:03:32,761 StorageService.java:1025 - JOINING: Starting to bootstrap... ERROR [main] 2015-02-07 06:03:32,799 CassandraDaemon.java:465 - Exception encountered during startup java.lang.RuntimeException: A node required to move...

Barrier after MPI non-blocking call, without bookkeeping?

c++,parallel-processing,mpi,cluster-computing,hpc
I'm doing a bunch of MPI_Iallreduce non-blocking communications. I've added these Iallreduce calls to several different places in my code. Every so often, I want to pause and wait for all the Iallreduce calls to finish. Version 1 with MPI_Request bookkeeping -- this works: MPI_Request requests[]; MPI_Iallreduce(..., requests[0]); ... MPI_Iallreduce(...,...

Cassandra loss of a node

cassandra,cluster-computing
I'm trying to figure out, how to parameter my 2 nodes cluster, in order to have an exact replica, if one of them is down... using this tools to check it out : http://www.ecyrd.com/cassandracalculator/ For the following parameters : Cluster size: 2 / Replication Factor: 2 / Write Level: All...

Editing /etc/hosts for MPI cluster

linux,ip,mpi,cluster-computing,host
I struggled trying to set up an MPI cluster, following Setting Up an MPICH2 Cluster in Ubuntu tutorial. However, I tangled things up and it did not work, so I undid all the changes (except of the passphrase in step 7, which I have no clue how to undo) and...

PBS torque email variable

cluster-computing,pbs,torque
Here is an example of pbs that I use: #!/bin/bash #PBS -S /bin/bash #PBS -N myJob #PBS -l nodes=1:ppn=4 #PBS -l walltime=50:00:00 #PBS -q route export [email protected] #PBS -m ae #PBS -M [email protected] ./script1.sh echo $PBS_JOBID $PBS_O_WORKDIR | mail -s "$PBS_JOBNAME script1 done" $MYMAIL ./script2.sh echo $PBS_JOBID $PBS_O_WORKDIR | mail...

Continuous Query Processing with baseline indexing in Endeca

cluster-computing,atg,endeca
I wanted to whether if it is possible to have Continuous Query Processing with baseline indexing in Endeca cluster environment. I have an Endeca central server and multiple Endeca agent only instances. If I run Baseline Indexing in the central server, it re-indexes the records in all the MDEX engines(all...

In ActiveMQ clustering, how to find the consumer who picked a particular message or set of messages?

activemq,cluster-computing
We have set of queues and camel routes have been written to consume messages from the queue - queues live in an ActiveMQ server and camel routes in Apache Karaf. We have put up a cluster with multiple karaf nodes; problem is that these messages get read by camel routes...

Behavior of multiprocessing module on cluster

python,multiprocessing,cluster-computing,qsub
There are modules which are suited for multiprocessing on clusters, listed here. But I have a script which is already using the multiprocessing module. This answer states that using this module on a cluster will only let it make processes within a node. But what is this behavior like? Lets...

bash: /usr/bin/hydra_pmi_proxy: No such file or directory

c,linux,mpi,cluster-computing,mpich
I am struggling to set up an MPI cluster, following the Setting Up an MPICH2 Cluster in Ubuntu tutorial. I have something running and my machine file is this: pythagoras:2 # this will spawn 2 processes on pythagoras geomcomp # this will spawn 1 process on geomcomp The tutorial states:...

Generate overlay map image from geo data

google-maps,png,overlay,cluster-computing
I need to generate a transparent image to apply on map similar to these: Image_1or Image_2 I need to generate this image from a dataset: I have many data points (with lat and long) and I would to generate this image (server side) to show density on map of my...

Websocket issue on wildfly cluster

java,websocket,cluster-computing,wildfly-8,mod-cluster
I am using richfaces a4j:push in my Spring-JSF integrated application. I am able to push messages to the browser using websocket in my non cluster environment on wildfly 8.0.0. When I deploy the application on wildfly8.0.0 on redhat enterprise 7.0 with httpd clustering the push messages are not working. I...

Mysql Cluster without replication

mysql,cluster-computing
Is posible to do a MySQL Cluster without replicated database in the diferent nodes, I need to do a kind of RAC with MySQL Cluster. In a few words I need all the nodes in the cluster pointing to the same data storage. Thanks. ...

My ganglia is not working on cluster so i want to monitor jobs using torque

cluster-computing,torque,ganglia
ganglia is not working on cluster so I want to monitor jobs using torque. Can I monitor all jobs through torque even which are not submitted through torque?

Transform column of strings with word cluster values

r,nlp,cluster-computing
I am doing some basic NLP work in R. I have two data sets and want to replace the words in one with the cluster value of each word from the other. The first data set holds sentences and the second one the cluster value for each word (assume that...

jdbc backed channel on a cluster mode

jdbc,cluster-computing,spring-integration
I want to use the jdbc backed message channel. Reading the doc says that it is supported but after that I found a thread in the forum that explicitly says don't. Could you please clarify me if it's supported? In this case, could you guide me to do so? thanks...

R plot generates completely blank PNG

r,fonts,cluster-computing
I'm running an R script on a Beowulf computing cluster, and it generates a blank PNG whenever the script runs on one of the compute nodes. Running on the head node works fine. To try and diagnose the problem, I stripped down my script to this: #!/usr/bin/env Rscript the_plot <-...

Which one of the following in not part of a Beowulf cluster?

cluster-computing
I need an answer for this, Which one of the following in not part of a Beowulf cluster? Windows. Linux. MPI. IP Stack. ...

Elastica Client with cluster

elasticsearch,cluster-computing,elastica,foselasticabundle
I have an Elasticsearch cluster with Compose.io but I can't connect with Elastica Client. This is my configuration : $elasticaClient = new \Elastica\Client(array( 'servers' => array( array('host' => 'https://myusername:[email protected]', 'port' => 10050), array('host' => 'https://myusername:[email protected]', 'port' => 10062) ) )); $elasticaIndex = $elasticaClient->getIndex('test'); I got this error : Couldn't resolve...

Servlet, Spring: share session across multiple servers

spring,jsp,session,servlets,cluster-computing
Let's say I have a Java Servlet+JSP app using Spring framework and Tomcat 6. This app must be hosted on multiple machines. How can I share the HTTP session across many computers? I usually get my session using this code: HttpServletRequest httpRequest = (HttpServletRequest) request; HttpSession session = httpRequest.getSession(); Should...

Liferay 6.2 clustering issue with multicast

lucene,cluster-computing,liferay-6,ehcache,jgroups
I am trying to cluster ehcache and lucene with Liferay 6.2 EE sp2 bundle on 2 servers with mutlicast enabled. WE have Apache HTTPD servers fronting tomcat servers using reverse proxy. A valid 6.2 license is deployed on both the nodes. We user the following properties in the portal-ext.properties: cluster.link.enabled=true...

Apache Spark NumberformatException with content not existing in file

apache-spark,cluster-computing,hdfs
Executing my spark app in local mode works perfectly fine but running on the cluster delivers an exception for a date field "yyyy-MM-dd hh:MM:ss" with following exception: 15/02/05 16:56:04 WARN TaskSetManager: Lost task 3.0 in stage 0.0 (TID 3, kmobd-dnode2.qudosoft.de): java.lang.NumberFormatException: For input string: ".1244E.1244E22" at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043) at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110) at...

Docker container running Mesos cluster and running other docker containers on cluster (using Marathon)

hadoop,docker,cluster-computing,mesos,marathon
I'm just starting off with Mesos, Docker and Marathon but I can't find anywhere where this specific question is answered. I want to set up a Mesos cluster running on Docker - there are a couple of internet resources to do this, but then I want to run Docker containers...

issue monitoring hadoop response

hadoop,cluster-computing,ganglia,gmetad
I am using ganglia to monitor Hadoop. gmond and gmetad are running fine. When I telnet on gmond port (8649) and when I telnet gmetad on its xml answer port, I get no hadoop data. How can it be ? cluster { name = "my cluster" owner = "Master" latlong...

How to hunt down obscure HA clustering bug in Wildfly 8.2.0.Final

rest,session,cluster-computing,wildfly,weld
The setup I have a Wildfly 8.2.0.Final application server running a cluster in domain mode using the full-ha profile. The cluster consists of two instances of wildfly, master and slave, each running on its own virtual machine. The application My project is deployed as a war-file on the application server....

What's a Cluster / Bucket in couchbase Server

server,cluster-computing,couchbase,bucket
I'm new to Couchbase and NoSql technologies in general, but I'm working on a web chat application running on node js using express and some other modules. I've chosen to work with NoSql to store sessions and all needed data on server-side. But I don't really understand some important features...

IBM Connections 5 custom theme not getting published to second cluster node

synchronization,themes,cluster-computing,ibm,connections
we have the following small cluster setup: Server 1: DMGr, Nodeagent, Connections_server1 Server 2: Nodeagent, Connections_server2 All shared resources (including the customization dir) are on a separate file server that is linked on /opt/IBM/Connections/data/shared on both servers. I want to customize the defaultTheme, so I copied it to /customizationDir/themes/defaultTheme/custom.css After...

IHS and WAS configuration OK, but can't connect to application via WebServer

cluster-computing,websphere-7,ibm-was,ibmhttpserver
I've successfully setup IBH HTTP Server, generated the plugin, propagated it from WAS admin console to the remote WebServer, but still cannot connect to the application via the Web Server. I've followed with success the steps for configuring the HTTP server as stated in IBM manuals here. The WebServer is...

Is OpenCL ready for use on CPU?

opencl,mpi,cluster-computing,hpc
In the lab we have an heterogeneous cluster setup, with many Intel CPUs, a few AMD CPUs and a couple of Nvidia GPUs. For HPC development, the one thing I know that I could write once and run everywhere on this setup is OpenCL (not even Java ;) ). But...

Predis: Pros and Cons of the two cluster strategies

php,session,redis,cluster-computing,predis
I am using Predis to connect to a Redis 3.0.2 cluster. So, I have an option to use either predis or redis cluster strategy. I am wondering if there are any significant pros and cons of choosing one of them over the other? Based on my wandering through documentation, I...

Normalization in Java (dataset of doubles)

java,double,cluster-computing,normalization
I'm working on a clustering program, and have a dataset of doubles that I need to normalize in order to make sure that every double (variable) has the same influence. I would like to use min-max normalization where for every variable the min and max value are determined, but I'm...

Unable to run sparkPi on Apache Spark cluster

apache-spark,cluster-computing,bigdata
The following is my spark master UI which shows 1 registered worker . I am trying to run the sparkPi application on the cluster ,using the following submit script ./bin/spark-submit \ --class org.apache.spark.examples.SparkPi \ --master spark://159.8.201.251:7077 \ /opt/Spark/spark-1.2.1-bin-cdh4/lib/spark-examples-1.2.1-hadoop2.0.0-mr1-cdh4.2.0.jar \ 1 but it keeps giving the following warning ,and never finishes...

HornetQ using JNDI / remoting when used with WildFly 8.1 cluster

cluster-computing,jndi,wildfly,hibernate-search,hornetq
Previously 2 JBoss 6 servers (master & slave) were started with 2 different IP addresses (127.0.0.1 and 127.0.0.2) and those 2 server instances were run in domain mode. The instances used HornetQ to communicate index jobs for Hibernate Search. I'm trying to create a similar configuration using Wildfly instead of...

Memsql, Is it possible to add more leaf nodes on the same host machine?

database,cluster-computing,memsql
I installed a functional test with Memsql on a machine with an aggregation node on port 3307 and a leaf node 3306. As queries being ran, there might be additional cpu available. Hence, is it possible to add more leaf nodes on the same machine to take advantages of available...

Hadoop - find out the resource utilization of every node and distribute load equally in a cluster

hadoop,mapreduce,cluster-computing,resource-utilization
I want to find out the resource utilization (CPU,RAM) and the Data processing taking place at every node in the Hadoop cluster. Is there any way using MapReduce or HDFS commands to find out the load distributed across each node ? Also, if one node is busy (overloaded) and another...

Starting Zookeeper Cluster. Error: Could not find or load main class org.apache.zookeeper.server.quorum.QuorumPeerMain

apache,cluster-computing,zookeeper
(I'm running on CentOS 5.8). I've been following the direction for a Clustered (Multiserver) Zookeeper Set-up, but getting an error when I try to start up my server. When I run the command as described in the documentation: java -cp zookeeper-3.4.6.jar:lib/log4j-1.2.16.jar:conf \ org.apache.zookeeper.server.quorum.QuorumPeerMain conf/zoo.cfg I get the error: Error: Could...