cassandra , Check if JNA is enabled in Cassandra

Check if JNA is enabled in Cassandra


Tag: cassandra

I know from other posts that Cassandra writes some warning in log files if JNA is not configured properly. My log files (cassandra.log and system.log) do not contain the word "jna" (all case checked). Can I assume that JNA is configured correctly for my instance of Cassandra ?

Is there any way to check if the JNA configuration is correct ?


If JNA is configured properly, you should see a message in your log like this:

INFO [main] 2014-06-06 15:55:11,664 (line 121) JNA mlockall successful

Also in your log, the JNA library should appear in your classpath output:

INFO [main] 2014-06-06 16:47:13,318 (line 191) Classpath:...


Where can I observer writes to Cassandra database, aka where are they logged?

Trying to track down a problem with one of our developers, mainly a program he wrote that modifies (adds some flags) to existing entries in the various tables in our Cassandra keyspace. The issue is that it seems to work just fine for many of the tables, but at least...

OutofMemoryErrory creating fat jar with sbt assembly

We are trying to make a fat jar file containing one small scala source file and a ton of dependencies (simple mapreduce example using spark and cassandra): import org.apache.spark.SparkContext import org.apache.spark.SparkContext._ import com.datastax.spark.connector._ import org.apache.spark.SparkConf object VMProcessProject { def main(args: Array[String]) { val conf = new SparkConf() .set("", "") .set("spark.executor.extraClassPath",...

How to update a field which is indexed?

I want to update a field in Cassandra which is indexed using phantom scala sdk like: this.update.where( eqs folderId) .and(_.owner eqs owner) .modify(_.parent setTo parentId) the parent field is a indexed field in table. But the operation is not allowed when compile the code, there will have compile exception like:...

Where are the API docs for org.apache.spark.sql.cassandra for Spark 1.3.x?

I'm writing a Spark job that uses Spark-Cassandra connector to connect to Cassandra from spark and then runs queries on Spark/Cassandra using Spark SQL. I was wondering where I could find the API docs for this? Looking at the api here It would seem like the package doesn't even...

What is the difference between broadcast_address and broadcast_rpc_address in cassandra.yaml?

GOAL: I am trying to understand the best way to configure my Cassandra cluster so that several different drivers across several different networking scenarios can communicate with it properly. PROBLEM/QUESTION: It is not entirely clear to me, after reading the documentation what the difference is between these two settings: broadcast_address...

Cassandra: Insert with older timestamp

(Cassandra 2.0.9, using CQL) I've accidentally updated a row in a table which was managing its own timestamp (100 * a specific sequence number). Now, because my timestamp is the current time, none of the updates are working. I understand why this is, but I'm trying to recover from it....

Exporting Data from Cassandra to CSV file

Table Name : Product uid | productcount | term | timestamp 304ad5ac-4b6d-4025-b4ea-8b7991a3fe72 | 26 | dress | 1433110980000 6097e226-35b5-4f71-b158-a1fe39a430c1 | 0 | #751104 | 1433861040000 Command : COPY product (uid, productcount, term, timestamp) TO 'temp.csv'; Error: Improper COPY command. Am I missing something? ...

Error running spark app using spark-cassandra connector

I have written a basic spark app that reads and writes to Cassandra following this guide ( This is what the .sbt for this app looks like: name := "test Project" version := "1.0" scalaVersion := "2.10.5" libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % "1.2.1", "" % "guava" % "14.0.1",...

Is it possible to use a timestamp in ms since epoch in select statement for Cassandra?

I know that using the formats listed here ( work to query cassandra. However, I'm having a hard time determining if it is even possible to use ms since epoch in the select statement. I feel like it should since it data can be sent to cassandra in the ms...

how to create a keyspace in cassandra?

i'm using a snippet to understand cassandra and syntax: import com.datastax.driver.core.Cluster; import com.datastax.driver.core.ResultSet; import com.datastax.driver.core.Row; import com.datastax.driver.core.Session; public class App { public static void main(String[] args) { Cluster cluster; Session session; // Connect to the cluster and key space "demo" cluster = Cluster.builder().addContactPoint("").build(); session = cluster.connect("demo"); // Insert one record...

Does Cassandra works with IBM JVM

Can I install and start Cassandra into a x-linux OS with a IBM SDK for Java? Will that work? Any specific version? 2.1, 2.0 that will work ? Thanks in advance.

Does Spark from DSE laod all data into RDD before running SQL Query?

Running DSE 4.7 So say I have a 4 node DSE Cassandra/Spark cluster... I have a Cassandra table with say 4,000,000 records in it. On Spark running the following Spark SQL "select * from table where email = ? or mobile = ?" Will Spark load all the data into...

Cassandra with uneven hardware, how to configure?

We are building a Cassandra (2.1.5) cluster for storing large amount of timeseries data, and we are planning to utilize existing hardware, problem is the hardware available is really different. 2 machines with: 4 core, 8 GB, SSD 2 machines with: 8 core, 16 GB, SSD 2 machines with: 32...

How to change the flush queue size of cassandra

How to assign more memory for the flush queue between memtable and sstable in Cassandra. I have getting timeout errors and the heap and young region usage seems to within limits. There is no other processing happening except Cassandra in the machine. Also how to find if any requests are...

How to get good performance on reading cassandra partitions in spark?

I am reading data from cassandra partition to spark using cassandra-connector.I tried below solutions for reading partitions.I tried to parallelize the task by creating rdds as much as possible but both solution ONE and solution TWO had same performance . In solution ONE , I could see the stages in...

Commenting Cassandra's keyspace, table, column

In Oracle there is possibility to add a comment about a table, view, materialized view, or column into the data dictionary, e.g. COMMENT ON COLUMN employees.job_id IS 'abbreviated job title'; I found this particularly usefull as a tester when trying to understand ideas behind the names which are not necessarily...

Using partition key along with secondary index

Following are the two queries that I need to perform. select * from where dept = 100 and emp_id = 1; select * from where dept = 100 and name = 'One'; Which of the below options is better ? Option 1: Use secondary index along with a partition key....

Modeling account for rest communication cassandra

I need to model account (first name, last name, email as username etc.) in cassandra along with currently active token. My initial idea was to create account_by_email which will have skinny row partitioned by email with static columns and have clustering by access_token (and maybe TTL) and than you can...

How to delete a record in Cassandra?

I have a table like this: CREATE TABLE mytable ( user_id int, device_id ascii, record_time timestamp, timestamp timeuuid, info_1 text, info_2 int, PRIMARY KEY (user_id, device_id, record_time, timestamp) ); When I ask Cassandra to delete a record (an entry in the columnfamily) like this: DELETE from my_table where user_id =...

Unbalanced Cassandra replicas storage

In our setup we have 2 DCs, 21 Cassandra nodes in each DC, and a total of 4 replicas per record (in one of the keyspaces) - two replicas per site. Every Cassandra node is setup with 16 VNodes. We did not manually set the initial_token for each node in...

update cassandra field using string concatenation

I am trying to update an existing string column in cassandra table. For example i want to append domain id in front of username. Following is my table id, username 1, agaikwad 2, xyz I want to write cql to update above table to reflect following id, username 1, homeoffice\\agaikwad...

java.sql.SQLNonTransientConnectionException:Keyspace names must be composed of alphanumerics and underscores (parsed: '')

I'm trying connect to cassandra db and verify users to login and sign up I'm getting this error: Keyspace names must be composed of alphanumerics and underscores (parsed: '') at org.apache.cassandra.cql.jdbc.Utils.parseURL( at org.apache.cassandra.cql.jdbc.CassandraDriver.connect( at java.sql.DriverManager.getConnection( at java.sql.DriverManager.getConnection( at

Error when running job that queries against Cassandra via Spark SQL through Spark Jobserver

So I'm trying to run job that simply runs a query against cassandra using spark-sql, the job is submitted fine and the job starts fine. This code works when it is not being run through spark jobserver (when simply using spark submit). Could someone tell my what is wrong with...

cassandra search a row by secondary index returns null

I have created a TABLE and index As follows CREATE TABLE refresh_token ( user_id bigint, refresh_token text, access_token text, device_desc text, device_type text, expire_time timestamp, org_id bigint, PRIMARY KEY (user_id, refresh_token) ) WITH CLUSTERING ORDER BY (refresh_token ASC) CREATE INDEX i_access_token ON demodb.refresh_token (access_token); After i insert or delete data...

Slicing over partition rows using tuple operation in CQL

I am trying to understand the behavior of tuple operator with clustering keys. Here is what I was trying to do: create table sampletable (a int,b int,c int, d int, e int, primary key((a,b,c),d,e)); insert into sampletable(a,b,c,d,e) values(1,1,1,1,1); insert into sampletable(a,b,c,d,e) values(1,1,1,1,1); insert into sampletable(a,b,c,d,e) values(1,1,1,1,2); insert into sampletable(a,b,c,d,e) values(1,1,2,1,1);...

Cassandra WordCount Hadoop

Can anyone explain to me the following lines from Cassandra 2.1.15 WordCount example? CqlConfigHelper.setInputCQLPageRowSize(job.getConfiguration(), "3"); CqlConfigHelper.setInputCql(job.getConfiguration(), "select * from " + COLUMN_FAMILY + " where token(id) > ? and token(id) <= ? allow filtering"); How do I define concrete values which will be used to replace "?" in the query?...

New Datastax driver for Tableau is not working

trying to run Tableau on top of DSE 4.7. It fails. I can't do something in worksheet or preview the data. Get this error: "Missing EOF at 'tablename_i_try_to_query' " What is the right way to fix it?...

Dataframe is not saved into Cassandra

I have one application with Spark (version 1.4.0) and Spark-Cassandra-connector (version 1.3.0-M1). In which, I am trying to store one dataframe into Cassandra table which has two columns (total, message). And i already created table into Cassandra with these two columns. Here is my Code, scoredTweet.foreachRDD(new Function2<JavaRDD<Message>,Time,Void>(){ @Override public Void...

to alter or create a new table in cassandra to add new columns

I am using DSE cassandra. I wanted to add new attributes to the existing table. I wanted to know what is the best practice to achieve this? Should i be adding new columns to existing table or creating new table? What are the pros and cons for either approach?...

Select first N rows of Cassandra table

As stated in this doc to select a range of rows i have to write this: select first 100 col1..colN from table; but when I launch this on cql shell I get this error: <ErrorMessage code=2000 [Syntax error in CQL query] message="line 1:13 no viable alternative at input '100' (select...

Spark Cassandra SQL can't perform DataFrame methods on query results

So I have a Spark-Cassandra cluster that I am trying to execute sql queries on. I build a jar with sbt assembly then I submit it with spark-submit. This works fine when I am not using spark-sql. When I am using spark sql I get an error, below is the...

Titan start fails: not found

I downloaded and unziped the and used the command ./ -v start. Now I get the output: ./ -v start Forking Cassandra... OpenJDK 64-Bit Server VM warning: The UseParNewGC flag is deprecated and will likely be removed in a future release Running nodetool statusthrift.Error: Config file not found: /usr/lib64/jvm/java-1.9.0-openjdk-1.9.0/jre/conf/management/

ImportError: No module named cassandra

I'm trying to get Cassandra to work with Django using Windows 7 for the first time. I've installed Django, Cassandra and django_cassandra_engine. However when I open a new Django project I get the following errors. C:\Python27\python.exe -u C:\Program Files (x86)\JetBrains\PyCharm 4.5.1\helpers\pydev\ 54467 54468 PyDev console: starting. import sys; print('Python %s...

Cassandra version differences

I started reading Cassandra the definitive guide, which is based on Cassandra 0.7. Now, I'm trying to experiment with Cassandra 2.1.5 and it seems that there's a lot of differences which is really confusing. For example, I see that in 0.7 version CQL did not exist. On the other hand,...

Cassandra queries performance, ranges

I'm quite new with Cassandra, and I was wondering if there would be any impact in performance if a query is asked with "date = '2015-01-01'" or "date >= '2015-01-01' AND date <= '2015-01-01'". The only reason I want to use the ranges like that is because I need to...

Cassandra-Hadoop integration

I'm interested in integrating Cassandra and Hadoop, more precisely, using Cassandra as input for Hadoop jobs. Each Cassandra node is also a Hadoop node. I found these tutorials 1 and 2 that somewhat explain the integration. I'm new to Cassandra so I'm still figuring out some things. My question is...

File Processing with Spark and Cassandra

Right now I'm working on loading a table from a Cassandra cluster into a Spark cluster with the Datastax Cassandra Spark Connector. Right now the spark program performs a simple mapreduce job that counts the number of rows in the Cassandra table. Everything is set up and run locally. The...

Apache Cassandra - cqlsh operation timeout

I am trying to start cqlsh and this is what I get: /bin$ ./cqlsh Connection error: ('Unable to connect to any servers', {'': OperationTimedOut('errors=None, last_host=None',)}) I tried removing ~/.cassandra, did not work. I also compared cassandra.yaml with a version that worked. Any ideas?...

Partially indexing Cassandra table with SOLR

One of the tables inside our Cassandra (DSE 4.7) Cluster contains south of 15 billion records. With the number of servers we have - it would be impossible to index them all with Solr. So, is it possible to somehow index the data partially/sample and/or start indexing and then "pause"...

Preparing Cassandra SELECT Statements in Python

I'm trying to run prepared select queries against a Cassandra table. The table is defined as such: class EmailAddressLookup(Model, ModelOperations, JSONSerializer): __table_name__ = 'email_address_lookup' email_address = columns.Text(primary_key=True) user_id = columns.Integer(primary_key=True) My INSERT works great. It looks like this: i_email_lookup = session.prepare("""INSERT INTO email_address_lookup (user_id, email_address) VALUES (?, ?)""") session.execute(i_email_lookup, (user_id,...

How to generate UUID(Long) using cassandra timestamp in cluster environment?

I have the requirement where we need to generate UUID as Long value using Java based on Cassandra timestamp which is in cluster. Can anyone help how to geranate it using java and cassandra cluster timestamp combination?

Cassandra node almost out of space, but nodetool cleanup is increasing disk use?

One of our nodes was at 95% disk use and we added another node to the cluster to hopefully rebalance but the disk space didn't drop on the node. I tried doing nodetool cleanup assuming that excess keys were on the node, but the disk space is increasing! Will cleanup...

How to un-nest a spark rdd that has the following type ((String, scala.collection.immutable.Map[String,scala.collection.immutable.Map[String,Int]]))

Its a nested map with contents like this when i print it onto screen (5, Map ( "ABCD" -> Map("3200" -> 3, "3350.800" -> 4, "200.300" -> 3) (1, Map ( "DEF" -> Map("1200" -> 32, "1320.800" -> 4, "2100" -> 3) I need to get something like this Case...

dse cassandra solr doesnt return _uniqueKey in response

Im using Datastax 4.6. My solr client queries data by using _uniqueKey. From version 4.6 the limitation about using simple primary key is removed. How can i configure solr or create table in cassandra, so that I receive in solr response information about synthetic key _uniqueKey. There is no problem...

Timeout using SSTableloader for Cassandra Aws Instance

I'm trying to use sstableloader to load SSTable (.db) files into a Cassandra Cluster running on an AWS EC2 instance. This error occurrs: Established connection to initial hosts Opening sstables and calculating sections to stream Streaming relevant part of C:\Users\SNCUser\dataquest\CassandraLoader\WrDir\beed5b97-0b52-45d7-be5d-fbbac00ac607\device_data\blob\device_data-blob-ka-1-Data.db to [/172.*.*.*] ERROR 16:08:36 [Stream #1114a0d0-1054-11e5-9ccc-65ee5fdd8902] Streaming error occurred

Cassandra add TTL to existing entries

How can I update an entire table and set a TTL for every entry? Current Scenario (Cassandra 2.0.11): table: CREATE TABLE external_users ( external_id text, type int, user_id text, PRIMARY KEY (external_id, type) ) currently there are ~40mio entries in this table and i want to add a TTL for...

gremlin hangs with sigle node hbase,titan-all 0.4.4

I have set up single node hadoop and hbase onto it. I also set up titan onto it. But as soon as I start gremlin and do , it hangs and nothing happens. my is as follows: storage.backend=hbase storage.hostname= storage.port=2181 cache.db-cache = true cache.db-cache-clean-wait = 20 cache.db-cache-time =...

How to export data from Cassandra to mongodb?

I am using Apache (Kafka-Storm-Cassandra) for real time processing.The problem I am facing is that I can't use aggregation queries on Cassandra directly(Datastax can be used but it is a paid service).Moreover, I also considered using mongodb but It is not good for more and frequent writes. So, I am...

Lucene how to index in Database (Cassandra)

I am just experimenting with Lucene and want to indexing objects in Database(Cassandra) as a table. But, I didnt realized out, how the indexing does work on Cassandra. Especially searching... When i take a simple Example Indexing in Lucene: Document doc = new Document(); doc.add(new TextField("id", "Hotel-1345", Field.Store.YES)); doc.add(new TextField("description",...

Cassandra data model to store embedded documents

In mongodb we can able to store embedded documents into a collection.Then, How do we store embedded documents into cassandra??? For this sample JSON representation??? UserProfile = { name: "user profile", Dave Jones: { email: {name: "email", value: "[email protected]", timestamp: 125555555}, userName: {name: "userName", value: "Dave", timestamp: 125555555} }, Paul...