csv,groovy , Refer to column headings csv Groovy

Refer to column headings csv Groovy


Tag: csv,groovy

I'm trying to figure out if there's a way of accessing column headings in a CSV using OpenCSV in groovy? This is what I have:

@GrabConfig( systemClassLoader=true )
@Grab( 'mysql:mysql-connector-java:5.1.27' )
@Grab(group = 'net.sf.opencsv', module = 'opencsv', version = '2.3')
import groovy.sql.Sql
import au.com.bytecode.opencsv.CSVReader
import au.com.bytecode.opencsv.CSVParser
def sql = Sql.newInstance("jdbc:mysql://localhost:3306/nid", "developer","whatever", "com.mysql.jdbc.Driver")
def notes = sql.dataSet("vdc_notifications")
def TEST_FILE_NAME = 'C:\\Users\\me\\Desktop\\test.csv'

List<String[]> rows = new CSVReader(new FileReader(new File(TEST_FILE_NAME)), CSVParser.DEFAULT_SEPARATOR, CSVParser.DEFAULT_ESCAPE_CHARACTER, CSVParser.DEFAULT_QUOTE_CHARACTER).readAll()

What i'm trying to get to is being able to do:

rows.each() { row -> println row.some_column_name }


usually the first row of a CSV file contains the names of the columns. so you can either use a simple parsing approach and take the first row for the header:

import au.com.bytecode.opencsv.CSVReader
import au.com.bytecode.opencsv.bean.CsvToBean;
import au.com.bytecode.opencsv.bean.HeaderColumnNameMappingStrategy;

def csv = """\

// read the files and keep the first one as the header
def csvr = new CSVReader(new StringReader(csv))
def header
while ((line=csvr.readNext())) {
    if (!header) {
        header = line
    } else {
        // create a map from the header and the line

Or you can use the CsvToBean and the HeaderColumnNameMappingStrategy to create beans:

class Person {
    String name
    Integer age

// use the mapper
def ctb = new CsvToBean<Person>()
def hcnms = new HeaderColumnNameMappingStrategy<Person>()
hcnms.type = Person
println ctb.parse(hcnms, new StringReader(csv))


Groovy: run SQL SELECT LIKE from file with params

The groovy code is import groovy.sql.* dbUrl = 'jdbc:sqlserver://server' dbUser = 'username' dbPassword = 'password' dbDriver = 'com.microsoft.sqlserver.jdbc.SQLServerDriver' sql = Sql.newInstance(dbUrl, dbUser, dbPassword, dbDriver) def sqlfromfile = new SqlFromFile() sqlfromfile.sql_filename='select_query.sql' sqlfromfile.read() try{ def result = sql.rows(sqlfromfile.result,[id:'01']) println result }catch(e){ println e } class SqlFromFile { def sql_filename def read(){ result=...

Print lines in a csv file if the values are consecutive

I have the following csv file: Value1,Value2,Value3,Value4 11,12,27,28 5,6,101,102 111,112,55,56 1,7,33,34 3,4,55,57 I want to print lines if Value1 & Value2 are consecutive AND Value3 & Value4 are consecutive. The desired answer would be: 11,12,27,28 5,6,101,102 111,112,55,56 I tried something like this but it didn't work f = open('test.csv', 'rU')...

Entity framework to CSV float values

I'm trying to write an object to csv, the thing is that my objects have float valure for exemple (14,9) i want to change them to (14.9) so it won't cause any problem with the csv format string csv = ""; using (var ctx = new NBAEntities2()) { var studentList...

Saying there are 0 arguments when I have 2? Trying to open a CSV file to write into

I'm trying to read from a CSV file and codify people into groups using an equation. I append the name of their group they fall into to the end of the array that their row creates. Then I write it to a new file so I don't overwrite the original...

create multidimensional associative array from CSV in PHP

I'm trying to create a multidimensional array in PHP where the inner arrays are associative for the following example CSV string $csv: # Results from 2015-06-16 to 2015-06-16. date,time,label,artist,composer,album,title,duration 2015-06-16,12:00 AM,Island,U2,"Clayton- Adam,The Edge,Bono,Mullen- Larry- Jr",Songs Of Innocence,SONG FOR SOMEONE,03:46 2015-06-16,12:04 AM,Lowden Proud,"Fearing & White, Andy White, Stephen Fearing","White- Andy,Fearing- Stephen",Tea...

Groovy - timestamp from minutes

I have an array or times/values coming back to be in an array like: [0, 60] Which are times in minutes, 0 = 12:00 a.m, 60 = 1:00 a.m. I am wanting to store these in an oracle database as timestamps. How do I convert minutes into timestamps in groovy?...

Spock's @Narrative and @Title annotations

Spock provides @Narrative and @Title annotations that you can use to provide a class-level description of your test, e.g. @Narrative('description of spec') @Title('title of spec') class ExampleSpec extends Specification { // tests omitted } What is the difference between these two? In other words, why do we need both of...

Specific rows from CSV as dictionary and logic when keys are the same - Python

abc11 bvc ex 123 456 somestuffhere abc11 bvc ex 456 476 somestuffhere abc12 bvc ex 173 426 somestuffhere abc12 bvc ex 426 496 somestuffhere abc13 bvc ex 143 796 somestuffhere abc13 bvc ex 743 896 somestuffhere I am trying to put the above CSV file as a dictionary, {'abc11':['123','476'],'abc12':['173','496'],'abc13':['143','896']}. I...

Perl: Using Text::CSV to print AoH

I have an array of hashes (AoH) which looks like this: $VAR1 = [ { 'Unit' => 'M', 'Size' => '321', 'User' => 'test' } { 'Unit' => 'M' 'Size' => '0.24' 'User' => 'test1' } ... ]; How do I write my AoH to a CSV file with separators,...

Special Groovy magic re property access and collections / iterables?

I understand what is happening here with the spread operator *. in Groovy (2.4.3): [].class.methods*.name => [add, add, remove, remove, get, ... But why does the leaving the * out produce the same results? [].class.methods.name => [add, add, remove, remove, get, ... I'd have expected that to be interpreted as...

BASH - conditional sum of columns and rows in csv file

i have CSV file with some database benchmark results here is the example: Date;dbms;type;description;W;D;S;results;time;id Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;570;265;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;420;215;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;500;365;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;530;255;50 Mon Jun 15 14:22:20 CEST 2015;hsql;on-disk;text;2;1;1;870;265;99 Mon Jun 15 14:22:20 CEST 2015;hsql;on-disk;text;2;1;1;620;215;99...

Confused about the invokeMethod method in the Groovy MOP

First look at the following Groovy code: class Car { def check() { System.out.println "check called..." } def start() { System.out.println "start called..." } } Car.metaClass.invokeMethod = { String name, args -> System.out.print("Call to $name intercepted... ") if (name != 'check') { System.out.print("running filter... ") Car.metaClass.getMetaMethod('check').invoke(delegate, null) } def validMethod...

How to get testStep responseAsXml in groovyScript

Concerning soapUI and groovy, I'm trying to get assertion (working) and response both in XML into a variable. I get the error groovy.lang.MissingMethodException: No signature of method: com.eviware.soapui.impl.wsdl.teststeps.WsdlTestRequestStep.getResponseAsXml() is applicable for argument types: () values: [] error at line: 6 I have tried adding import com.eviware.soapui.impl.wsdl.teststeps.WsdlTestRequestStep but still cant figure...

Read CSV and plot colored line graph

I am trying to plot a graph with colored markers before and after threshold value. If I am using for loop for reading the parsing the input file with time H:M I can plot and color only two points. But for all the points I cannot plot. Input akdj 12:00...

CSV File header part is coming parsing by LINQ

the below way i am parsing csv file by LINQ but i found header part is coming when i inspect user class data. what is wrong there in code. var csvlines = File.ReadAllLines(filename); // IEnumerable<string> var csvLinesData = csvlines.Select(l => l.Split(',').Skip(1).ToArray()); // IEnumerable<string[]> int flag = 0; var users =...

Exporting Data from Cassandra to CSV file

Table Name : Product uid | productcount | term | timestamp 304ad5ac-4b6d-4025-b4ea-8b7991a3fe72 | 26 | dress | 1433110980000 6097e226-35b5-4f71-b158-a1fe39a430c1 | 0 | #751104 | 1433861040000 Command : COPY product (uid, productcount, term, timestamp) TO 'temp.csv'; Error: Improper COPY command. Am I missing something? ...

Is it possible to output to a csv file with multiple sheets?

I need to output data to a CSV file from Java, but in that csv file I hope to create multiple sheets so that data can be organized in a better way. After some googling, it seems this is not possible. A CSV file can only receive one-sheet data. Is...

Replace improper commas in CSV file

This may have been asked before, but I couldn't find it. I have a list of CSV files (439 or so) where, in a few of the files, someone also used commas in editorial comments. The result is that I can't put the files into a data frame, since the...

Parse text from a .txt file using csv module

I have an email that comes in everyday and the format of the email is always the same except some of the data is different. I wrote a VBA Macro that exports the email to a text file. Now that it is a text file I want to parse the...

Compare 2 seperate csv files and write difference to a new csv file - Python 2.7

I am trying to compare two csv files in python and save the difference to a third csv file in python 2.7. import csv f1 = open ("olddata/file1.csv") oldFile1 = csv.reader(f1) oldList1 = [] for row in oldFile1: oldList1.append(row) f2 = open ("newdata/file2.csv") oldFile2 = csv.reader(f2) oldList2 = [] for...

How to collect data from text file to dict in Python?

I have the following table generated in a text file,"fasta.txt" A C G T 0 0.195965417867 0.322766570605 0.35446685879 0.126801152738 A1 0.25 0.1875 0.3125 0.25 C1 0.25 0.475 0.225 0.05 G1 0.135135135135 0.243243243243 0.405405405405 0.216216216216 T1 0.142857142857 0.285714285714 0.285714285714 0.285714285714 A2 0.125 0.208333333333 0.625 0.0416666666667 C2 0.0833333333333 0.416666666667 0.305555555556 0.194444444444 G2...

export mysql query array using fputcsv

I am trying to export the results of a sql query to CSV using fputcsv however keep getting the error "fputcsv() expects parameter 2 to be array, string given". I followed the answer given here - Query mysql and export data as CSV in PHP - but it throws the...

Write Spock test cases for Spring boot application

I am working on spring boot application. I have to write test cases for it. I haven't written test cases before, so someone suggested using spock framework for it. I explored spock and i think it is more related to groovy language. Can i write spock test cases for my...

How to rearrange CSV / JSON keys columns? (Javascript)

I am converting a JSON object array to CSV using Papa Parse JavaScript Library. Is there a way to have the CSV columns arranged in a certain way. For e.g; I get the column as: OrderStatus, canOp, OpDesc, ID, OrderNumber, FinishTime, UOM, StartTime but would like to be arranged as:...

Python CSV reader/writer handling quotes: How can I wrap row fields in quotes? (Getting triple quotes as output)

I have a problem with the csv reader and writer in python. Whenever I try to take one CSV file and par down the number of columns from roughly 37 to 6, this is the kind of output I am getting. Example of one row: 0,"JOHNSON, JOHN J.",JOHN J. JOHNSON,TECH879,INSPECTION...

How can i add two tables in excel csv file?

I am creating excel CSV file using the following code its working fine for me generating the first image table.But i want to generate the CSV for the image second.Can anyone please tell me how can i add two tables in single file and how can i insert record where...

How to process the irregular json file in python to get the serialized values in CSV?

{\n \"1 & 1 Internet\": {\n \"category\": \"Infrastructure\",\n \"xyz\": 55,\n \"abc\": \"low\"\n },\n \"1 website hosting\": {\n \"category\": \"Infrastructure\",\n \"xyz\": 0,\n \"abc\": \"poor\"\n },\n \"10000ft\": {\n \"category\": \"Collaboration\",\n \"xyz\": 48,\n \"abc\": \"poor\"\n },\n \"1010data\": {\n \"category\": \"Big Data\",\n \"xyz\": 56,\n \"abc\": \"low\"\n },\n \"101domains\": {\n \"category\": \"Infrastructure\",\n \"xyz\": 0,\n \"abc\":...

Compare 2 csv files and output different rows to a 3rd CSV file using Python 2.7

I am trying to compare two csv files and find the rows that are different using python 2.7. The rows are considered different when all columns are not the same. The files will be the same format with all the same columns and will be in this format. oldfile.csv ID...

Convert strings of data to “Data” objects in R [duplicate]

This question already has an answer here: as.Date with dates in format m/d/y in R 2 answers My problem is that the as.Date function does not convert the values in a "date" column of a data frame into Date objects. I have a data.frame nmmaps. Here is a short...

Convert delimited string to array and group using LINQ in C#

I have a string that has a delimited format like this: orgname: firstname lastname, firstname lastname; (this can repeat with orgnames and variable number of names for each org) Example: **XXX University**: Martha Zander, Rick Anderson; **Albert School**: Nancy Vanderburg, Eric Towson, George Branson; **Hallowed Halls**: Jane Goodall, Ann Crabtree,...

How to stop foreach loop from printing duplicate data?

I am trying to generate unique CSV files from the csv data that I have using the following loop. $k =1; foreach ($csv_tbl as $_csv) { $filename = "Agent_" . $k . ".csv"; $file_path = "agents/$filename"; file_put_contents($file_path, $_csv); if (file_exists($_csv)) { header('Content-Description: File Transfer'); header('Content-type: text/csv'); header('Content-Disposition: attachment; filename=' ....

Resampling and merging data frame with python

Hi I have created a dictionary of dataFrame with this code import os import pandas import glob path="G:\my_dir\*" dataList={} for files in glob.glob(path): dataList[files]=(read_csv(files,sep=";",index_col='Date')) The different dataframe present in the dictory have different time sample. An example of dataFrame(A) is Date Volume Value 2014-01-04 06:00:02 6062 108000.0 2014-01-04 06:06:05 6062...

Panda's Write CSV - Append vs. Write

I would like to use pd.write_csv to write "filename" (with headers) if "filename" doesn't exist, otherwise to append to "filename" if it exists. If I simply use command: df.to_csv('filename.csv',mode = 'a',header ='column_names') The write or append succeeds, but it seems like the header is written every time an append takes...

Group instances based on NA values in r

I am reading a csv file and unfortunately my dataframe has many missing values. A small snip is as following: df <- data.frame(Size= c(800, 850, 1100, 1200, 1000), Value= c(900, NA, 1300, 1100, NA), Location= c(NA, 'midcity', 'uptown', NA, 'Lakeview'), Num1 = c(2, NA, 3, 2, NA), Num2 = c(2,3,3,1,2),...

Spock Framework: problems with spying

I have an issue with using Spy in Spock, it either doesn't work as it should or my understanding is wrong so I'm trying to clarify this. Consider this code (Java): public class CallingClass { public String functionOne() { //does stuff return "one"; } public String functionTwo() { String one...

groovy/XML: Replace a node by another one

I try to replace an XML node by another one by using XmlSlurper (or XmlParser). The original XML: <myXml> ... <myNode> <Name>name1</Name> <Name>name2</Name> <Name>name3</Name> </myNode> ... </myXml> The list that contains the items to build my new node def namelist = ['name4','name5','name6','name7'] What I want to have <myXml> ... <myNode>...

Addition of two dates on python 3

I try adding date and hours from csv file in one datetime variable. I read questions about adding some timedelta and official doc https://docs.python.org/3/library/datetime.html#timedelta-objects, but don't understend how it works. My csv row looks like - ['2005.02.28', '17:38', '1.32690', '1.32720', '1.32680', '1.32720', '5'].I convert row[0] = 2005.02.28 to date and...

How can I use a variable to get an Input$ in Shiny?

I am new to R and I am creating a shiny application to read a csv and filter data. I am reading the csv file, then creating dropdowns with a loop using the column names and the unique values: output$dropdowns <- renderUI({ if (is.null(x())) { return(NULL) } lapply(1:ncol(x()), function(i) {...

Run 3 variables at once in a python for loop.

For loop with multiple variables in python 2.7. Hello, I am not certain how to go about this, I have a function that goes to a site and downloads a .csv file. It saves the .csv file in a particular format: name_uniqueID_dataType.csv. here is the code import requests name =...

type conversion performance optimizable?

The following snippet converts xml data to csv data in a data processing application. element is a XElement. I'm currently trying to optimize the performance of the application and was wondering if I could somehow combine the two operations going on below: Ultimately I still want access to the string...

How to pivot array into another array in Ruby

I have a multidimensional array like this one : myArray = [["Alaska","Rain","3"],["Alaska","Snow","4"],["Alabama","Snow","2"],["Alabama","Hail","1"]] I would like to end up with CSV output like this. State,Snow,Rain,Hail Alaska,4,3,nil Alabama,2,nil,1 I know that to get this outputted to CSV the way I want it I have to have output array like this: outputArray =[["State","Snow","Rain","Hail"],["Alaska",4,3,nil],["Alabama",2,nil,1]]...

Grails JAX-RS Calling a class in src/groovy giving error - Message: No signature of method: is applicable for argument types

While developing the rest api using jaxrs plugin I need to create some common class which I have created in "src/groovy". Below is the class class ValidateToken { String validate(String token){ println(token) return "test" } //... In resource file(jaxrs) this is what I am doing def instance=ValidateToken.validate("test") This throws error...

How Do I Transform This CSV / Tabular Data Into A Different Shape?

I have a sparse n-column spreadsheet where the first two columns describe a person and the rest of the (n-2) columns track RSVP and attendance data for various events (each of which take up one column). It looks like this: PersonID, Name, Event29108294, Event01838401, Event10128384 12345, John Smith, Registered -...

sonar maven analysis only picks .java file

I am trying to run a sonar maven analysis on my multilanguage project which contains many languages like *.java, *.groovy, *.js etc. I have installed all the languages plugin in my sonar and configured my pom sonar.sources parameter as src/main,src/test but still it picks up only java files. In the...

Cron expression must consist of 6 fields (found 1 in “#{systemEnvironment['db_cron']}”)

I'm trying to set a cron Scheduled annotation as follows: @Scheduled(cron = "#{systemEnvironment['db_cron']}") def void schedule() { } Next set the environment variable as: export db_cron="0 19 21 * * *" However, I get the runtime error: Cron expression must consist of 6 fields (found 1 in "#{systemEnvironment['db_cron']}") What can...

Python: isolating re.search results

So I have this code (probably super inefficient, but that's another story) that is pulling urls from html code of a blog. I have the html in a .csv, which I am putting into python, then running the regex to get the urls. Here is the code: import csv, re...

Adding time/duration from CSV file

I am trying to add time/duration values from a CSV file that I have but I have failed so far. Here's the sample csv that I'm trying to add up. Is getting this output possible? Output: I have been trying to add up the datetime but I always fail: finput...

Create an external Hive table from an existing external table

I have a set of CSV files in a HDFS path and I created an external Hive table, let's say table_A, from these files. Since some of the entries are redundant, I tried creating another Hive table based on table_A, say table_B, which has distinct records. I was able to...

Model to LazyMap

How do I convert/deserialize these models public class AccessCredentials { String userName = '' String password = '' LoginOptions loginOptions = new LoginOptions() } public class LoginOptions { String partnerId = '' String applicationId = '' } into a LazyMap like : [ userName : userName, password : password, loginOptions...

Hive external table not reading entirety of string from CSV source

Relatively new to the Hadoop world so apologies if this is a no-brainer but I haven't found anything on this on SO or elsewhere. In short, I have an external table created in Hive that reads data from a folder of CSV files in HDFS. The issue is that while...