FAQ Database Discussion Community


Form new dataframes from list by combining similar entries in all dataframes

r,list,lapply,cbind,rbindlist
I have a list with many dataframes (example provided below). G100=structure(list(Return.Period = structure(c(4L, 6L, 2L, 3L, 5L, 1L), .Label = c("100yrs", "10yrs", "20yrs", "2yrs", "50yrs", "5yrs"), class = "factor"), X95..lower.CI = c(54.3488053692529, 73.33363378538, 84.0868168935697, 91.6191228597281, 96.3360349026068, 95.4278817251266), Estimate = c(61.6857930414643, 84.8210149260708, 101.483909733627, 118.735593472652, 143.33257990536, 163.806035490329 ), X95..upper.CI = c(69.0227807136758, 96.3083960667617,...

Create multiple lattice plots from a data table using lapply in R

r,for-loop,plot,lattice,lapply
I am trying to generate mean values by group for each of numerous variables (species) and then plot each of these separately. I have tried list and data table formats. The base plot function works in a for loop: library(data.table) for (i in 3:5) { # generate a list of...

How to recombine values after a split in R?

r,split,lapply
I have a data variable X that I have done the following to Xnew = split(X$col1,list(X$col3,X$col4)) S = sapply(Xnew,mean) I now have a vector where each element can be accessed by S['SomeValCol3.SomeValCol4'] Now I would like to create a vector containing columns equal to the number of unique values in...

Simplify conditional table loop without matrix notation in R

r,data.table,dplyr,lapply
Using the example below, I was wondering if there is a more efficient package or function to do conditional counting and tables on matching string elements--e.g, using the data.table package, dplyr package, lapply() like function? produce = c("apple", "blueberry", "blueberry", "corn", "horseradish", "rutabega", "rutabega", "tomato") # Long list veggies =...

R: Weight parameter in array based on other parameters

r,lapply,weight,sapply
I've got this table/array in csv: GroupID Channel Daysbeforelast 1 A 35 1 B 31 1 C 29 1 D 17 1 E 15 1 D 5 1 C 0 2 B 66 2 E 17 2 D 15 2 A 2 2 C 0 2 F 0 2 A...

Apply a list of n *expressions* to each row of a dataframe?

r,apply,lapply,mapply
In short, I have a list of expressions that I want to apply to each row of a dataframe. This is very similar to this question, but there is a subtle difference in that I do not have a list of functions, but have a list of expressions. Here's what...

Creating a dataframe from an lapply function with different numbers of rows

r,data.frame,lapply
I have a list of dates (df2) and a separate data frame with weekly dates and a measurement on that day (df1). What I need is to output a data frame within a year prior to the sample dates (df2) and the measurements with this. eg1 <- data.frame(Date=seq(as.Date("2008-12-30"), as.Date("2012-01-04"), by="weeks"))...

Apply function over columns of dataframe in R, compile results

r,for-loop,apply,lapply
I've searched here and on Google and haven't found an answer that I can apply to my situation. Lets say I have a dataframe with columns for Element 1, Element 2, Element 3, Metric, Other. I have another internal function that has three arguments (input_dataframe, element_position, metric_position) that I use...

Updating only certain values of data frame based on match

r,match,lapply
I'm trying to update a variable (popsnp) in a higher scope within an lapply, on the basis of a match. I can't quite figure out the syntax for updating the values though, what I have currently overwrites any previously existing values with NA: lapply(1:22, function(i){ in.name<-paste("/data/mdp14aps/ld/chr", i, ".ld", sep="") out.name<-paste("/data/mdp14aps/R/ldatachr",...

How to merge elements of dataframes of different sizes containded in a list [R]?

r,merge,lapply
I have a list composed by 5 data.frames of different sizes, each data.frame has two variables, Group.1 and x, i would like to merge those data.frames by the Group.1 variables. Of course I expect NA values to be generated. Example: Group.1 = c(01, 03, 05) x = c(2000, 4000, 5000)...

Cumulative sum of numeric vectors in list

r,vector,add,lapply,sapply
I hope someone will be able to help me on this problem. I have a list object that includes 48 vectors and each vector has a length of 2,000,000 observations in it. Here is a code that creates the same structure with only 100,000 items per vector: mtx_sim <- matrix(data...

Use lapply on a subset of list elements and return list of same length as original in R

regex,r,plyr,lapply
I want to apply a regex operation to a subset of list elements (which are character strings) using lapply and return a list of same length as the original. The list elements are long strings (derived from reading in long text files and collapsing paragraphs into a single string). The...

Simplify extraction of list of data frame

r,data.frame,lapply
My question is a continuation of the following question. (Couldn't comment on that thread due to reputational constraints) Print the Nth Row in a List of Data Frames I want the results to be printed as a data frame and not as a list (assuming I have multiple columns and...

lapply undoes my previous function, but I don't know why

r,lapply
I want to split a dataframe by the year column into a list of 33 different dataframes. Then I want to change it so that each year column in the list reads "2015". Then I want to write each dataframe to a file with files labeled by the name of...

lapply() emptied list step by step while processing

r,null,scope,environment-variables,lapply
First of all, excuse me for the bad title. I'm still so confused about this behavior, that I wasn't able to describe it; however I was able to reproduce it and broke it down to an (goofy) example. Please, could you be so kind and explain why other.list appears to...

Extract a List of values Filtered by criteria

r,dplyr,lapply
I have a list of data some rows need to be filtered. I have some criteria to extract those rows which I called them crit. For each crit that qualifies data, I want to get sub-set of data as output. Sometimes there is a set of functions that can extract...

apply, sapply and lappy return NULL

r,apply,lapply,sapply
I have a matrix: mat <- matrix(c(0,0,0,0,1,1,1,1,-1,-1,-1,-1), ncol = 4 , nrow = 4) and I apply the following functions to filter out the columns with only positive entries, but for the columns that have negative entries I get a NULL. How can I suppress the NULLs from the output...

Using magrittr and lapply to divide a column in each df in a list by a list of values

r,lapply,magrittr
I have a list of dataframes containing different time series of different lengths. I want to summarize the count of a variable and then normalize it by the number of years of data that is contained in that particular dataset. so with a sample dataframe: data_list <- list(data.frame(temp_bin = rep(1:4,...

How to append value within nested lapply/sapply function?

r,lapply,sapply
Is there a reason why I can't append values to an empty vector when called within a nested lapply/apply function? I have an empty vector bucket where I'd like to push values into, however, the output says the bucket is reinitialized with each iteration. I would appreciate any insight into...

Assigning a value for each dataset and plot it with ggplot

r,ggplot2,lapply,assign,seq
It can be complicated problem but I will try to explain it as much as I could. I have a sequenced dataset containing number of observations which should be processed before plotting. I want do it with a function. because I have another datasets in the directory (around 20) #reproducible...

In “R” iterate over specific columns

r,loops,for-loop,rstudio,lapply
Update: Thanks jason and Buckminster - I used a variation of your suggestions I used the below then adjusted for my function/data Again thanks myFun<-function(x) { myDF$multiple[grep(" Mbps",myDF[,x])] <- 1000000 myDF[,x] <- gsub(" Mbps","",myDF[,x]) myDF$multiple[grep(" Kbps",myDF[,x])] <- 1000 myDF[,x] <- gsub(" Kbps","",myDF[,x]) myDF$multiple[grep(" bps",myDF[,x])] <- 1 myDF[,x] <- gsub(" bps","",myDF[,x])...

merging matrices with cbind and match: name of the last column is dropped

r,matrix,merge,lapply,cbind
I have a list of matrices: [[1]] a b X2005.06 NA 2504179 X2006.06 NA 3746905 X2007.06 NA 5468607 X2008.06 NA 6664545 X2009.06 NA 7339310 X2010.06 4602635 5587625 X2011.06 5188408 6498543 X2012.06 5190519 6831415 X2013.06 5246169 8214118 X2014.06 5856140 9384405 [[2]] c d X2005.06 354981 1729284 X2006.06 423136 1902213 X2007.06 602389...

How to assign new values from lapply to new column in dataframes in list

r,data.frame,lapply
I have a list of dataframes. I want to perform an operation on columns of the dataframes and then create a new column in the dataframes with the resulting new column. a <- data.frame(c(1,2,3), c(2,3,4)) b <- data.frame(c(7,8,9), c(5,6,2)) l <- list(a, b) lapply(l, function(x) x[,2]*2) What I want is...

generating and filling new data frames in lapply-“do not know how to convert x to class POSIXlt”

r,lapply
I am trying to generate a new data frame containing weekly encounter histories for an animal based on one row in a data frame that contains the animal ID (BandNo) first and last day we tracked it (FDay, Lday), and the fate of the animal when we stopped tracking it...

Using lapply with Dates and the minus function

r,lapply
I have a vector of dates and a vector of number-of-days. dates <- seq(Sys.Date(), Sys.Date()+10, by='day') number.of.days <- 1:4 I need to get a list with an entry for each number-of-days, where each entry in the list is the dates vector minus the corresponding number-of-days, ie., list(dates-1, dates-2, dates-3, dates-4)...

Aggregate variables separetly in R [lapply + aggregate]

r,aggregate,lapply
I have a data.frame with a set of records and as variables different measurements. I would like to create a new data.frame containing the amount of records having a specific measurement value for each measurement. Basically what I am trying to do is: record <- c("r1", "r2", "r3") firstMeasurement <-...

Apply confusionMatrix() to Elements of a Split List in R

r,matrix,elements,lapply
I'm searching for a solution to apply the confusionMatrix() function from {caret} to specific elements of a split list. I have 3 Groups, with each group having 10 observations of Actuals and 3 Preds columns. library(caret) set.seed(10) dat <- data.frame(Group = c(rep(1, 10), rep(2, 10), rep(3, 10)), Actual = round(runif(30,...

lapply(makeCacheMatrix(a),solveCache) does not work fine

r,lapply
I am facing a strange issue. Could someone help where I am going wrong. I am a noob at R. What I have known about lapply is it applies a function over a list. Now, when I do this, lapply(makeCacheMatrix(a),solveCache) it gives an error Error in x$getInverse : object of...

Write a list, as seen in R console output, into a text file

r,cat,lapply
I have problem with writing a list into a text file in r. Here is my code: library(e1071) mydata = read.table("TRAIN.txt", sep = ",", header = FALSE) model <- naiveBayes(as.factor(V1) ~., data = my data) and I want to write the "model" into a text file. Here is the "model"...

cbind two lists of data.frames to a new list [duplicate]

r,data.frame,lapply,cbind
This question already has an answer here: Combine two lists of dataframes, dataframe by dataframe 1 answer I have two lists of data.frames. Both lists have the same length and contain fitting data.frames in their according list elements. So the scenario looks like this dfa = data.frame(a=1:3, b =...

changing for loop to lapply over several files

r,for-loop,dataframes,lapply
I have many csv files and I need to read all of them and perform some operations in the variables. I have used a “for loop”, but it is taking too long. I have searched many answers here and I know that lapply would be much more efficient, but I...

How to use parameters from data frame in R and loop through time holding them constant

r,nested,time-series,lapply,sapply
I have a function (weisurv) that has 2 parameters - sc and shp. It is a function through time (t). Time is a sequence, i.e. t<-seq(1:100). weisurv<-function(t,sc,shp){ surv<-exp(-(t/sc)^shp) return(surv) } I have a data frame (df) that contains a list of sc and shp values (like 300+ of them). For...

Empty rows in list as NA values in data.frame in R

r,list,lapply,na,rbind
I have a dataframe as follows: hospital <- c("PROVIDENCE ALASKA MEDICAL CENTER", "ALASKA REGIONAL HOSPITAL", "FAIRBANKS MEMORIAL HOSPITAL", "CRESTWOOD MEDICAL CENTER", "BAPTIST MEDICAL CENTER EAST", "ARKANSAS HEART HOSPITAL", "MEDICAL CENTER NORTH LITTLE ROCK", "CRITTENDEN MEMORIAL HOSPITAL") state <- c("AK", "AK", "AK", "AL", "AL", "AR", "AR", "AR") rank <- c(1,2,3,1,2,1,2,3) df...

Rename columns for matrices nested inside a list in R

r,matrix,lapply
I have a list that contains several matrices. The rows in each matrix are unique, but the columns represent variables that are common across each matrix. The example below provides an example of the type of data I have: mat1 <- matrix(sample(1:100, 10, replace=TRUE), 2, 5) mat2 <- matrix(sample(1:100, 15,...

Have lapply continue even after encountering an error using getSymbols from quantmod [duplicate]

r,error-handling,xts,lapply,quantmod
This question already has an answer here: quantmod omitting tickers in getSymbols 1 answer I am downloading some information from yahoo finance using quantmod wrapped in an lapply statement: require(quantmod) tickers <- c("AAPL", "MSFT", "MKQ", "TSLA") quotes <- lapply(tickers,function(x) getSymbols(x, src="yahoo", from="2015-02-01", auto.assign=FALSE)) The ticker MKQ is made up...

r- if inside lapply

r,if-statement,lapply
I have a table like: dir E_numdir last2 1 a 1 1 2 PJE INDEPENDENCIA 96 5 96, 5 96, 5 3 PJE INDEPENDENCIA 96 5 96, 5 96, 5 4 B 34 VIVIENDA RECRE 53 00 34, 53, 00 34, 53, 00 5 SARMIENTO CASA DE GO 613 613...

How to loop dcast function in reshape package in R

r,loops,reshape,lapply
Being a relatively new R user, I have trouble with any looping functions. I have looked at many tutorials but the examples in them are usually very basic and therefore easy to execute. However I need create slightly more complex loops and am having a lot of trouble figuring out...

How do you invert function parameters for lapply in R?

r,parameters,lapply
I've been wondering for awhile: in R, what is the recommendend way to invert function parameters for order-dependent functions to allow apply-style processing on a list of items? For a simple example, say I have a list of numbers, and a single divisor. numerator = list(5, 10, 15, 250) denominator...

vectorized simulation in R

r,simulation,vectorization,lapply
I've written a simulation function in R. I'd like to do num simulations. Rather than using a for loop, I'm trying to use some sort of apply function, such as lapply or parallel::mclapply. lapply, as I'm currently using it, is failing. For example: # t1() is a generic example function...

Saving elements of a list as data.frames using R

r,list,save,lapply
How can I save each element of a list in a in a separate .RData file? Consider the following example: # Generating a list containing 3 matrices set.seed(1) mylist=list(M1=matrix(LETTERS[sample(1:26,9)],3),M2=matrix(LETTERS[sample(1:26,9)],3),M3=matrix(LETTERS[sample(1:26,9)],3)) mylist[1:2] # $M1 # [,1] [,2] [,3] # [1,] "G" "U" "W" # [2,] "J" "E" "M" # [3,] "N" "S"...

R loading multiple excel files and merge

r,loops,merge,excel-2007,lapply
I have many excel files that I need to load and merge into a single data frame. The script below works! However, before merging all files I want to paste each file name in a new column. library(gdata) library(tools) filelist <- list.files(pattern = "*\\.xlsx$") files = lapply(filelist, read.xls, header=TRUE) new...

Alternatives to a for loop with indexing - R

r,lapply
I am converting unstructured data into a long format and need to create an ID (grouping) variable. I want to assign an ID variable based on sets of values contained in another variable. More specifically, consider the following data set. set.seed(1234); x.1 <- rep(letters[1:5], 10) x.2 <- sample(c(0:10), 50, replace=TRUE)...

Performing column select over multiple dataframes

r,function,dataframes,lapply
I have looked around a lot for this answer, they get close but no cigar. I am trying to perform a selection of columns over multiple dataframes. I can do this and return a list, but I wish to preserve the dataframes in the global environment. I want to keep...

Lapply in a dataframe over different variables using filters

r,data.frame,lapply
I'm trying to calculate several new variables in my dataframe. Take initial values for example: Say I have: Dataset <- data.frame(time=rep(c(1990:1992),2), geo=c(rep("AT",3),rep("DE",3)),var1=c(1:6), var2=c(7:12)) time geo var1 var2 1 1990 AT 1 7 2 1991 AT 2 8 3 1992 AT 3 9 4 1990 DE 4 10 5 1991 DE...

subset multiple data tables using lapply

r,lapply
I have multiple data tables and all have a common column called ID. I have a vector vec that contains a set of ID values. I would like to use lapply to subset all data tables using vec I understand how to use lapply to subset the data tables but...

Add values to data frames in list based on names

r,list,data.frame,lapply
I'm sure this must have been asked before, but I can't find the answer. I have a list of data frames, like so: x <- c(2, 3, 5) y <- c("a", "b", "c") df1 <- data.frame(x, y) x <- c(3, 6, 9) y <- c("a", "b", "c") df2 <- data.frame(x,...

R sub select from a list of vectors using do.call, lapply or mapply?

r,lapply,mapply,do.call
I have the following data. x is a list of vectors and indices is a list of indices. x = list(c("a", "b", "c", "a"), c("b", "x", "a", "c")) indices = list(c(1, 2), c(3, 4)) What I want to do is step through each vector represented within the list x and...

Using lapply to list percentage of null variables in every column in R

r,apply,lapply,mapply
I was given a large csv that is 115 columns across and 1000 rows. The columns have a variety of data, some is character-based, some is integer, etc. However, the data has a LOT of null variables of varying types (NA, -999, NULL, etc.). What I want to do is...

using a function with lapply to create a column and match values

r,function,match,lapply
I have two datasets H and G. They have a column named 'diff' that as the name suggests, holds difference between two columns within each dataset. I used lapply to calculate the percentage for each dataset (I have more datasets than H and G, so would like to calculate the...

R — For every element in a list, how can I set the same maximum value in each of j columns?

r,list,dictionary,lapply
In R, I have a list of i data frames. For j columns in each data frame, I want to recode values greater than 100 to 100. I can do this in a loop but am at a loss for how to do this using one of the apply functions....

removing and aggregating duplicates

r,duplicates,subset,lapply
I've posted a sample of the data I'm working with here. "Parcel.." is the main indexing variable and there are good amount of duplicates. The duplicates are not consistent in all of the other columns. My goal is to aggregate the data set so that there is only one observation...

Calculations on data frames in a list

r,list,data.frame,lapply
I have a list of data frames: str(Test) List of 3 $ A:'data.frame': 32400 obs. of 4 variables: ..$ X : num [1:32400] -0.0152 -0.0302 -0.0453 -0.0604 -0.0755 ... ..$ Y : num [1:32400] 0.00875 0.01745 0.02615 0.0349 0.0436 ... ..$ Z : num [1:32400] -1 -0.999 -0.999 -0.998 -0.996...

create boxplots with first element of first row of multiple dataframes

r,boxplot,lapply
I have a list of dataframes. Each dataframe has 6 rows. I want to create 6 boxplots. The first boxplot should take the values of the first row of the first column. The second boxplot should take the values of the second row of the first column, etc. I want...

R: rbind a list of data.frames with different columns in different data frames

r,lapply,rbind
I've a list of 11,383 data frames. I need to merge them into one big data frame, but the have different columns (2,3,4 columns) so when i use rbind_all from Dplyr i get not desired result. One way around would be to rbind data frames that have the same number...

Accessing items in a list using R, combining after split and apply

r,plyr,dplyr,lapply
I'm just getting started in the process of cleaning up strings with R and have learned a ton thus far. The portion of the current project that I'm stuck on is (I think) largely related to referencing values in a list. I'm trying the split-apply-combine approach and am having trouble...

Could you please help me to understand an R code?

r,function,matrix,lapply,cbind
I'm completely new to R. Would you please explain me the following code? Thank you a lot in advance. patterns <- matrix(c(0,0,0,1,0,1,0,1,1,1,1,0),4,3, byrow = T) input <- unlist(lapply(1:100, function(i) { ps <- patterns[sample(nrow(patterns)),] as.vector(t(ps)) })) teach <- c(input[-1],input[1]) data <- cbind(input, teach) ...