FAQ Database Discussion Community

## R, how to use pch with time series plot

r,time-series,lattice
I would like to plot a time series (meaning line graph with x axis as time) and specify a plotting character to use. None of the following has worked a1 = as.xts(ts(c(5,3,7,2,4,8,3), start=c(1980,1), freq=4)) library('lattice') xyplot(a1, col="red", pch=2) xyplot(a1, col="red", par.settings = list(superpose.symbol = list(col = 1, pch = 2)),)...

## Calculating the difference in dates in a Pandas GroupBy object

python,pandas,time-series
I have a Pandas DataFrame with the following format: In [0]: df Out[0]: col1 col2 date 0 1 1 2015-01-01 1 1 2 2015-01-09 2 1 3 2015-01-10 3 2 1 2015-02-10 4 2 2 2015-02-10 5 2 3 2015-02-25 In [1]: df.dtypes Out[1]: col1 int64 col2 int64 date datetime64[ns]...

## Adding date tick marks to a Matlab plot

matlab,plot,time-series
I have a plot of time series data, and I would like to replace the tick marks of the x-axis (automatically I have the number of the ordered observations) with the date when the value is observed. I would like to have a tick mark every 5 years for example....

## R Nested For Loop for a Sensitivity Analysis

r,for-loop,nested,time-series
I'm fairly new to R and I've been trying for a while to do something, which I assumed to be very simple, but I keep failing at it (unfortunately for me, it doesn't mean it's not simple!). I have defined a function that takes a time series as an input...

## small line x-axis for years

r,time-series
I have data covering a time period of over 25 years. In the data set are over 6300 days. I would like to show the years on the x-axix. dates <- as.Date(DOL[,1], "%d.%m.%Y") DOL <- as.xts(DOL[,2], dates) plot(DOL, xaxt = "n", main="SMA", ann = FALSE) axis(1, at=as.POSIXct(dates),labels=format(dates,"%Y"),tick=TRUE) title(ylab = "Value")...

## How to use parameters from data frame in R and loop through time holding them constant

r,nested,time-series,lapply,sapply
I have a function (weisurv) that has 2 parameters - sc and shp. It is a function through time (t). Time is a sequence, i.e. t<-seq(1:100). weisurv<-function(t,sc,shp){ surv<-exp(-(t/sc)^shp) return(surv) } I have a data frame (df) that contains a list of sc and shp values (like 300+ of them). For...

## R conditionally matching date-time from one dataframe to closest date-time field in second dataframe

r,datetime,merge,data.frame,time-series
I have two data frames, df.events and df.activ. df.activ has very granular minute level data and an order of magnitude more records (1,000,000+) than df.events which has ~100,000 records, also at minute level granularity. The two dataframes have two common fields, DateTime and Geo. Both DateTime columns are in as.POSIXlt,...

## Use Python/Pandas to Match Sample Pairs Yearly Data

python,pandas,match,time-series,multi-index
While this may start out sounding like as statistics question, please bear with me. I have several calcium concentrations from water samples collected at different sampling locations. The water is resampled at some of the stations on a monthly, yearly, every other year basis. I want to measure yearly and...

## ggplot vertical line with date axis R

r,ggplot2,time-series,vline
I'm having problems ploting vertical lines with ggplot in R. I want to draw a vertical line each Sunday on my time serie: VisitDate VisitMonth VisitYear City Weekday VisitWeek Code_CxF Centre Location 1 2014-05-02 05 2014 Barcelona 05Friday 2014-04-28 CxF_BCN CaixaForum Barcelona Catalunya 2 2014-05-03 05 2014 Barcelona 06Saturday 2014-04-28...

## Time series forecasting use SVM

python,time-series
I am trying to set-up a python code for forecasting a time-series, using SVM libraries of scikit-learn. My data consists of X values at a day interval for the last one years, and I need to predict y for a month of the next year . Here's what I have...

## Cassandra Time-Series: Allow Filtering, Buckets, or Other

database,cassandra,time-series,data-modeling,cql
I know there are many time-series questions on here but mine does not seem to comfortably fit with the given solutions. I am also new to Cassandra so I might be approaching this with the wrong mindset. Bear with me. I am receiving search data in the form: datetime_searched, term_used,...

## geom_vlines multiple vlines per plot

r,ggplot2,time-series,timeserieschart
How can I get ggplot to produce something similar like library(ggplot2) library(reshape2) library(ecp) synthetic_control.data <- read.table("/path/synthetic_control.data.txt", quote="\"", comment.char="") n <- 2 s <- sample(1:100, n) idx <- c(s, 100+s, 200+s, 300+s, 400+s, 500+s) sample2 <- synthetic_control.data[idx,] df = as.data.frame(t(as.matrix(sample2))) #calculate the change points changeP <- e.divisive(as.matrix(df[1]), k=8, R = 400,...

## Calculate the maximum price fluctuation in a 24 hour window

r,time-series
I have a data frame that contains two columns - time and price. It contains a series of observations for price of a certain item at various times. Here is a sample. > df time price 1 2014-12-12 14:57:15 45.81 2 2014-12-12 14:57:15 45.90 3 2014-12-12 15:00:08 45.76 4 2014-12-12...

## time series data in mongodb - how to query embedded document

mongodb,time-series
I'm working on a design for a time series dataset, basically I have servers I monitor, and I would like to know some metric about it over a period of time. based on http://blog.mongodb.org/post/65517193370/schema-design-for-time-series-data-in-mongodb I created a design that is a document per server+month, and in it an embedded document...

## R, lag( ) has inconsistent behavior for xts and ts objects

r,time-series,xts
I would like to take a lag of an xts variable, and the lag() function returns a lag. However, if I use it on a ts variable, it gives a lead. Is this a bug, or working as intended? library('xts') a = as.xts(ts(c(5,3,7,2,4,8,3), start=c(1980,1), freq=4)) cbind(a, lag(a)) # provides lag...

## seas start year error R

r,time-series
I am currently trying to deseasonalize data for ARIMA models using the U.S. Census Bureau's package seasonal install.packages("seasonal") you can find a modified version of the data set here. also there is a vignette of the package here which gives very straight-forward directions as to how to install and specify...

## How to create a function and a loop to calculate growth rates of variables in a data frame in R

r,time-series,growth-rate
New to R and Stack Overflow. Suppose I have the following macroeconomic data loaded into a data frame called testdata in R. > testdata date gdp cpi_index rpi_index 21 2013 Q1 409985 125.067 247.4 22 2013 Q2 412620 125.971 249.7 23 2013 Q3 415577 126.352 250.9 24 2013 Q4 417265...

## Converting time series to data frame, matrix, or table

r,time-series,tapply
With reference to this question: transforming a ts in a data.frame and back I have a list of monthly averages that start in May 2012 and go through May 2015. It looks like this initially: head (AVG_LOSCAT2) month AVG_LOSCAT YEAR MONTH 1 2012-05 5.342066 2012 05 2 2012-06 6.544096 2012...

## Matlab's VAR[X] coefficient constraints for vector time series

matlab,time-series
Matlab's VARMAX model allows the user to set flags that determine whether individual linear coefficients are to be estimated. In particular, vgxset accepts an ARsolve parameter containing flags that determine whether individual time series lag coefficients are estimated. The fact that there are individual scalar flags for each scalar lag...

## how to convert data frame into time series in R

r,time-series
I have one csv file in which I have 2 closing prices of stock(on daily basis) Dates Bajaj_close Hero_close 3/14/2013 1854.8 1669.1 3/15/2013 1850.3 1684.45 3/18/2013 1812.1 1690.5 3/19/2013 1835.9 1645.6 3/20/2013 1840 1651.15 3/21/2013 1755.3 1623.3 3/22/2013 1820.65 1659.6 3/25/2013 1802.5 1617.7 3/26/2013 1801.25 1571.85 3/28/2013 1799.55 1542 I...

## Deleting duplicates in a time series

sql-server,duplicates,time-series,sql-delete
I have a large set of measurements taken every 1 millisecond stored in a SQL Server 2012 table. Whenever there are 3 or more duplicate values in some rows that I would like to delete the middle duplicates. Highlighted values in this image of sample data are the ones that...

## R: HAC by NeweyWest using dynlm

r,time-series,regression

## Replace list of permutations with getSymbols data in R

r,time-series,permutation,quantmod
I downloaded some stock data: require("quantmod") s <- c("AAPL", "ADBE", "ADI", "ADP", "ADSK") e <- new.env() getSymbols(s, src='yahoo', from='2015-01-10', env = e ) #get closing prices close <- do.call(merge, eapply(e, function(x) Cl(x))) I found all the pairs of symbol names: #find all the pairwise permutations perm<-combn(s,2) perm [,1] [,2] [,3]...

## Pandas Dataframe Plot

python,pandas,time-series,ipython-notebook
I am trying to plot a time series data. The dataframe looks like this [1]:Index ship_date cost_amount 0 1/8/2010 34276 1 1/8/2010 12375 2 1/8/2011 12343 3 2/9/2011 15435 [2]: df1.plot(figsize(20,5)) I am trying to plot the data but for some reason plot doesn't have x-axis in ascending order. How...

## How to get a column of lists of contract payout pairs (date:amounts)?

r,time-series,dplyr
I'm trying to add a column of list objects to a data.frame of payments built like ID <- c("A", "B", "B", "c", "A", "B", "c", "c", "A", "B") Date = seq(as.Date("2000/07/01"), as.Date("2000/07/10"), "days") Amt <- rnorm(10, 10, 3) E <- data.frame(Date = Date, ID = ID, Amt = Amt) Date...

## Matlab: trying to estimate multifractal spectrum from time series by histogram box-counting

matlab,statistics,time-series,histogram,fractals
I am using the approach from this Yale page on fractals: http://classes.yale.edu/fractals/MultiFractals/Moments/TSMoments/TSMoments.html which is also expounded on this set of lecture slides (slide 32): http://multiscale.emsl.pnl.gov/docs/multifractal.pdf The idea is that you get a dataset, and examine it through many histograms with increasing numbers of bars i.e. resolution. Once resolution is high...

## Plotting Probability Density Heatmap Over Time in R

r,plot,time-series,kriging
Let's say I have the output of a monte-carlo simulation of one variable over several different iterations (think millions). For each iteration, I have the values of the variable at each point in time (ranging from t=1 to t=365). I would like to produce the following plot: For each point...

## Calculate days since last event in R

r,time-series
My question involves how to calculate the number of days since an event last that occurred in R. Below is a minimal example of the data: df <- data.frame(date=as.Date(c("06/07/2000","15/09/2000","15/10/2000","03/01/2001","17/03/2001","23/05/2001","26/08/2001"), "%d/%m/%Y"), event=c(0,0,1,0,1,1,0)) date event 1 2000-07-06 0 2 2000-09-15 0 3 2000-10-15 1 4 2001-01-03 0 5 2001-03-17 1 6 2001-05-23...

## Cassandra storage internal

cassandra,apache-spark,time-series,cql
I'm trying to understand what exactly happens internally in storage engine level when a row(columns) is inserted in a CQL style table. CREATE TABLE log_date ( userid bigint, time timeuuid, category text, subcategory text, itemid text, count int, price int, PRIMARY KEY ((userid), time) - #1 PRIMARY KEY ((userid), time,...

## Creating a running counting variable in R?

r,time-series,running-total
I have a dataset of soccer match results, and I am hoping to learn R by creating a running set of ratings similar to the World Football Elo formula. I am running into trouble with things that seem to be simple in Excel aren't exactly intuitive in R. For instance,...