FAQ Database Discussion Community

## Pandas Time-Series: Find previous value for each ID based on year and semester

python,pandas,time-series
I realize this is a fairly basic question, but I couldn't find what I'm looking for through searching (partly because I'm not sure how to summarize what I want). In any case: I have a dataframe that has the following columns: * ID (each one represents a specific college course)...

## Forecasting an Arima Model in R Returning Strange Error

r,time-series,shiny,forecasting
I'm working on building a Shiny App for forecasting time series. One component of this is using ARIMA models to forecast. The user specifies the start and end of the historical data, what p, d, and q they would like to use in the ARIMA model (if they don't want...

## Convert data frame with epoch timestamps to time-series with milliseconds in R

r,time-series,xts,zoo
I have the following data.frame: df <- data.frame(timestamp=c(1428319770511, 1428319797218, 1428319798182, 1428319803327, 1428319808478), session=c("A","A","B","A","A")) I'd like to convert this data frame to a time series and work on time windows shorter than one second. I already tried zoo and xts, but I found it difficult to represent the epoch times as...

## Matlab: trying to estimate multifractal spectrum from time series by histogram box-counting

matlab,statistics,time-series,histogram,fractals
I am using the approach from this Yale page on fractals: http://classes.yale.edu/fractals/MultiFractals/Moments/TSMoments/TSMoments.html which is also expounded on this set of lecture slides (slide 32): http://multiscale.emsl.pnl.gov/docs/multifractal.pdf The idea is that you get a dataset, and examine it through many histograms with increasing numbers of bars i.e. resolution. Once resolution is high...

## Matlab's VAR[X] coefficient constraints for vector time series

matlab,time-series
Matlab's VARMAX model allows the user to set flags that determine whether individual linear coefficients are to be estimated. In particular, vgxset accepts an ARsolve parameter containing flags that determine whether individual time series lag coefficients are estimated. The fact that there are individual scalar flags for each scalar lag...

## How to use parameters from data frame in R and loop through time holding them constant

r,nested,time-series,lapply,sapply
I have a function (weisurv) that has 2 parameters - sc and shp. It is a function through time (t). Time is a sequence, i.e. t<-seq(1:100). weisurv<-function(t,sc,shp){ surv<-exp(-(t/sc)^shp) return(surv) } I have a data frame (df) that contains a list of sc and shp values (like 300+ of them). For...

## Enlarge time series and fill with -9999 R

r,merge,time-series
I run am model from 2007-01-01 00 to 2013-12-31 23. Not all my observations are that long, they start later and/or end earlier. In that case I want to fill in -9999 values. I have: [1,] "2003 09 01 01" "0" [2,] "2003 09 01 02" "0" [3,] "2003 09...

## R: faster alternative of period.apply

r,time-series,apply
I have the following data prepared Timestamp Weighted Value SumVal Group 1 1600 800 1 2 1000 1000 2 3 1000 1000 2 4 1000 1000 2 5 800 500 3 6 400 500 3 7 2000 800 4 8 1200 1000 4 I want to calculate for each group...

## Replace list of permutations with getSymbols data in R

r,time-series,permutation,quantmod
I downloaded some stock data: require("quantmod") s <- c("AAPL", "ADBE", "ADI", "ADP", "ADSK") e <- new.env() getSymbols(s, src='yahoo', from='2015-01-10', env = e ) #get closing prices close <- do.call(merge, eapply(e, function(x) Cl(x))) I found all the pairs of symbol names: #find all the pairwise permutations perm<-combn(s,2) perm [,1] [,2] [,3]...

## How to best compress timeseries into a different duration?

r,time-series
I've got a timeseries object defined like so: tser <- ts(cumsum(1 + rnorm(48)), frequency = 12, start = c(2010, 1)) The data looks similar to the below (clipped to only show one year) Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2010 0.6055677 2.8650543 2.6115597 3.1496051...

## Measure the STD of RMSE

matlab,time-series,forecasting
I'm working on a time series forecasting problem and I would like to confirm if it makes sense to compute the standard deviation of the root mean squared error. If so, is this the correct way? STD_test = std(sqrt((y_real-y_pred).^2)) Also, imagine that the output of the model is 100, the...

## plotting; adding own x-axis does not work

r,plot,time-series
I would like to plot time-series data. To illustrate the dates on the x-asis, I first removed the values on the axis to then add my on axsis with the correct dates: set.seed(1) r <- rnorm(20,0,1) z <- c(1,1,1,1,1,-1,-1,-1,1,-1,1,1,1,-1,1,1,-1,-1,1,-1) data <- as.data.frame(na.omit(cbind(z, r))) series1 <- ts(cumsum(c(1,data[,2]*data[,1]))) series2 <- ts(cumsum(c(1,data[,2]))) d1y...

## Why does R give me a time series of row numbers instead of values?

r,time-series
Why does R give me a time series of row numbers instead of values? I load a CSV with a single column of values in the order I need them. I'm trying to make it a time series. Instead of giving me the values I've entered, R gives me the...

## R function to return start & end date of a time series ts() object?

r,datetime,time-series,forecasting
I have created a list of 300 time series. Now I want to create a training sample(by holding out most recent 3 weeks) for each of the time series to build forecast models. So I want to use window function to subset the time series to skip the most recent...

## Dates on x-axis, time series

r,date,time-series,as.date
I have data covering a time period of over 25 years and I would like to see the years on the x-axis. dates <- as.Date(Dollar[,1], "%d.%m.%Y") Dollar <- as.xts(Dollar[,2], dates) plot(SWEDOLall, xaxt = "n", main="SMA", ann = FALSE) axis.Date(side = 1, dates, at = labDates, format = "%y", labels =...

## Cassandra storage internal

cassandra,apache-spark,time-series,cql
I'm trying to understand what exactly happens internally in storage engine level when a row(columns) is inserted in a CQL style table. CREATE TABLE log_date ( userid bigint, time timeuuid, category text, subcategory text, itemid text, count int, price int, PRIMARY KEY ((userid), time) - #1 PRIMARY KEY ((userid), time,...

## how to plot multiple time series in the same graph with customized x axis

javascript,django,python-2.7,highcharts,time-series
I want to plot multiple time series in the same graph using the same xaxis witch a customized one. this is my code: in views.py this is my function: def cdr_weekly_comparison(request): #import ipdb; ipdb.set_trace() acc = cdr_data.find() donnees=[] dt = datetime.now() y = dt.year m = dt.month d = dt.day...

## Plotting Probability Density Heatmap Over Time in R

r,plot,time-series,kriging
Let's say I have the output of a monte-carlo simulation of one variable over several different iterations (think millions). For each iteration, I have the values of the variable at each point in time (ranging from t=1 to t=365). I would like to produce the following plot: For each point...

## R - how to calculate “global” monthly means of a zoo object

r,time-series,mean,zoo
Let's say I have this zoo object: library(zoo) df <- structure(list(date = structure(c(0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334, 365, 396, 424, 455, 485, 516, 546, 577, 608, 638, 669, 699, 730, 761, 790, 821, 851, 882, 912, 943, 974, 1004, 1035, 1065, 1096, 1127,...

## Plotting multivariate time-series data in R

r,graph,plot,ggplot2,time-series
My data looks like this: > head(Full.df) Date Month Week Year Count.S Count.G Count.W Count.F 1 2006-01-02 2006-01-01 2006-01-02 2006-01-01 0 7 9 6 2 2006-01-03 2006-01-01 2006-01-02 2006-01-01 0 13 12 4 3 2006-01-04 2006-01-01 2006-01-02 2006-01-01 0 13 15 4 4 2006-01-05 2006-01-01 2006-01-02 2006-01-01 0 20 6...

## How to create a function and a loop to calculate growth rates of variables in a data frame in R

r,time-series,growth-rate
New to R and Stack Overflow. Suppose I have the following macroeconomic data loaded into a data frame called testdata in R. > testdata date gdp cpi_index rpi_index 21 2013 Q1 409985 125.067 247.4 22 2013 Q2 412620 125.971 249.7 23 2013 Q3 415577 126.352 250.9 24 2013 Q4 417265...

## Calculate days since last event in R

r,time-series
My question involves how to calculate the number of days since an event last that occurred in R. Below is a minimal example of the data: df <- data.frame(date=as.Date(c("06/07/2000","15/09/2000","15/10/2000","03/01/2001","17/03/2001","23/05/2001","26/08/2001"), "%d/%m/%Y"), event=c(0,0,1,0,1,1,0)) date event 1 2000-07-06 0 2 2000-09-15 0 3 2000-10-15 1 4 2001-01-03 0 5 2001-03-17 1 6 2001-05-23...

## Munging Time Series in Excel

vba,excel-vba,time-series
I want to sort data by the date from latest to earliest. My trouble is that the data i have has dates in mm-dd-yyyy text format. I could easily clean this up using Pandas in python but don't know the tools available in excel. Even when I try to change...

## plotting a graph with 3 curves time series data

r,plot,time-series
I have a pretty basic questions, but I wasn't able to find an answer. I would like to create a graph with three curves (time series data) without using ts.plot. Here are the three data sets: a1 <- seq(as.Date("2001-01-01"),as.Date("2021-01-01"),"years") a2 <- rnorm(21,10,1) Dollar <- data.frame(a1,a2) dates <- as.Date(Dollar[,1], "%d.%m.%Y",tz="GMT") xtsplot1...

## Unable to pass xreg values to hts ARIMA forecast

r,time-series,forecasting
I am trying to pass xreg arguments in my forecast but keep running into an error which says: fc=forecast(gy,fmethod="arima",h=days,method="bu",xreg=z,newxreg=fz) Error in as.matrix(newxreg) %*% coefs : non-conformable arguments In addition: Warning message: In cbind(intercept = rep(1, n), xreg) : number of rows of result is not a multiple of vector length...

## ETS multiplicative trend model written in state space form

time-series,forecasting,state-space
I have an ETS(M,Md,N) model and would like to write it in state space form: yt=w(x{t-1})+r(x{t-1})ɛt xt=f(x{t-1})+g(x{t-1})ɛt For additive trend, the state vector xt=(lt,bt)'. But I have no idea how to write the state vector xt for multiplicative trend. Can anyone help please:D...

## How to aggregate time series documents in mongodb

mongodb,mapreduce,time-series,mongodb-query,nosql-aggregation
i have a mongo sharded cluster where i save data from a virtual machines monitoring system (zabbix ecc). Now I want to get some information from the db, for example the avg memfree in the last 2 days of one vm. I read the tutorials about aggregation and also the...

## i10test for integration/stationary time series

matlab,time-series
In http://www.mathworks.com/help/econ/examples/time-series-regression-iv-spurious-regression.html, I am examing the use of the i10test for integration/stationary time series. The online help at http://www.mathworks.com/help/econ/i10test.html shows that this is done through a series of hypothesis tests. From my (OK, limited) exposure to hypthothesis testing, there is usually a threshold for the p-value, e.g. the alpha, that...

## Adding date tick marks to a Matlab plot

matlab,plot,time-series
I have a plot of time series data, and I would like to replace the tick marks of the x-axis (automatically I have the number of the ordered observations) with the date when the value is observed. I would like to have a tick mark every 5 years for example....

## R: HAC by NeweyWest using dynlm

r,time-series,regression

## Arima.sim issues in R

r,math,statistics,time-series,forecasting
I am working on making a prediction in R using time-series models. I used the auto.arima function to find a model for my dataset (which is a ts object). fit<-auto.arima(data) I can then plot the results of the prediction for the 20 following dates using the forecast function: plot(forecast(fit,h=20)) However...

## R, how to use pch with time series plot

r,time-series,lattice
I would like to plot a time series (meaning line graph with x axis as time) and specify a plotting character to use. None of the following has worked a1 = as.xts(ts(c(5,3,7,2,4,8,3), start=c(1980,1), freq=4)) library('lattice') xyplot(a1, col="red", pch=2) xyplot(a1, col="red", par.settings = list(superpose.symbol = list(col = 1, pch = 2)),)...

## ggplot2: arranging multiple boxplots as a time series

r,ggplot2,time-series,boxplot
I would like to create a multivariate boxplot time series with ggplot2 and I need to have an x axis that positions the boxplots based on their associated dates. I found two posts about this question: one is Time series plot with groups using ggplot2 but the x axis is...

## Matlab Reintroduction of AR and GARCH processes

matlab,for-loop,return,time-series,volatility
I am trying reintroduce autocorrelation and heteroskedasticity to my simulated residuals. My simulated (standardized) residuals have the dimension (horizon, nTrials, nIndices). In order to calculate today's mean / variance (i.e. t), I need to use the last periods mean /variance (i.e. t-1) as an input. This is where I am...

## Read Data into Time Series Object in R

r,data,time-series
My data looks as follows: Month/Year;Number 01/2010; 1.0 02/2010;19.0 03/2010; 1.0 ... How can I read this into a ts(object) in R?...

## Using dplyr to mutate() contract duration, payment counts, and total payout

r,timestamp,time-series,dataframes,dplyr
A simplified structure of my data is as follows: >ID <- c("A", "B", "B", "C", "A", "B", "C", "C", "A", "B") >Date = seq(as.Date("2000/07/01"), as.Date("2000/07/10"), "days") >Amt <- rnorm(10, 10, 3) >E <- data.frame(Date = Date, ID = ID, Amt = Amt) >E Date ID Amt 1 2000-07-01 A 5.9...