FAQ Database Discussion Community

## Pandas Dataframe Plot

python,pandas,time-series,ipython-notebook
I am trying to plot a time series data. The dataframe looks like this [1]:Index ship_date cost_amount 0 1/8/2010 34276 1 1/8/2010 12375 2 1/8/2011 12343 3 2/9/2011 15435 [2]: df1.plot(figsize(20,5)) I am trying to plot the data but for some reason plot doesn't have x-axis in ascending order. How...

## ggplot vertical line with date axis R

r,ggplot2,time-series,vline
I'm having problems ploting vertical lines with ggplot in R. I want to draw a vertical line each Sunday on my time serie: VisitDate VisitMonth VisitYear City Weekday VisitWeek Code_CxF Centre Location 1 2014-05-02 05 2014 Barcelona 05Friday 2014-04-28 CxF_BCN CaixaForum Barcelona Catalunya 2 2014-05-03 05 2014 Barcelona 06Saturday 2014-04-28...

## Replace -inf, NaN and NA values with zero in a dataset in R

r,time-series,nan,zoo
I am trying to run some trading strategies in R. I have downloaded some stock prices and calculated returns. The new return dataset has a number of -inf, NaN, and NA values. I am reproducing a row of the dataset (log_ret). Its a zoo dataset. library(zoo) log_ret <- structure( c(0.234,-0.012,-Inf,NaN,0.454,Inf),...

## Using Cassandra for time series data

cassandra,time-series,composite-key
I'm on my research for storing logs to Cassandra. The schema for logs would be something like this. EDIT: I've changed the schema in order to make some clarification. CREATE TABLE log_date ( userid bigint, time timeuuid, reason text, item text, price int, count int, PRIMARY KEY ((userid), time) -...

## SQL Server Time Series Modelling Huge datacollection

sql,.net,sql-server,time-series
I have to implement data collection for replay for electrical parameters for 100-1000's of devices with at least 20 parameters to monitor. This amounts to huge data collection as it will be based very similar to time series.I have to support resolution for 1 second. thinking about 1 year [365*24*60*60*1000]=31536000000...

## Matlab: trying to estimate multifractal spectrum from time series by histogram box-counting

matlab,statistics,time-series,histogram,fractals
I am using the approach from this Yale page on fractals: http://classes.yale.edu/fractals/MultiFractals/Moments/TSMoments/TSMoments.html which is also expounded on this set of lecture slides (slide 32): http://multiscale.emsl.pnl.gov/docs/multifractal.pdf The idea is that you get a dataset, and examine it through many histograms with increasing numbers of bars i.e. resolution. Once resolution is high...

## Calculating the difference in dates in a Pandas GroupBy object

python,pandas,time-series
I have a Pandas DataFrame with the following format: In [0]: df Out[0]: col1 col2 date 0 1 1 2015-01-01 1 1 2 2015-01-09 2 1 3 2015-01-10 3 2 1 2015-02-10 4 2 2 2015-02-10 5 2 3 2015-02-25 In [1]: df.dtypes Out[1]: col1 int64 col2 int64 date datetime64[ns]...

## Cassandra storage internal

cassandra,apache-spark,time-series,cql
I'm trying to understand what exactly happens internally in storage engine level when a row(columns) is inserted in a CQL style table. CREATE TABLE log_date ( userid bigint, time timeuuid, category text, subcategory text, itemid text, count int, price int, PRIMARY KEY ((userid), time) - #1 PRIMARY KEY ((userid), time,...

## R: faster alternative of period.apply

r,time-series,apply
I have the following data prepared Timestamp Weighted Value SumVal Group 1 1600 800 1 2 1000 1000 2 3 1000 1000 2 4 1000 1000 2 5 800 500 3 6 400 500 3 7 2000 800 4 8 1200 1000 4 I want to calculate for each group...

## plotting a graph with 3 curves time series data

r,plot,time-series
I have a pretty basic questions, but I wasn't able to find an answer. I would like to create a graph with three curves (time series data) without using ts.plot. Here are the three data sets: a1 <- seq(as.Date("2001-01-01"),as.Date("2021-01-01"),"years") a2 <- rnorm(21,10,1) Dollar <- data.frame(a1,a2) dates <- as.Date(Dollar[,1], "%d.%m.%Y",tz="GMT") xtsplot1...

## vgxset command: Q parameter for resulting model object?

matlab,time-series
Matlab's command for defining a vector time series model is vgxset, the formalism for which can be accessed by the command "doc vgxset". It says that the model parameter Q is "[a]n n-by-n symmetric innovations covariance matrix". No description of what it is for. I assumed that it was the...

## Unable to pass xreg values to hts ARIMA forecast

r,time-series,forecasting
I am trying to pass xreg arguments in my forecast but keep running into an error which says: fc=forecast(gy,fmethod="arima",h=days,method="bu",xreg=z,newxreg=fz) Error in as.matrix(newxreg) %*% coefs : non-conformable arguments In addition: Warning message: In cbind(intercept = rep(1, n), xreg) : number of rows of result is not a multiple of vector length...

## Matlab's VARMAX regression parameters/coefficients nX & b

matlab,time-series
I'm having a bit of trouble following the explanation of the parameters for vgxset. Being new to the field of time-series is probably part of my problem. The vgxset help page (http://www.mathworks.com/help/econ/vgxset.html) says that its for a generalized model structure, VARMAX, and I assume that I just use a portion...

## Plotting multivariate time-series data in R

r,graph,plot,ggplot2,time-series
My data looks like this: > head(Full.df) Date Month Week Year Count.S Count.G Count.W Count.F 1 2006-01-02 2006-01-01 2006-01-02 2006-01-01 0 7 9 6 2 2006-01-03 2006-01-01 2006-01-02 2006-01-01 0 13 12 4 3 2006-01-04 2006-01-01 2006-01-02 2006-01-01 0 13 15 4 4 2006-01-05 2006-01-01 2006-01-02 2006-01-01 0 20 6...

## Matlab's VAR[X] coefficient constraints for vector time series

matlab,time-series
Matlab's VARMAX model allows the user to set flags that determine whether individual linear coefficients are to be estimated. In particular, vgxset accepts an ARsolve parameter containing flags that determine whether individual time series lag coefficients are estimated. The fact that there are individual scalar flags for each scalar lag...

## How to use parameters from data frame in R and loop through time holding them constant

r,nested,time-series,lapply,sapply
I have a function (weisurv) that has 2 parameters - sc and shp. It is a function through time (t). Time is a sequence, i.e. t<-seq(1:100). weisurv<-function(t,sc,shp){ surv<-exp(-(t/sc)^shp) return(surv) } I have a data frame (df) that contains a list of sc and shp values (like 300+ of them). For...

## Fitted values in R forecast missing date / time component

r,time-series,forecasting
I've been doing a variety of models in R with time series data (in XTS format) and I keep running into the same issue where there's no date / time component to the fitted values / forecasts and thus I can't graph them on the same graph as the original...

## How to get a column of lists of contract payout pairs (date:amounts)?

r,time-series,dplyr
I'm trying to add a column of list objects to a data.frame of payments built like ID <- c("A", "B", "B", "c", "A", "B", "c", "c", "A", "B") Date = seq(as.Date("2000/07/01"), as.Date("2000/07/10"), "days") Amt <- rnorm(10, 10, 3) E <- data.frame(Date = Date, ID = ID, Amt = Amt) Date...

## R: HAC by NeweyWest using dynlm

r,time-series,regression

## How to best compress timeseries into a different duration?

r,time-series
I've got a timeseries object defined like so: tser <- ts(cumsum(1 + rnorm(48)), frequency = 12, start = c(2010, 1)) The data looks similar to the below (clipped to only show one year) Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2010 0.6055677 2.8650543 2.6115597 3.1496051...

## Combining time series data into a single data frame

r,date,data.frame,time-series
I have multiple data frames that look like this: > head(Standard.df) Count.S Date Month Week Year 552 15 2008-01-01 2008-01-01 2007-12-31 2008-01-01 594 11 2008-01-02 2008-01-01 2007-12-31 2008-01-01 1049 10 2008-01-03 2008-01-01 2007-12-31 2008-01-01 511 12 2008-01-04 2008-01-01 2007-12-31 2008-01-01 717 10 2008-01-06 2008-01-01 2007-12-31 2008-01-01 1744 3 2008-01-07 2008-01-01...

## Matlab Reintroduction of AR and GARCH processes

matlab,for-loop,return,time-series,volatility
I am trying reintroduce autocorrelation and heteroskedasticity to my simulated residuals. My simulated (standardized) residuals have the dimension (horizon, nTrials, nIndices). In order to calculate today's mean / variance (i.e. t), I need to use the last periods mean /variance (i.e. t-1) as an input. This is where I am...

## Arima.sim issues in R

r,math,statistics,time-series,forecasting
I am working on making a prediction in R using time-series models. I used the auto.arima function to find a model for my dataset (which is a ts object). fit<-auto.arima(data) I can then plot the results of the prediction for the 20 following dates using the forecast function: plot(forecast(fit,h=20)) However...

## 150x150 crosstab in stata, showing timeseries movement between categories

time-series,stata,crosstab
I'm a bit over my head here and I hope you can help me, or at least point me in the right direction. I got a massive dataset (5.8 mio observations per year, over 14 years), which deals with individuals' occupation over time. I need to sum up the changes...

## Difference between multi year timeseries and it's 'standard year'

python,datetime,numpy,pandas,time-series
Assume I've a timeseries of a certain number of years as in: rng = pd.date_range(start = '2001-01-01',periods = 5113) ts = pd.TimeSeries(np.random.randn(len(rng)), rng) Than I can calculate it's standard year (the average value of each day over all years) by doing: std = ts.groupby([ts.index.month, ts.index.day]).mean() Now I was wondering how...

## Plotting Probability Density Heatmap Over Time in R

r,plot,time-series,kriging
Let's say I have the output of a monte-carlo simulation of one variable over several different iterations (think millions). For each iteration, I have the values of the variable at each point in time (ranging from t=1 to t=365). I would like to produce the following plot: For each point...