I am trying to plot a time series data. The dataframe looks like this [1]:Index ship_date cost_amount 0 1/8/2010 34276 1 1/8/2010 12375 2 1/8/2011 12343 3 2/9/2011 15435 [2]: df1.plot(figsize(20,5)) I am trying to plot the data but for some reason plot doesn't have x-axis in ascending order. How...

I'm having problems ploting vertical lines with ggplot in R. I want to draw a vertical line each Sunday on my time serie: VisitDate VisitMonth VisitYear City Weekday VisitWeek Code_CxF Centre Location 1 2014-05-02 05 2014 Barcelona 05Friday 2014-04-28 CxF_BCN CaixaForum Barcelona Catalunya 2 2014-05-03 05 2014 Barcelona 06Saturday 2014-04-28...

I am trying to run some trading strategies in R. I have downloaded some stock prices and calculated returns. The new return dataset has a number of -inf, NaN, and NA values. I am reproducing a row of the dataset (log_ret). Its a zoo dataset. library(zoo) log_ret <- structure( c(0.234,-0.012,-Inf,NaN,0.454,Inf),...

I'm on my research for storing logs to Cassandra. The schema for logs would be something like this. EDIT: I've changed the schema in order to make some clarification. CREATE TABLE log_date ( userid bigint, time timeuuid, reason text, item text, price int, count int, PRIMARY KEY ((userid), time) -...

I have to implement data collection for replay for electrical parameters for 100-1000's of devices with at least 20 parameters to monitor. This amounts to huge data collection as it will be based very similar to time series.I have to support resolution for 1 second. thinking about 1 year [365*24*60*60*1000]=31536000000...

I am using the approach from this Yale page on fractals: http://classes.yale.edu/fractals/MultiFractals/Moments/TSMoments/TSMoments.html which is also expounded on this set of lecture slides (slide 32): http://multiscale.emsl.pnl.gov/docs/multifractal.pdf The idea is that you get a dataset, and examine it through many histograms with increasing numbers of bars i.e. resolution. Once resolution is high...

I have a Pandas DataFrame with the following format: In [0]: df Out[0]: col1 col2 date 0 1 1 2015-01-01 1 1 2 2015-01-09 2 1 3 2015-01-10 3 2 1 2015-02-10 4 2 2 2015-02-10 5 2 3 2015-02-25 In [1]: df.dtypes Out[1]: col1 int64 col2 int64 date datetime64[ns]...

I'm trying to understand what exactly happens internally in storage engine level when a row(columns) is inserted in a CQL style table. CREATE TABLE log_date ( userid bigint, time timeuuid, category text, subcategory text, itemid text, count int, price int, PRIMARY KEY ((userid), time) - #1 PRIMARY KEY ((userid), time,...

I have the following data prepared Timestamp Weighted Value SumVal Group 1 1600 800 1 2 1000 1000 2 3 1000 1000 2 4 1000 1000 2 5 800 500 3 6 400 500 3 7 2000 800 4 8 1200 1000 4 I want to calculate for each group...

I have a pretty basic questions, but I wasn't able to find an answer. I would like to create a graph with three curves (time series data) without using ts.plot. Here are the three data sets: a1 <- seq(as.Date("2001-01-01"),as.Date("2021-01-01"),"years") a2 <- rnorm(21,10,1) Dollar <- data.frame(a1,a2) dates <- as.Date(Dollar[,1], "%d.%m.%Y",tz="GMT") xtsplot1...

Matlab's command for defining a vector time series model is vgxset, the formalism for which can be accessed by the command "doc vgxset". It says that the model parameter Q is "[a]n n-by-n symmetric innovations covariance matrix". No description of what it is for. I assumed that it was the...

I am trying to pass xreg arguments in my forecast but keep running into an error which says: fc=forecast(gy,fmethod="arima",h=days,method="bu",xreg=z,newxreg=fz) Error in as.matrix(newxreg) %*% coefs : non-conformable arguments In addition: Warning message: In cbind(intercept = rep(1, n), xreg) : number of rows of result is not a multiple of vector length...

I'm having a bit of trouble following the explanation of the parameters for vgxset. Being new to the field of time-series is probably part of my problem. The vgxset help page (http://www.mathworks.com/help/econ/vgxset.html) says that its for a generalized model structure, VARMAX, and I assume that I just use a portion...

My data looks like this: > head(Full.df) Date Month Week Year Count.S Count.G Count.W Count.F 1 2006-01-02 2006-01-01 2006-01-02 2006-01-01 0 7 9 6 2 2006-01-03 2006-01-01 2006-01-02 2006-01-01 0 13 12 4 3 2006-01-04 2006-01-01 2006-01-02 2006-01-01 0 13 15 4 4 2006-01-05 2006-01-01 2006-01-02 2006-01-01 0 20 6...

Matlab's VARMAX model allows the user to set flags that determine whether individual linear coefficients are to be estimated. In particular, vgxset accepts an ARsolve parameter containing flags that determine whether individual time series lag coefficients are estimated. The fact that there are individual scalar flags for each scalar lag...

I have a function (weisurv) that has 2 parameters - sc and shp. It is a function through time (t). Time is a sequence, i.e. t<-seq(1:100). weisurv<-function(t,sc,shp){ surv<-exp(-(t/sc)^shp) return(surv) } I have a data frame (df) that contains a list of sc and shp values (like 300+ of them). For...

I've been doing a variety of models in R with time series data (in XTS format) and I keep running into the same issue where there's no date / time component to the fitted values / forecasts and thus I can't graph them on the same graph as the original...

I'm trying to add a column of list objects to a data.frame of payments built like ID <- c("A", "B", "B", "c", "A", "B", "c", "c", "A", "B") Date = seq(as.Date("2000/07/01"), as.Date("2000/07/10"), "days") Amt <- rnorm(10, 10, 3) E <- data.frame(Date = Date, ID = ID, Amt = Amt) Date...

This is what I would like to do: library("lmtest") library("dynlm") test$Date = as.Date(test$Date, format = "%d.%m.%Y") zooX = zoo(test[, -1], order.by = test$Date) f <- d(Euribor3) ~ d(Ois3) + d(CDS) + d(Vstoxx) + d(log(omo)) + d(L(Euribor3)) m1 <- dynlm(f, data = zooX, start = as.Date("2005-01-05"),end = as.Date("2005-01-24")) m2 <- dynlm(f,...

I have data covering a time period of over 25 years. In the data set are over 6300 days. I would like to show the years on the x-axix. dates <- as.Date(DOL[,1], "%d.%m.%Y") DOL <- as.xts(DOL[,2], dates) plot(DOL, xaxt = "n", main="SMA", ann = FALSE) axis(1, at=as.POSIXct(dates),labels=format(dates,"%Y"),tick=TRUE) title(ylab = "Value")...

I would like to plot a time series (meaning line graph with x axis as time) and specify a plotting character to use. None of the following has worked a1 = as.xts(ts(c(5,3,7,2,4,8,3), start=c(1980,1), freq=4)) library('lattice') xyplot(a1, col="red", pch=2) xyplot(a1, col="red", par.settings = list(superpose.symbol = list(col = 1, pch = 2)),)...

I realize this is a fairly basic question, but I couldn't find what I'm looking for through searching (partly because I'm not sure how to summarize what I want). In any case: I have a dataframe that has the following columns: * ID (each one represents a specific college course)...

I'm trying to compute the variance of a time series for many sampling frequencies (the so called signature plot), I used the resample method looping on a set of frequencies but python stops before completing the task (no errors, just freezed). Here the code is var_list = [timeseries.resample(rule=str(int(freq))+'min',how='first').var() for i...

I would like to create a horizontal ‘stacked bar’ type plot in which date runs along the x-axis and my samples appear as bars on the y-axis. In the simple example below, I have three samples (a, b, c) each containing three values (0, 1, 2). I would like the...

I want to compute the following time series regression using R: $\Delta y_t=\beta_1 \Delta x_t+\beta_2 \Delta z_t+\beta_3 \Delta m_t+\beta_4 \Delta y_{t−1}$ Since I have not that much experience with R I want to ask if the following R code gives me what I want: y <- ts(diff(YY)) x <- ts(diff(XX))...

I have the following data.frame: df <- data.frame(timestamp=c(1428319770511, 1428319797218, 1428319798182, 1428319803327, 1428319808478), session=c("A","A","B","A","A")) I'd like to convert this data frame to a time series and work on time windows shorter than one second. I already tried zoo and xts, but I found it difficult to represent the epoch times as...

I have created a list of 300 time series. Now I want to create a training sample(by holding out most recent 3 weeks) for each of the time series to build forecast models. So I want to use window function to subset the time series to skip the most recent...

I'm working on building a Shiny App for forecasting time series. One component of this is using ARIMA models to forecast. The user specifies the start and end of the historical data, what p, d, and q they would like to use in the ARIMA model (if they don't want...

I would like to create a multivariate boxplot time series with ggplot2 and I need to have an x axis that positions the boxplots based on their associated dates. I found two posts about this question: one is Time series plot with groups using ggplot2 but the x axis is...

I have been trying to do some basic analysis on some timeseries data. However, I keep getting this error on anything I am trying to do Error in decompose(data_ts, type = c("additive")) : time series has no or less than 2 periods I assume the problem is that I am...

I have an embedded "thing" which generates data samples from several sensors at 1kHz. It has a fairly bandwidth constrained 3G connection to the outside world. Does anyone know of a platform which can provide the following (or at least a subset of the following): A Publish/Subscribe interface to send/receive...

I wish to remove values in a time series which are surrounded by blocks of NA of a certain minimal length. Some toy data: x = seq(0,10,length.out = 100) y = sin(x) + rnorm(length(x), mean=0, sd=0.1) y[20:21] = rep(NA, 2) y[50:54] = rep(NA, 5) y[55:59] = seq(-0.1, -0.8, length.out =...

I have one csv file in which I have 2 closing prices of stock(on daily basis) Dates Bajaj_close Hero_close 3/14/2013 1854.8 1669.1 3/15/2013 1850.3 1684.45 3/18/2013 1812.1 1690.5 3/19/2013 1835.9 1645.6 3/20/2013 1840 1651.15 3/21/2013 1755.3 1623.3 3/22/2013 1820.65 1659.6 3/25/2013 1802.5 1617.7 3/26/2013 1801.25 1571.85 3/28/2013 1799.55 1542 I...

I want to plot multiple time series in the same graph using the same xaxis witch a customized one. this is my code: in views.py this is my function: def cdr_weekly_comparison(request): #import ipdb; ipdb.set_trace() acc = cdr_data.find() donnees=[] dt = datetime.now() y = dt.year m = dt.month d = dt.day...

I have a dataset of soccer match results, and I am hoping to learn R by creating a running set of ratings similar to the World Football Elo formula. I am running into trouble with things that seem to be simple in Excel aren't exactly intuitive in R. For instance,...

Suppose I made a groupby on the valgdata DataFrame like below: grouped_valgdata = valgdata.groupby(['news_site','dato_uden_tid']).mean() Now I get this: sentiment news_site dato_uden_tid dr.dk 2015-06-15 54.777183 2015-06-16 54.703167 2015-06-17 54.948775 2015-06-18 54.424881 2015-06-19 53.290554 eb.dk 2015-06-15 53.279251 2015-06-16 53.285643 2015-06-17 53.558753 2015-06-18 52.854750 2015-06-19 54.415988 jp.dk 2015-06-15 56.590428 2015-06-16 55.313752 2015-06-17 53.771377...

I had two dataframes of differing sizes that I would like to do calculations with. The first dataset is a time series. The second dataset are the long-term monthly averages. The first: year month snow_depth 0 1979 1 18.322581 1 1979 2 11.535714 2 1979 3 5.322581 3 1979 4...

Apologies if this is a simple question/error, but when I try and predict a timeseries using statsmodels.tsa AR the prediction flatlines very quickly past the data I have. This doesn't depend on the order of the model or the length of the data used to fit the AR model. What...

Let's say I have this zoo object: library(zoo) df <- structure(list(date = structure(c(0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334, 365, 396, 424, 455, 485, 516, 546, 577, 608, 638, 669, 699, 730, 761, 790, 821, 851, 882, 912, 943, 974, 1004, 1035, 1065, 1096, 1127,...

My data looks as follows: Month/Year;Number 01/2010; 1.0 02/2010;19.0 03/2010; 1.0 ... How can I read this into a ts(object) in R?...

I have two data frames, df.events and df.activ. df.activ has very granular minute level data and an order of magnitude more records (1,000,000+) than df.events which has ~100,000 records, also at minute level granularity. The two dataframes have two common fields, DateTime and Geo. Both DateTime columns are in as.POSIXlt,...

I want to compute the following two regressions using R: library("dynlm") zooX = zoo(test[, -1]) lmx <- dynlm(d(Euribor3)~d(Ois3)+d(CDS)+d(Vstoxx)+d(log(omo))+d(L(Euribor3, 1)), data=zooX[1:16]) summary(lmx) zooX = zoo(test[, -1]) lmx <- dynlm(d(Euribor3)~d(Ois3)+d(CDS)+d(Vstoxx)+d(log(omo))+d(L(Euribor3, 1)), data=zooX[17:31]) summary(lmx) The only difference between those two models is the subset (the first[1:16] and the second [17:31]). Now these two...

Is it possible to predict directly into the future using epsilion-svr? My dataset is a univariate time series and has per line a record in this format: Y(t-W), Y(t-W+1), ..., Y(t), Y(t+PH) W is the number of time steps to consider PH controls how many steps into the future I...

Matlab defines LinearModel and GeneralizedLinearMixedModel classes. Browsing the documentation indicates that either (i) one is derived from the other, or (ii) there is automatic conversion. These are complex objects, and I am just starting to explore them, so I apologize if their relationship is obvious, but what exactly is their...

I want to set bounds for the x-axis for a plot of time-series data which features only time (no dates). My limits are: lims <- strptime(c("03:00","16:00"), format = "%H:%M") And my ggplot prints fine, but when I add this to scale_x_datetime scale_x_datetime(limits = lims) I get Error: Invalid input: time_trans...

I ran a regression first using lm and then using dynlm(from the package dynlm). Here is what I did using lm: Euribor3t <- ts(diff(Euribor3)) OIS3t <- ts(diff(Ois3)) x <- ts(diff(Eurepo3-Ois3)) Vstoxxt <- ts(diff(Vstoxx)) CDSt <- ts(diff(CDS)) omo2 <- ts(diff(log(Open.Market.Operations))) l1 <- (lag(Euribor3t, k=-1)) axx <- ts.intersect(Euribor3t, OIS3t, x, Vstoxxt, CDSt,...

Long time lurker first time poster. I have data that roughly follows a y=sin(time) distribution, but also depends on other variables than time. In terms of correlations, since the target y-variable oscillates there is almost zero statistical correlation with time, but y obviously depends very strongly on time. The goal...

I simulate a time series with periodic and linear components and try to use the R stl function to analyze it n = 1000 x = ts(0.1*rnorm(n) + sin(6*pi*(1:n)/n) + (1:n)/n,frequency=n) plot(x) stl(x,"per") but get a message Error in stl(x, "per") : series is not periodic or has less than...

I am currently getting timestamps from accelerometers, magnetometers, and gyroscopes and performing sensor fusion with GPS Location on an android device. I am getting the sensor timestamp using SensorEvent.timestamp and Location.getElapsedRealtimeNanos(). My code is as follows: Sensor Timestamp public void onSensorChanged( SensorEvent event ) { if( event.sensor.getType() == Sensor.TYPE_ACCELEROMETER )...

I have a data frame that contains two columns - time and price. It contains a series of observations for price of a certain item at various times. Here is a sample. > df time price 1 2014-12-12 14:57:15 45.81 2 2014-12-12 14:57:15 45.90 3 2014-12-12 15:00:08 45.76 4 2014-12-12...

I have some time series data that can be 1Hz, 10Hz, or 100Hz. the file I load in happens to be 1Hz: In [6]: data = pd.read_csv("ftp.csv") In [7]: data.Time Out[7]: 0 NaN 1 11:30:08 AM 2 11:30:09 AM 3 11:30:10 AM 4 11:30:11 AM 5 11:30:12 AM 6 11:30:13...

The following produces an error a1 = as.xts(ts(rnorm(20), start=c(1980,1), freq=4)) a2 = as.xts(ts(rnorm(30), start=c(1983,1), freq=4)) a = ts.intersect(a1,a2) Error in .cbind.ts(list(...), .makeNamesTs(...), dframe = dframe, union = FALSE) : no time series supplied The documentation says argument should be two or more univariate or multivariate time series, or objects which...

I would like to plot time-series data. To illustrate the dates on the x-asis, I first removed the values on the axis to then add my on axsis with the correct dates: set.seed(1) r <- rnorm(20,0,1) z <- c(1,1,1,1,1,-1,-1,-1,1,-1,1,1,1,-1,1,1,-1,-1,1,-1) data <- as.data.frame(na.omit(cbind(z, r))) series1 <- ts(cumsum(c(1,data[,2]*data[,1]))) series2 <- ts(cumsum(c(1,data[,2]))) d1y...

My question involves how to calculate the number of days since an event last that occurred in R. Below is a minimal example of the data: df <- data.frame(date=as.Date(c("06/07/2000","15/09/2000","15/10/2000","03/01/2001","17/03/2001","23/05/2001","26/08/2001"), "%d/%m/%Y"), event=c(0,0,1,0,1,1,0)) date event 1 2000-07-06 0 2 2000-09-15 0 3 2000-10-15 1 4 2001-01-03 0 5 2001-03-17 1 6 2001-05-23...

I have daily stock price data from yahoo finance in a dataframe called price_data. I would like to add a column to this which provides the fitted value from a time series trend of the Adj Close column. Here is the structure of the data I am using: In [41]:...

I run am model from 2007-01-01 00 to 2013-12-31 23. Not all my observations are that long, they start later and/or end earlier. In that case I want to fill in -9999 values. I have: [1,] "2003 09 01 01" "0" [2,] "2003 09 01 02" "0" [3,] "2003 09...

I'm fairly new to R and I've been trying for a while to do something, which I assumed to be very simple, but I keep failing at it (unfortunately for me, it doesn't mean it's not simple!). I have defined a function that takes a time series as an input...

Assume I have a DataFrame like the following: Month, Gender, State, Value 2010-01, M, S1, 10 2010-02, M, S1, 20 2010-05, M, S1, 26 2010-03, F, S2, 11 I want to add another column for the given Gender and state from the previous month (or X months past) if it...

I want to sort data by the date from latest to earliest. My trouble is that the data i have has dates in mm-dd-yyyy text format. I could easily clean this up using Pandas in python but don't know the tools available in excel. Even when I try to change...

Why does R give me a time series of row numbers instead of values? I load a CSV with a single column of values in the order I need them. I'm trying to make it a time series. Instead of giving me the values I've entered, R gives me the...

I have an xts time series object: head(mtrx) ADS.DE.Close ALV.DE.Close BAS.DE.Close BAYN.DE.Close BEI.DE.Close BMW.DE.Close CBK.DE.Close CON.DE.Close DAI.DE.Close 2007-12-28 01:00:00 51.26 147.95 101.41 62.53 53.00 42.35 21.04 86.06 66.50 2008-01-02 01:00:00 50.00 145.92 100.94 61.45 52.39 42.73 20.75 83.76 64.68 2008-01-03 01:00:00 50.09 144.93 101.60 61.71 51.18 42.09 20.48 81.74 62.91...

I have data covering a time period of over 25 years and I would like to see the years on the x-axis. dates <- as.Date(Dollar[,1], "%d.%m.%Y") Dollar <- as.xts(Dollar[,2], dates) plot(SWEDOLall, xaxt = "n", main="SMA", ann = FALSE) axis.Date(side = 1, dates, at = labDates, format = "%y", labels =...

I ran the following two regressions: library("dynlm") library("lmtest") zoop <- (test1[, -1]) f <- d(y) ~ d(x)+ d(z) + d(m) + d(log(p)) m1 <- dynlm(f, data = zoop, start = 1,end = 15) coeftest(m1, vcov=NeweyWest) m2 <- dynlm(f, data = zoop, start = 16,end = 31) coeftest(m2, vcov=NeweyWest) which gives...

I am trying to set-up a python code for forecasting a time-series, using SVM libraries of scikit-learn. My data consists of X values at a day interval for the last one years, and I need to predict y for a month of the next year . Here's what I have...

I would like to take a lag of an xts variable, and the lag() function returns a lag. However, if I use it on a ts variable, it gives a lead. Is this a bug, or working as intended? library('xts') a = as.xts(ts(c(5,3,7,2,4,8,3), start=c(1980,1), freq=4)) cbind(a, lag(a)) # provides lag...

I have a time series graph with monthly article frequency as the y axis. The data looks like this: Count.V Date Month Week Year 2637 6 2006-01-02 2006-01-01 2006-01-02 2006-01-01 406 4 2006-01-03 2006-01-01 2006-01-02 2006-01-01 543 4 2006-01-04 2006-01-01 2006-01-02 2006-01-01 998 3 2006-01-05 2006-01-01 2006-01-02 2006-01-01 1400 4...

I have an ETS(M,Md,N) model and would like to write it in state space form: yt=w(x{t-1})+r(x{t-1})ɛt xt=f(x{t-1})+g(x{t-1})ɛt For additive trend, the state vector xt=(lt,bt)'. But I have no idea how to write the state vector xt for multiplicative trend. Can anyone help please:D...

I am trying to impute missing values in a time series with an ARIMA model in R. I tried this code but no success. x <- AirPassengers x[90:100] <- NA fit <- auto.arima(x) fitted(fit)[90:100] ## this is giving me NAs plot(x) lines(fitted(fit), col="red") The fitted model is not imputing the...

I am currently trying to deseasonalize data for ARIMA models using the U.S. Census Bureau's package seasonal install.packages("seasonal") you can find a modified version of the data set here. also there is a vignette of the package here which gives very straight-forward directions as to how to install and specify...

Apologies if this is easy to solve, but I can't get my head around it. I have this data frame: > aaci Date Plate.1 Plate.2 Plate.3 Plate.4 Plate.5 Plate.6 Plate.7 Plate.8 Average SE Species 1 2014-06-19 0.0000000 0.0000000 0.000000 0.000000 0.0000000 0.0000000 0.000000 0.0000000 0.000000 0.0000000 aa 7 2014-08-04 7.0057778...

How can I get ggplot to produce something similar like library(ggplot2) library(reshape2) library(ecp) synthetic_control.data <- read.table("/path/synthetic_control.data.txt", quote="\"", comment.char="") n <- 2 s <- sample(1:100, n) idx <- c(s, 100+s, 200+s, 300+s, 400+s, 500+s) sample2 <- synthetic_control.data[idx,] df = as.data.frame(t(as.matrix(sample2))) #calculate the change points changeP <- e.divisive(as.matrix(df[1]), k=8, R = 400,...

I downloaded some stock data: require("quantmod") s <- c("AAPL", "ADBE", "ADI", "ADP", "ADSK") e <- new.env() getSymbols(s, src='yahoo', from='2015-01-10', env = e ) #get closing prices close <- do.call(merge, eapply(e, function(x) Cl(x))) I found all the pairs of symbol names: #find all the pairwise permutations perm<-combn(s,2) perm [,1] [,2] [,3]...

New to R and Stack Overflow. Suppose I have the following macroeconomic data loaded into a data frame called testdata in R. > testdata date gdp cpi_index rpi_index 21 2013 Q1 409985 125.067 247.4 22 2013 Q2 412620 125.971 249.7 23 2013 Q3 415577 126.352 250.9 24 2013 Q4 417265...

With reference to this question: transforming a ts in a data.frame and back I have a list of monthly averages that start in May 2012 and go through May 2015. It looks like this initially: head (AVG_LOSCAT2) month AVG_LOSCAT YEAR MONTH 1 2012-05 5.342066 2012 05 2 2012-06 6.544096 2012...

We have a boolean variable X which is either true or false and alternates at each time step with a probability p. I.e. if p is 0.2, X would alternate once every 5 time steps on average. We also have a time line and observations of the value of this...

I am having trouble converting daily data into weekly using averages over the week. My Data looks like this: > str(daily_FWIH) 'data.frame': 4371 obs. of 6 variables: $ Date : Date, format: "2013-03-01" "2013-03-02" "2013-03-04" "2013-03-05" ... $ CST.OUC : Factor w/ 6 levels "BVG11","BVG12",..: 1 1 1 1 1...

I would like to use a timeline that just shows years. The special about my data is that I just have years. But these years are ordered. So I know the exact order but not any more detail like day or month. So first of all I would like to...

I'm working on a time series forecasting problem and I would like to confirm if it makes sense to compute the standard deviation of the root mean squared error. If so, is this the correct way? STD_test = std(sqrt((y_real-y_pred).^2)) Also, imagine that the output of the model is 100, the...

While this may start out sounding like as statistics question, please bear with me. I have several calcium concentrations from water samples collected at different sampling locations. The water is resampled at some of the stations on a monthly, yearly, every other year basis. I want to measure yearly and...

I have a sequence of datetime objects and a series of data which spans through several years. A can create a Series object and resample it to group it by months: df=pd.Series(varv,index=dates) multiMmean=df.resample("M", how='mean') print multiMmean This, however, outputs 2005-10-31 172.4 2005-11-30 69.3 2005-12-31 187.6 2006-01-31 126.4 2006-02-28 187.0 2006-03-31...

I'm relatively new to pandas, and trying to figure out what the best way of calculating this information is, so any help is much appreciated. Essentially I have a dataframe that looks like so: id activity_date 1 2015-01-01 1 2015-01-02 1 2015-01-03 2 2015-01-02 2 2015-01-05 3 2015-01-10 And I...

I have a data set for motor vehicle crashes happening daily in NYC from 1 Jan 2014 to 31 Dec 2012. I want to plot time series of the number of injured cyclists, and motorists, monthly in a single plot. My data looks like this: Date Time Location Cyclists injured...

i have a mongo sharded cluster where i save data from a virtual machines monitoring system (zabbix ecc). Now I want to get some information from the db, for example the avg memfree in the last 2 days of one vm. I read the tutorials about aggregation and also the...

A simplified structure of my data is as follows: >ID <- c("A", "B", "B", "C", "A", "B", "C", "C", "A", "B") >Date = seq(as.Date("2000/07/01"), as.Date("2000/07/10"), "days") >Amt <- rnorm(10, 10, 3) >E <- data.frame(Date = Date, ID = ID, Amt = Amt) >E Date ID Amt 1 2000-07-01 A 5.9...

I know there are many time-series questions on here but mine does not seem to comfortably fit with the given solutions. I am also new to Cassandra so I might be approaching this with the wrong mindset. Bear with me. I am receiving search data in the form: datetime_searched, term_used,...

I am looking for help with getting a volatility function to work with my dataframe. In the function below, I'm just trying to get price daily log returns for each security (each column in my data is a different security's prices over time), and then calculate an annualized vol. volcalc=...

I am trying to fit a subset model with only lag 4. In the manual it's written "you must use p=c(0,0,0,4) since p=4 will fit a full AR(4)". I did this. #fit a subset model with just lag 4 Fit=FitAR(p=c(0,0,0,4), lag.max = "default", ARModel = "ARz") However, I get the...

I have a plot of time series data, and I would like to replace the tick marks of the x-axis (automatically I have the number of the ordered observations) with the date when the value is observed. I would like to have a tick mark every 5 years for example....

I have a large set of measurements taken every 1 millisecond stored in a SQL Server 2012 table. Whenever there are 3 or more duplicate values in some rows that I would like to delete the middle duplicates. Highlighted values in this image of sample data are the ones that...

I have a times series with some missing entries, that looks like this: date value --------------- 2000 5 2001 10 2003 8 2004 72 2005 12 2007 13 I would like to do create a column for the "previous_value". But I only want it to show values for consecutive years....

I'm working on a design for a time series dataset, basically I have servers I monitor, and I would like to know some metric about it over a period of time. based on http://blog.mongodb.org/post/65517193370/schema-design-for-time-series-data-in-mongodb I created a design that is a document per server+month, and in it an embedded document...

In http://www.mathworks.com/help/econ/examples/time-series-regression-iv-spurious-regression.html, I am examing the use of the i10test for integration/stationary time series. The online help at http://www.mathworks.com/help/econ/i10test.html shows that this is done through a series of hypothesis tests. From my (OK, limited) exposure to hypthothesis testing, there is usually a threshold for the p-value, e.g. the alpha, that...

I want to have both month and day in the x-axis of the time series plot when using facet for years in ggplot2. My MWE is below: set.seed(12345) Date <- seq(as.Date("2010/1/1"), as.Date("2014/1/1"), "week") Y <- rnorm(n=length(Date), mean=100, sd=1) df <- data.frame(Date, Y) df$Year <- format(df$Date, "%Y") df$Month <- format(df$Date, "%b")...

I've got a timeseries object defined like so: tser <- ts(cumsum(1 + rnorm(48)), frequency = 12, start = c(2010, 1)) The data looks similar to the below (clipped to only show one year) Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2010 0.6055677 2.8650543 2.6115597 3.1496051...

I have multiple data frames that look like this: > head(Standard.df) Count.S Date Month Week Year 552 15 2008-01-01 2008-01-01 2007-12-31 2008-01-01 594 11 2008-01-02 2008-01-01 2007-12-31 2008-01-01 1049 10 2008-01-03 2008-01-01 2007-12-31 2008-01-01 511 12 2008-01-04 2008-01-01 2007-12-31 2008-01-01 717 10 2008-01-06 2008-01-01 2007-12-31 2008-01-01 1744 3 2008-01-07 2008-01-01...

I am trying reintroduce autocorrelation and heteroskedasticity to my simulated residuals. My simulated (standardized) residuals have the dimension (horizon, nTrials, nIndices). In order to calculate today's mean / variance (i.e. t), I need to use the last periods mean /variance (i.e. t-1) as an input. This is where I am...

I am working on making a prediction in R using time-series models. I used the auto.arima function to find a model for my dataset (which is a ts object). fit<-auto.arima(data) I can then plot the results of the prediction for the 20 following dates using the forecast function: plot(forecast(fit,h=20)) However...

I'm a bit over my head here and I hope you can help me, or at least point me in the right direction. I got a massive dataset (5.8 mio observations per year, over 14 years), which deals with individuals' occupation over time. I need to sum up the changes...

Assume I've a timeseries of a certain number of years as in: rng = pd.date_range(start = '2001-01-01',periods = 5113) ts = pd.TimeSeries(np.random.randn(len(rng)), rng) Than I can calculate it's standard year (the average value of each day over all years) by doing: std = ts.groupby([ts.index.month, ts.index.day]).mean() Now I was wondering how...

Let's say I have the output of a monte-carlo simulation of one variable over several different iterations (think millions). For each iteration, I have the values of the variable at each point in time (ranging from t=1 to t=365). I would like to produce the following plot: For each point...