I am trying to compute pairwise correlations over rolling windows for n= 40 variables where all rolled pairwise correlations for 2 given variables are saved in a new variable.

My dataset has the following structure:

```
Date V1 V2 V3 . . .
01/01/2009 0.3 0.6 0.5
02/01/2009 0.1 0.5 0.2
03/01/2009 0.7 0.1 0.1
.
.
.
```

The following code does the trick. **However, it fails to ignore the lower diagonal of the correlation matrix** (it estimates twice the correlation between vars + once the 1s on the diagonal):

```
ds (Date), not
gl vars `r(varlist)'
local i = 0
local j = 0
foreach current_variable1 in $vars {
local i = `i'+1
foreach current_variable2 in $vars {
local j = `j'+1
if (`j' > `i') {
mvcorr `current_variable1' `current_variable2', ///
generate(corr_`current_variable1'_`current_variable2') window(60) force
}
}
}
```

In particular the `if`

condition on (`j > i`

) intended to capture the doubled calculation of each pairwise correlation fails to do so. Including the `if`

condition in the `mvcorr`

yields only the error message `'no observations'`

.

What could be the solution for this issue?

Answer:

@William Lisowski debugged your code in a comment, but you can simplify the whole procedure.

Create the tuples beforehand using the user-written command `tuples`

(`ssc install tuples`

).

```
clear
set more off
*----- example data -----
sysuse auto
keep mpg weight price
gen time = _n
tsset time
*----- what you want -----
tuples price mpg weight, min(2) max(2)
forvalues i = 1/`ntuples' {
local v1 : word 1 of `tuple`i''
local v2 : word 2 of `tuple`i''
mvcorr `tuple`i'', generate(`v1'_`v2') window(10) force
}
describe
```

@William's advice leads to the same result:

```
local i = 0
local j = 0
foreach v1 of varlist `vars' {
local i = `i' + 1
foreach v2 of varlist `vars' {
local j = `j' + 1
if (`j' > `i') {
mvcorr `v1' `v2', ///
generate(`v1'_`v2') window(10) force
}
}
local j = 0
}
describe
```

python,matrix,julia-lang

Using example from Andrew Ng's class (finding parameters for Linear Regression using normal equation): With Python: X = np.array([[1, 2104, 5, 1, 45], [1, 1416, 3, 2, 40], [1, 1534, 3, 2, 30], [1, 852, 2, 1, 36]]) y = np.array([[460], [232], [315], [178]]) θ = ((np.linalg.inv(X.T.dot(X))).dot(X.T)).dot(y) print(θ) Result: [[...

python,numpy,matrix,scipy

I want to get dot product of N vector pairs (a_vec[i, :], b_vec[i, :]). a_vec has shape [N, 3], bvec has the same shape (N 3D vectors). I know that it can be easily done in cycle via numpy.dot function. But cannot it be done somehow simpler and faster?...

r,data.table,stata,code-translation

Just came across a .do file that I need to translate into R because I don't have a Stata license; my Stata is rusty, so can someone confirm that the code is doing what I think it is? Here's the Stata code: collapse (min) MinPctCollected = PctCollected /// (mean) AvgPctCollected...

arrays,matlab,matrix

I'm trying to remove the rows which has duplicates in sequence. I have only 2 possible values which are 0 and 1. I have nXm which n shows possible number of bits and m is not important for my question. My goal is to find an matrix which is nX(m-a)....

r,matrix,aggregate,transformation,outer-join

I need to transform the output of an aggregate (mean) into a matrix outer-like style. data(mtcars) aggregate(disp ~ cyl + gear, data = mtcars, FUN = mean ) cyl gear disp 4 3 120.1000 6 3 241.5000 8 3 357.6167 4 4 102.6250 6 4 163.8000 4 5 107.7000 6...

matlab,matrix,merge

I am trying to merge two matrices that share same values in multiple columns. The following matrices should exemplify my problem and provide a MWE. However, my data is very long size(500000, 4), therefore I am searching an efficient way to merge them. The data consists of option data where...

r,matrix,sparse-matrix,lme4,lmer

I have a question regarding the extraction of the (raw) model matrix of random effects from models fitted with lmer (lme4) in R. More specifically, I want to obtain a data frame or a matrix that contains all variables that are involved in random effects terms. The matter is further...

stata

I can use tabout var1 var2...style(csv) to export a matrix-like object with var1 on one axis, var2 on the other, and the frequencies that each combination of the values var1 and var2 occur. Is it possible to use tabout or some other commmand (preferably native to Stata) to do something...

javascript,arrays,algorithm,recursion,matrix

I'm trying to come up with a solution that takes in a matrix like this: [[1,2,3,4], [5,6,7,8], [9,10,11,12], [13,14,15,16]] and returns an array traversing the array as a spiral, so in this example: [1,2,3,4,8,12,16,15,14,13,9,5,6,7,11,10] I'm having trouble getting this recursive solution to work, in which the result array takes the...

matrix,ocaml

I want to write a function that is modify every elements inside a matrix. But I have some problem when compiling. Warning 10: this expression should have type unit. I think because all function in ocaml need to return to a value or unit, so if I implement two tasks...

r,loops,matrix,lag

This issue is quite tricky to explain but I am sure some of you already faced it. So I have two matrix. Matrix 1 (mat 1) and Matrix 2 (mat 2) What I want to do is to record in a third matrix (mat3) the value of mat2, after checking...

matlab,matrix

I have a 1*3000 double matrix. I am doing some calculation and I need to calculate sum of n-1 columns in each step. For example, if I have this matrix: A = [1 2 3 4 5 6] The summation I'm looking for should be like this: B = [0...

c++,opencv,matrix,computer-vision,transform

I am trying to image coordinates to 3D coordinates. Using the solvePnP function (in C++)has given me 3X1 rotation matrix and 3X1 translation matrix. But isn't the [R|t] matrix supposed to be 3X4? Any help will be greatly appreciated!...

javascript,html,css,arrays,matrix

i have a matrix of objects: var data = [[ {"value": "1"}, {"value": "2"}, {"value": "2"}, {"value": "4"} ], [ {"value": "1"}, {"value": "2"}, {"value": "2"}, {"value": "4"} ], [ {"value": "1"}, {"value": "3"}, {"value": "3"}, {"value": "5"} ]]; and I would like to render the matirx out as a...

matlab,matrix,multidimensional-array,scalar

I'm having problems multiplying specific values within my multidimensional matrix by a scalar. My matrix has the following dimension: size(comDatabe) = 5 10 3 397 10 The third dimension is an x-y-z coordinate frame. Something went wrong and now my y-axis is defined upside down for one subject (#8 out...

c,arrays,matrix

Here is the question - Write a program to find the maximum element in a matrix using functions. Function specification: int findMax(int **a, int m, int n) The first argument corresponds to the pointer to the matrix. The second argument corresponds to the number of rows in the matrix. The...

r,matrix,data.frame

I have a symmetric matrix (dimension: 12,000 X 12,000) named A and I want to create another one based on a formula, which depends on the elements position. To explain: I want to create the D matrix (based on the values from A) using the formula: Dij = 1 -...

arrays,matlab,matrix

I have matrix A and matrix B. Matrix A is 100*3. Matrix B is 10*3. I need to insert one row from matrix B each time in a sequence into matrix A after every 10th row. The result would be Matrix A with 110*3. How can I do this in...

c#,matrix,xna

I'm making a game in C# and XNA 4.0. In some levels I need a camera matrix to make the level appear to scroll left/right. Additionally, I also need a scaling matrix to scale the graphics when the settings are used to change the window size. Both of my matrices...

r,performance,matrix,cluster-analysis,sparse-matrix

I have a large, sparse binary matrix (roughly 39,000 x 14,000; most rows have only a single "1" entry). I'd like to cluster similar rows together, but my initial plan takes too long to complete: d <- dist(inputMatrix, method="binary") hc <- hclust(d, method="complete") The first step doesn't finish, so I'm...

r,matlab,matrix,sparse-matrix

I have a data set (D) of (nxd) where n=number of rows and d= number of dimensions, I create a similarity matrix (S)(nxn) by comparing each row of the data set (D) and then convert it into a sparse matrix (tx3) where t is the number of non-zero elements of...

r,matrix

I have two matrices with the same columns and rows names: > metilacion[1:5,1:5] A2BP1 A2M A2ML1 A4GALT AAAS paciente1 0.2804884 0.5816559 1.1814702 -0.6234276 -0.3997400 paciente2 0.5122471 1.2944264 0.5673766 0.4490407 -0.6045510 paciente3 -0.3116356 1.6085049 0.9970350 0.3379215 -0.4787046 paciente4 -0.7220941 0.8771948 2.1445474 -0.5837802 -0.4848246 paciente5 -0.3369999 1.5885716 0.8185654 0.2374583 -0.5698858 > expresion[1:5,1:5]...

r,matrix

I'm trying to remove "singletons" from a binary matrix. Here, singletons refers to elements that are the only "1" value in the row AND the column in which they appear. For example, given the following matrix: > matrix(c(0,1,0,1,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,1,0,0,0,0,0,1,1), nrow=6) [,1] [,2] [,3] [,4] [,5] [,6] [,7] [1,] 0 1 0...

python,numpy,matrix,factorial

I'd like to know how to calculate the factorial of a matrix elementwise. For example, import numpy as np mat = np.array([[1,2,3],[2,3,4]]) np.the_function_i_want(mat) would give a matrix mat2 such that mat2[i,j] = mat[i,j]!. I've tried something like np.fromfunction(lambda i,j: np.math.factorial(mat[i,j])) but it passes the entire matrix as argument for np.math.factorial....

java,matrix,polynomials

I have a function that defines a very long polynomial useful in operations with matrices. The formula multiplies binomials of the form (1+xi*yj) , where i and j are subscripted and correspond to the rows and columns of the matrix in question, respectively. The polynomial that results is the product...

c++,file,matrix,io

So I have a C++ code that finds the inverse of a square matrix. The inverse is full and it would be bad to keep it all in memory because I am working with hundreds of thousands of columns. My code generates the columns one by one. After finding the...

c#,matrix,camera,xna,viewport

Problem description: I am trying to develop some 2D game with terrain collision in C# using XNA framework. I have already completed simple terrain generating and object, for example car, can move on this terrain. But now, I get a problem with camera fix to that car. I tried to...

c++,matlab,opencv,matrix

I have a [32678 x 10] matrix (w2c) and I want to copy 24700 rows of it to another matrix(out). I have the index of the rows to be copied in a vector(index). For doing this in matlab I do: out = w2c(index_im,:); It takes approximately 0.002622 seconds. In OpenCV:...

r,for-loop,matrix,vector,conditional

I have been trying to solve the following problem. Suppose I have the following vector: aux1<-c(0,0,0,4,5,0,7,0,0,10,11,12) where the numbers represent the number of the row. I want to calculate the distance between the differents elements of this vector fixing the first component, then the second and so on. If the...

matlab,function,matrix,max

Is there any function that can calculate cumulative maximum for a double matrix? I have a 1*3000 matrix and I need to calculate the cumulative maximum. For example if the matrix is: A = [8 3 6 7 9 10 6 2 2 3] The cumulative maximum array will be:...

python,numpy,matrix,modeling

I'm working on setting some boundary conditions for a water table model, and I am able to set the entire first row to a constant value, but not the entire first column. I am using np.zeros((11,1001)) to make an empty matrix. Does anyone know why I am successful at defining...

matlab,matrix

I have a matrix looks like: 0 0 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 2 0 0 0 0 0 1 0 1 0 0 0 1 0 4 0 0 0 0 0 3 0 0 6 0 0 4...

java,arrays,matrix,2d

I realize this problem and solutions are all over StackOverflow like here however I'm still unable to make this work. Most of the examples say that I just need to multiply the row by the width and add the column meaning the location (4, 3) in a 4x4 square grid...

r,matrix,transpose

I have some legacy R code that does: b = t(a) c = t(b) What does this code do? Looks like a noop to me. a is a vector constructed by c(1:20). Edit: bonus points on how to do this better....

plot,stata

I'm working with the ciplot graphing module for Stata and am encountering a problem with the alignment of bars when I use the by() option. Here's a trivial example demonstrating the issue: webuse citytemp, clear ciplot heatdd cooldd, by(region) horizontal recast(conn) So, the graph shows means and confidence intervals for...

r,matrix,head,corners

I am looking for a function in any package that can print matrix[1:5, 1:5]. head() is good for those that have few columns but not for a large matrix. I know I can create my own function for it but I am wondering if there is a function for it...

c++,algorithm,matrix,divide-and-conquer

I'm starting to learn how to implement divide and conquer algorithms, but I'm having some trouble with this exercise. I have written an algorithm but unfortunately it returns a 0 value. I need to count how many times a number finds in the matrix using D&C. My idea is to...

matlab,matrix,least-squares

How can I solve , where and and in the least squares sense in matlab? So I'd like to have the minimizing as output....

grouping,social-networking,stata

[I copied part of the below example from a separate post and changed it to suit my specific needs] pos_1 pos_2 2 4 2 5 1 2 3 9 4 2 9 3 The above is read as person_2 is connected to person_4,...,person_4 is connected to person_2, and person_9 is...

python,matrix,multidimensional-array,indices

I convert a list of integers to a two dimensinal array like this: data = numpy.array( l ) shape = ( 10, 30 ) data = data.reshape( shape ) I try to get the indices x,y of the matrix of the values that are bigger than some threshold and lower...

r,matrix,filter,data.frame,subset

Suppose I had the following matrix: matrix(c(1,1,2,1,2,3,2,1,3,2,2,1),ncol=3) Result: [,1] [,2] [,3] [1,] 1 2 3 [2,] 1 3 2 [3,] 2 2 2 [4,] 1 1 1 How can I filter/subset this matrix by whether or not each row has duplicate values? For example, in this case, I would only...

c++,matrix,vector

I am trying to fill a vector with a matrix of values in c++. I'm not very self confident with this procedure (I don't know well about pointers and I don't know if I need it here) however I am trying this int auxMat[gray.rows][gray.cols]; vector<int> collectionSum; collectionSum.push_back(auxMat); When I try...

c#,performance,matrix,jagged-arrays

I've created a lower triangular distance matrix (because of size issues) as jagged array Note: Distances between objects are symmetric var dm = new double[size][] for (var i = 0; i < size; i++) { dm[i] = new double[i+1]; for (var j = 0; j < i+1; j++) { dm[i][j]...

matrix,segmentation-fault,fortran,derived-types

[UPDATE] the code and a few sentences were changed to reflect a realization explained in my second comment. The code should compile with the line below, however, I have an older gfortran and may not be seeing some errors that you might. gfortran BLU_implementation_copy.f90 -o BLU_implementation_copy.x I'm getting an incredibly...

c++,algorithm,matrix

I am trying to print the right hemisphere of a matrix. If we draw the main and the secondary diagonals in a matrix we see that we got 4 equal parts, the right part is called (in my algorithms textbook) the right hemisphere of a square matrix. For example, in...

r,matrix

I have a question about subsetting in R. Say I have the following Matrices: Ch1.Amplitude Ch2.Amplitude [1,] 6968.577 9637.309 [2,] 11903.564 11385.656 [3,] 13503.292 9928.314 Ch1.Amplitude Ch2.Amplitude [1,] 11903.564 11385.656 [2,] 2519.582 8042.450 [3,] 9878.749 5899.139 I would like to match row 2 of the first Matrix A with row...

r,matlab,matrix,sparse-matrix

I have a symmetric matrix S(n*n) where approximately 70% of the data are 0. Symmetric matrix I want to convert the symmetric matrix into a sparse matrix with t rows. What would be the time complexity of generating a sparse matrix from the original symmetric matrix? Is it O(n^2), because...

c,variables,matrix

I've been coding for my university where we are working with matrices and I can't find the error in the code that is changing the value of the variable where I save the columns of the matrix. I've tried debugging it and can´t find it, it simply ends the function...

matrix,octave,matrix-multiplication,broadcasting

a = 1 2 3 b = 1 2 3 a.*b ans = 1 2 3 2 4 6 3 6 9 I used the .* operator to multiply a row vector and a column vector in Octave to see the results. I dont understand how the answer is obtained....

python,numpy,matrix

>>> A = np.matrix(np.zeros(2, 3))) >>> A.shape (2, 3) >>> A matrix([[ 0., 0., 0.], [ 0., 0., 0.]]) Does the matrix A have two rows with three zeros or two columns with three zeros?...