FAQ Database Discussion Community

## Subsetting Matrix Based on contents of dataframe

r,matrix,correlation
I have an 100X100 correlation matrix with zip codes as the column and row names. I also have a data frame that contains the latitude and longitude for all zipcdes and a function that calculates the distance based on lat and long. Here is a snippet of the correlation matrix...

## Is it possible to create data with a specific slope and pearson correlation coefficient in R?

r,correlation,linear
For example, I would like two variables, x and y to have a correlation coefficient of 0.7 and a slope of 1.5, with a specified mean and sample size for both variables. I don't care if the data is normal or not. I messed around with MASS a lot, using...

## R: Create dataset with specific correlation in r

r,random,dataset,correlation
How can i create a dataset with two columns having a specific correlation to each other? I want to be able to define the number of values which will be created and specify the correlation the output should have. The question is similar to this one: Generate numbers with specific...

## Correlate a single time series with a large number of time series

python,numpy,correlation
I have a large number (M) of time series, each with N time points, stored in an MxN matrix. Then I also have a separate time series with N time points that I would like to correlate with all the time series in the matrix. An easy solution is to...

## Computing the correlation coefficient between two multi-dimensional arrays

python,arrays,numpy,scipy,correlation
I have two arrays that have the shapes N X T and M X T. I'd like to compute the correlation coefficient across T between every possible pair of rows n and m (from N and M, respectively). What's the fastest, most pythonic way to do this? (Looping over N...

## Spearman - pairwise correlation

r,correlation
I have a data frame who looks like this (the real ones has 7000+ rows and 17 columns): V1 V2 V3 V4 1 93 98 79 170 2 17 17 22 85 3 1 57 137 320 4 9 1 0 440 5 NA NA NA 22 I would like...

## I want to do correlation for many data frame

r,function,loops,correlation
i have almost 13 files and i want to conduct three types of correlations to it. all the files have the same content except the values. for example: v1 v2 v3 v4 v5 v6 v7 v8 ........... v50 first correclation between v6 and v20 second correlation between v7 and v21...

## Jmeter : Application uses _session_id in cookie to maintain session. It is being maintained from login to logout. I need to correlate it

Jmeter : Application uses _session_id in cookie to maintain session. It is visible in get request. It is being maintained from the time of login till logout. I need to correlate it to test the performance. I have used regex extractor to extract the _session_id as below: In regular expression...

## Pandas Correlation Groupby

python,pandas,group-by,correlation
Assuming I have a dataframe similar to the below, how would I get the correlation between 2 specific columns and then group by the 'ID' column? I believe the Pandas 'corr' method finds the correlation between all columns. If possible I would also like to know how I could find...

## Fast rolling correlation in Matlab

matlab,correlation,accumarray
I am trying to derive a function for calculating a moving/rolling correlation for two vectors and speed is a high priority, since I need to apply this function in an array function. What I have (which is too slow) is this: Data1 = rand(3000,1); Data2 = rand(3000,1); function y =...

## Getting different correlation values for the same combination of vectors

r,correlation
why am I getting different correlations for the same combination below? > cor(finalDB[2:6],use="complete.obs") rocky1Rating rocky2Rating rocky3Rating rocky4Rating rocky5Rating rocky1Rating 1.0000000 ***0.6476523*** 0.5435555 0.4964198 0.3483168 rocky2Rating 0.6476523 1.0000000 0.7507204 0.6653651 0.5288312 rocky3Rating 0.5435555 0.7507204 1.0000000 0.7284123 0.5897088 rocky4Rating 0.4964198 0.6653651 0.7284123 1.0000000 0.6006595 rocky5Rating 0.3483168 0.5288312 0.5897088 0.6006595 1.0000000 > cor(finalDB[2],finalDB[3],use...

## Trigger by suffix from same element class

javascript,jquery,triggers,correlation,prefix
I want make every button which have suffix class -dummy (let's call it "dummy") to trigger another button who have same class but without that suffix (call it "real button"). <button class="btn-submit hidden"> triggered by <button class="btn-submit-dummy"> <button class="btn-edit hidden"> triggered by <button class="btn-edit-dummy"> <button class="btn-delete"> triggered by nothing since...

## Convert an image to a 2D array of coordinates in Python for two point correlation

python,opencv,numpy,fft,correlation
I need to perform two-point correlation function from astroML Python module, my data is originally a jpg image, black and white, and I convert it to binary image using OpenCV image thresholding(not sure that I did it right). The question is how now I convert the 2D binary matrix or...

## Bootstrap correlation CI with increasing sample size

r,ggplot2,replication,correlation,confidence-interval
I would like to demonstrate how the width of a 95% confidence interval around a correlation changes with increasing sample size, from n = 10 to n=100 in increments of 5 samples per round. I would assume we can use a bootstrap function to do this and replicate each round...

## SoapUI correlation (property transfer)

properties,soapui,correlation
I have a REST request that respond with the following: { "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IlQwWE8xNnAtMmZzMWxremV5", "expires_in": 2592000, "token_type": "Bearer" } I want to take the value of access_token, store it in a property and reuse it for two subsequent requests. Following some tutorial here, when running the request that obtains the access_token...

## cummerbund, scatterplots and a statistical value

r,correlation
I'm working in cummerbund with cuffdiff files from a RNA-Seq analysis. I made a scatterplot with two conditions, but I'd like to see de correlation value of my data. Is it possible? Is there a command to do this? Any idea? Thanks!!

## Quantifying importance of variables in Canonical Correspondence Analysis using R? (x-post from researchgate)

r,statistics,correlation,vegan
I currently have species abundance data for multiple lakes along with measurements of some environmental variables of those lakes. I decided to do Canonical Correspondence Analysis of the data in R, as demonstrated by ter Braak and Verdenschot (1995), see link: http://link.springer.com/article/10.1007%2FBF00877430 (section: "Ranking environmental variables in importance") I am not...

## Pairwise correlations over rolling periods ignoring double calculations

matrix,stata,correlation
I am trying to compute pairwise correlations over rolling windows for n= 40 variables where all rolled pairwise correlations for 2 given variables are saved in a new variable. My dataset has the following structure: Date V1 V2 V3 . . . 01/01/2009 0.3 0.6 0.5 02/01/2009 0.1 0.5 0.2...

## Cronbach's alpha, by Category, in New Column in SAS

sas,correlation
I am new in SAS and I am trying to calculate Cronbach's alpha. The code I am using is: proc corr data=test alpha; var A B C; run; However, This way I only get the Cronbach's alpha in a table in the results section. Is there a way by only...

## Customizing what variables are shown on the sides and on the top of the outputs `corrplot` or `corrgram` in R

r,data-visualization,heatmap,correlation,r-corrplot
Suppose I have the following data: > print(data) date gdp unemp_rate cpi_index rpi_index var1 var2 var3 var4 1 8/31/2009 23:00:00 0.002000575 0.0 0.006539081 0.008466604 0.041601305 0.193230747 0.002260496 0.016428674 2 12/1/2009 0:00:00 0.003890642 0.0 0.007278347 0.011660448 0.012048193 0.022703903 0.003004489 0.015541372 3 3/1/2010 0:00:00 0.005088272 0.2 0.007439852 0.011065007 0.028750000 -0.222946928 0.002789741 0.015225019...

## Finding aggregate correlation of multiple columns against one column in r

r,correlation
I have a data frame with 11 columns out of which 9 are numeric. I am trying to find out the correlation of 8 columns together against the remaining column i.e., correlation of 8 variables with 1 variable which should generate one value of correlation instead of generating 9 different...

## Plot only one or few rows of a correlation matrix

python,numpy,matplotlib,heatmap,correlation
I have a correlation matrix named corrdata that I calculated using numpy.corrcoef. Then what I do is extract one or a few rows of this matrix, and now just want to plot them instead of the whole matrix. Because the matrix is no longer square, it is not possible to...

## Creating Dummy Sets in MATLAB for statistics

matlab,correlation,categorical-data
i am looking at creating dummy sets in MATLAB, first i created an array of random variable with 10 instances of min=5 and max =10, and here is my code r = (10-5).*rand(10,1) + 5; Please I need help on how to create the following; 1: create additional variable that...

## Rolling average pairwise correlation - code doesn't work as expected

r,statistics,time-series,correlation,xts
I have an xts time series object: head(mtrx) ADS.DE.Close ALV.DE.Close BAS.DE.Close BAYN.DE.Close BEI.DE.Close BMW.DE.Close CBK.DE.Close CON.DE.Close DAI.DE.Close 2007-12-28 01:00:00 51.26 147.95 101.41 62.53 53.00 42.35 21.04 86.06 66.50 2008-01-02 01:00:00 50.00 145.92 100.94 61.45 52.39 42.73 20.75 83.76 64.68 2008-01-03 01:00:00 50.09 144.93 101.60 61.71 51.18 42.09 20.48 81.74 62.91...

## Error:3201 during Cordova App deployment on Windows

correlation,windows-applications,visual-studio-cordova
I am trying to create package of cordova application. App Passes the local validations but gives following error on file uploading: 3201: The publisher display name, \$username\$, specified in the package doesn’t match the publisher name, My App, that’s associated with the developer account. Its look like my AppxManifest.xml file...

## grouped correlation loop in r

r,loops,correlation
I want to calculate the correlation between Col1 and all of the other columns for each group. My input data set looks like this: Group1 Col1 Col2 Col3 Col4 A 3 1 0 1 A 8 0 1 0 B 4 1 1 1 B 2 1 0 1 And...

## Creating a correlation loop to print and save png for 30 different data sets

r,loops,correlation
I'm new to stackoverflow but have found the community amazingly helpful in all my previous questions. However, I couldn't quite find an answer to this question so here I am. My question has to deal with multiple data sets that I have and if there is a way to loop...

## Link and brush in Matplotlib/Pandas — plotting selected points from a plot in another plot

python,pandas,matplotlib,correlation
I'm wondering how could I implement some sort of 'link and brush' in Matplotlib/Pandas or if there is another lib that provides this. For instance, using the scatter-plot matrix from Pandas I obtained this graph: It can be seen, for example, that there is a relation between some points of...

## Making sense of CCA (Matlab implementation) 2

matlab,correlation,cross-correlation
I am using CCA for my work and want to understand something. This is my MATLAB code. I have only taken 100 samples to better understand the concepts of CCA. clc;clear all;close all; load carbig; data = [Displacement Horsepower Weight Acceleration MPG]; data(isnan(data))=0; X = data(1:100,1:3); Y = data(1:100,4:5); [wx,wy,~,U,V]...

## Rolling Mean of Rolling Correlation dataframe in Python?

python,matrix,pandas,yahoo,correlation
Python beginner here. What I've done so far: Imported price data from Yahoo Finance from a list of stocks. Between the stocks (every combination), computed the 20 day rolling correlation into a dataframe. I would like to: 1) Calculate the 200 day simple moving average for each of the 20...

## How to use biserial to calculate correlation between continuous and categorical variable?

r,correlation,continuous,categorical-data
I am trying to calculate the correlation between x (continuous variable) and y (categorical variable) in R. The function biserial in the psych package is used to calculate this. See here. But when I actually used it, I got a warning message and NA as the correlation: Warning message: In...

## Cannonical Correlation Analysis

matlab,statistics,correlation
I have just started working using CCA in Matlab. I have two vectors X and Y of dimension 60x1920 and 60x1536 with the number of samples being 60 and variables in the different set of vectors being 1920 and 1536 respectively. I want to know do CCA for reducing them...

## Smart way to loop over all rows in A and correlate with all columns in B

r,performance,for-loop,matrix,correlation
First post, so be gentle ;-) I have a scenario, where I want to correlate all the rows of mat A (around 50,000) with all columns of mat B (around 100). I have solved this by doing so: output = c() for( i in 1:nrow(A) ){ for(j in 1:ncol(B) ){...

## How to use `cor.test` for correlation of specific columns?

r,correlation
I have the following data example: A<-rnorm(100) B<-rnorm(100) C<-rnorm(100) v1<-as.numeric(c(1:100)) v2<-as.numeric(c(2:101)) v3<-as.numeric(c(3:102)) v2[50]<-NA v3[60]<-NA v3[61]<-NA df<-data.frame(A,B,C,v1,v2,v3) As you can see df has 1 NA in column 5, and 2 NA's in column 6. Now I would like to make a correlation matrix of col1 and 3 on the one hand,...

## Correlate by levels of a variable in R

r,correlation
I would like to correlate two variables and have the output reported separately for levels of a third variable. My data are similar to this example: var1 <- c(7, 8, 9, 10, 11, 12) var2 <- c(18, 17, 16, 15, 14, 13) categories <- c(1, 2, 3, 1, 2, 3)...