matlab,sorting,text,awk,libsvm , Wrapper for the matlab to read sparse data file


Wrapper for the matlab to read sparse data file

Question:

Tag: matlab,sorting,text,awk,libsvm

Libsvm can read the following datafile and convert it into sparse data structure in matlab (using libsvmread).

-1 3:1 11:1 14:1 19:1 39:1 42:1 55:1 64:1 67:1 73:1 75:1 76:1 80:1 83:1 
-1 3:1 6:1 17:1 27:1 35:1 40:1 57:1 63:1 69:1 73:1 74:1 76:1 81:1 103:1 

First column is a label for the binary classification and other columns are feature vectors. For example in first column only positions 3,11,14,19... are non zero.

I have a file in which these positions are not sorted. For example it could be like -

-1 11:1 3:1 14:1 19:1 39:1 42:1 55:1 64:1 67:1 73:1 75:1 76:1 80:1 83:1 

Libsvmread won't work in such a situation. Is there anyway, where I can sort the data (according to positions) or is there any exisiting code which can help me extract this data in matlab?

The goal is that given this sample Input

-1 11:1 3:1 14:1 19:1 39:1 42:1 55:1 64:1 67:1 73:1 75:1 76:1 80:1 83:1 
-1 3:1 2:1 6:1 4:1 17:1 27:1 35:1 40:1 57:1 63:1 69:1 73:1 74:1 76:1 81:1 103:1

We get the following output:

-1 3:1 11:1 14:1 19:1 39:1 42:1 55:1 64:1 67:1 73:1 75:1 76:1 80:1 83:1
-1 2:1 3:1 4:1 6:1 17:1 27:1 35:1 40:1 57:1 63:1 69:1 73:1 74:1 76:1 81:1 103:1

Answer:

Store all the info in an array a[] and then sort using indices:

awk '{delete a
      for (i=2; i<=NF; i++) 
          a[$i+0]=$i
      n=asorti(a, sorted, "@ind_num_asc")
      printf "%s%s", $1, OFS
      for (i=1;i<=n;i++)
          printf "%s%s", a[sorted[i]], (i==n?ORS:OFS)}' file

Explanation

This uses asorti() and @ind_num_asc to define the ordering mode.

For every line, we store in an array a[] all the data starting on the 2nd field. Then, we sort it numerically and print it back, in the sorted order.

Test

$ awk '{delete a; for (i=2; i<=NF; i++) {a[$i+0]=$i}; n=asorti(a, sorted, "@ind_num_asc"); printf "%s%s", $1, OFS; for (i=1;i<=n;i++) printf "%s%s", a[sorted[i]], (i==n?ORS:OFS)}' a
1 3:1 11:1 14:1 19:1 39:1 42:1 55:1 64:1 67:1 73:1 75:1 76:1 80:1 83:1
-1 2:1 3:1 4:1 6:1 17:1 27:1 35:1 40:1 57:1 63:1 69:1 73:1 74:1 76:1 81:1 103:1

Related:


MATLAB - How to merge figure sections vertically


matlab,plot
I want to display three figures in a figure window. Assuming that I divide 2x2 regions. subplot(2,2,1) ---------+----------- | R1 | R2 | ---------+----------- | R3 | R4 | ---------+----------- I want to show a figure merging R1 and R3 ant other two in R2 and R4 I can display...

C# sorting arrays in ascending and descending order


c#,arrays,sorting
I'm having trouble writing a method that returns true if the elements of an array (numbers) are in sorted order, ascending or descending, and false, if they are not in any sorted order. I can return a correct boolean value if the array is ascending but i do not know...

Plotting random signal on circle


matlab,plot,signals,circle
I have some random signal (for example sin signal) with the time scale. t=0:0.1:2*pi y=sin(t) plot(t,y) Now I want to draw this signal on this circle. So the time vector actually becomes an envelope of the circle. Envelope of the circle represents "y = 0" in cartesian coordinate system. Here...

Matlab Distribution Sampling


matlab,distribution,sampling,random-sample
How can I create a vector x in Matlab that has values between 0.8 and 1.2, randomly sampled from a: 1. Uniform 2. Normal distribution? There are a lot of functions dealing with distributions, but I'm having trouble using them properly....

How to access variables in the properties block of a Matlab System Object?


matlab,simulink
I am working on a simple System Object in Matlab/Simulink. It looks like this : classdef realtime_header_detectorSO < matlab.System & matlab.system.mixin.Propagates % correlateHeader % % This template includes the minimum set of functions required % to define a System object with discrete state. properties Header %nrOfBitsInPreviousStep=0; s=100; d=zeros(1,s); end properties...

Saving images with more than 8 bits per pixel in matlab


image,matlab,image-processing,computer-vision
I need to save a set of pre-processing images in matlab, resulting in grayscale images. The problem is the fact that these pre-processed images have pixel values higher than 255. If I save them with imwrite() as, for instance, .PNG files, does matlab normalize the values to be in [0,255]...

matlab plots as movie with legend


matlab,plot,legend,movie
i have a question regarding legend for movies. This is my code: fig = figure(); for i = 1: 70000 plot(signal1) hold on; plot([i,i],[-5,5]) plot(signal2,'r') hold off; title('\fontsize{14} my data'); legend('signal1','signal2'); axis tight; f(i) = getframe(fig); end The legend shows the same colors for the first two things I plot....

Matlab - Multiply specific entries by a scalar in multidimensional matrix


matlab,matrix,multidimensional-array,scalar
I'm having problems multiplying specific values within my multidimensional matrix by a scalar. My matrix has the following dimension: size(comDatabe) = 5 10 3 397 10 The third dimension is an x-y-z coordinate frame. Something went wrong and now my y-axis is defined upside down for one subject (#8 out...

MATLAB Access Classreg


matlab
So, I want to be able to look at (read: copy) MATLAB's NonLinearModel method of printing the regression results to the screen such as this. Nonlinear regression model: y ~ (alpha1 - alpha2*t^0.5) Estimated Coefficients: Estimate SE tStat pValue alpha1 1.0253 0.0082253 124.66 4.8823e-24 alpha2 0.0061783 0.00073277 8.4314 4.4834e-07 Number...

Matlab — SVM — All Majority Class Predictions with Same Score and AUC = .50


matlab,svm,auc
I'm having a weird problem in training an SVM with an RBF kernel in Matlab. The issue is that, when doing a grid search, using 10-fold cross-validation, for the C and Sigma values I always get AUC values equal to approximately .50 (varying between .48 and .54 depending) -- I...

Sorting in Ruby on rails


ruby-on-rails,json,sorting
I'm very new to Ruby on rails. I try to edit the following api. I want to sorting "can_go" which is true are shown at the top of list. I added this row before sending data, but the result is still order by "user_id". user_infos.sort { |a, b| - (a['can_go']<=>b['can_go'])...

xcorr function with impulse response


matlab,filtering,convolution
I'm trying to design a Wiener filter in Matlab for a deconvolution problem but I'm having a lot of problems. I have a gaussian white noise process with a variance of 1.2 and a impulse response which has length two. Its values are g(0) = 5 and g(1) = 4....

How to normalise polynomial coefficients in a fraction?


matlab,polynomial-math
I have the following code: syms z Gc=1.582*(1-0.3679*z^-1)/(1+.418*z^-1); Ghp=.3679*(z^-1)*(1+.718*z^-1)/((1-z^-1)*(1-.3679*z^-1)); T=(Gc*Ghp)/(1+Gc*Ghp); clipboard('copy', latex(simplifyFraction(T))); Which results in following for T: How can I normalise coefficients? I.e. I want the z2 in denominator and z in numerator to have the coefficient of 1. Is there any function in Matlab to do so?...

Sort array by keys in custom order


php,arrays,sorting
I have the following multidimensional array Array ( [June 2015] => Array ( [LOW] => Array ( [0] => 160.50 ) [MEDIUM] => Array ( [0] => 0.00 ) [HIGH] => Array ( [0] => 60.80 ) ) [July 2015] => Array ( [MEDIUM] => Array ( [0] => 226.00...

solve symbolic system of equations inside an array


matlab,system,equation
sorry if it already has a answer..i tried other links but it didn't understand i have 2 1*63 array .landa and v. each of their members are syms. and each v member is a function of all landa members.i have already calculated v members and they are all symbolic equations...

two dimensional unique values in Matlab


arrays,matlab
I have two vectors, one of them stores the width dimension of a set of images and another one the height of these set of images. I want to use these values as two dimensional vectors [width height] and store them in a matrix. The first line, for instance, keeps...

Operating a C++ class from Matlab without mex [closed]


c++,matlab
Is there an alternative way to call a C++ class using MATLAB, and operate its methods on MATLAB variables?

Creating a cell array of workspace variables without manually writing them all out. MATLAB


matlab
I have 20 column vectors in the workspace and I'd like them all to be contained within a cell array. They're all labelled data1, data2, data3,...data20. Is there a faster way to do this than: cell = {data1, data2, data3, data4, data5, data6, ...} all the way up to data20?...

Plot multiple functions on one figure


matlab,matlab-figure
I'm struggling to plot multiple functions on one figure. Here is the code that I have: syms t a; a=0.9514; F1=0.5*sqrt(3*t^2); F2=-0.28375*t^2+1.155*a*(t-a)+1; F3=1; E1=diff(F1,t); E2=diff(F2,t); E3=diff(F3,t); I want to plot E1, E2 and E3, each only within a certain range, to make a "composite" line. I've tried plotting with ezplot...

Sort List of Numbers according to Custom Number Sequence


list,python-2.7,sorting
Question :A set of numbers will be passed as input. Also the redefined relationship of the digits 0-9 in ascending order will be passed as input. Based on the redefined relationship, the set of numbers must be listed in ascending order. Input Format: The first line will contain the the...

Should checking loop conditions be counted towards total number of comparisons?


c++,algorithm,sorting,c++11
I have implemented three different sorting algorithms and now I want to confirm that my approach of counting the total number of comparisons is correct. In my mind, the number of comparisons shouldn't be tied to the conditional branches because if the condition isn't met, the comparison was still made...

how to calculate probability for each class for predicate with knn without fitcknn?


matlab
my matlab version is 2012a. when I use fitcknn,has this error: Undefined function 'fitcknn' for input arguments of type 'cell'. how to calculate probability for each class for predicate with knn without fitcknn? after use this code, I want to calculate prob_estimates for each neighbors: knn = ClassificationKNN.fit(trainsample, trainlabel,'NumNeighbors',7); y...

Interpolation inside a matrix. Matlab


matlab,matrix
I have a matrix looks like: 0 0 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 2 0 0 0 0 0 1 0 1 0 0 0 1 0 4 0 0 0 0 0 3 0 0 6 0 0 4...

How to solve for matrix in Matlab?


matlab,matrix,least-squares
How can I solve , where and and in the least squares sense in matlab? So I'd like to have the minimizing as output....

Can we add a statement in between MATLAB codes?


matlab
Is it possible to add statements in between the codes. For example: If I have a code like this, r(:,1) = a(:,1) - a(:,2); Then can I write it as, r(:,1) = a(:,1)("this is a constant") - a(:,2)("this is a variable"); ...

function wait to execute


matlab,events,delay
In Matlab functions can be started at events,but occasionally, like with the resize function, the events are called in rapid order and the function is called many times in succession, which can cause weird behavior and lag. Is there a way to have it listen for the event but only...

elastic search sort in aggs by column


sorting,elasticsearch,group-by,order
I am trying to sort in elastic search in aggs, equivalent in mysql "ORDER BY Title ASC/DESC". Here is the index structure: 'body' => array( 'mappings' => array( 'test_type' => array( '_source' => array( 'enabled' => true ), 'properties' => array( 'ProductId' => array( 'type' => 'integer', 'index' => 'not_analyzed'...

Find column with unique values move to the first column and sort worksheets


excel,vba,excel-vba,sorting
I have 2 worksheets with the same headers in different orders. Headers are I.D, Name, Department, Sales, Start date, End Date and a few others. What I am aiming to do is search through the workbooks in which the headers may be in different orders, find the column which has...

Connecting two binary objects in matlab


matlab,image-processing
I have a binary matrix containing several binary objects and I want to bridge between them. Actually I have the following picture: And the result has to be like this: Is there any function or a shortcut way, other than loops, for this problem?...

How to switch Matlab plot tick labels to scientific form?


matlab,plot
I have a semilogarithmic plot which works so far with semilogx. Now I would like to change the tick labels. Now it says 10^8 10^9 ... 10^13, but I would like to see 1e8, 1e9, ... 1e13 on the x-axis. How can I change that? Cheers Manuel...

Sort function giving floating point exception for a large input of 0's


c++,sorting,radix-sort,floating-point-exceptions
I have written a code for this problem: Given a list of non negative integers, arrange them such that they form the largest number. For example, given [3, 30, 34, 5, 9], the largest formed number is 9534330. Note: The result may be very large, so you need to return...

Why black surf from this Matlab command?


matlab,time-frequency
Code tfr = abs ( tfr ); [row_size, column_size] = size(tfr); tfr = tfr(1:round(row_size/2), 1:row_size); surf(tfr); view(2); I get in R2014b of OSX 10.10.3 Yosemite but rotating around shows that the cells should not be black Why is the output black? I wonder if this is a hardware problem or...

How to force my output data in a inputdlg on Matlab be a double?


matlab,typeconverter
I'm currently using a MATLAB to work and I need some help: I need to convert my output data (variable: units) be a double instead of a cell because I must perform a sum: units = inputdlg(question,title); sum = units + i; I've tried this code also but didn't solve...

Create string without repeating the same element jn the string (Matlab)


string,matlab
I have a string "FDFACCFFFBDCGGHBBCFGE" . Could anyone help me to generate a new string with the same order but no element inside repeated twice. Thanks ! The expected output should be like this : "FDACBGHE"...

what does ellipsis mean in a Matlab function's argument list?


matlab
What is the ellipsis for in this Matlab statement? frame = insertObjectAnnotation(frame, 'rectangle', ... bboxes, labels); ...I could not find in their online doc....

thicken an object of image to a curve in matlab


matlab,image-processing
I have a labeled matrix containing two objects. How can I thicken an object to a curve? Actually I have the following image: and I want this: Each pixel of the resulting curve is the median of each column. But if you have another idea, it is acceptable, because I...

Sorting a HTML structure


javascript,html,sorting
Im trying to sort a div structure based on a paramter using a small javscript i found. It seems to not perform exactly as expected. I understand the sorting function is not parsing the values perfectly... This is the sorting logic is use... <script type="text/javascript"> // Sorting Logic $(function() {...

Javascript Sorting Array of Objects [duplicate]


javascript,arrays,sorting,object
This question already has an answer here: Sorting an array of JavaScript objects 14 answers (Please excuse any errors - this is my first post and I am also relatively new to Javascript) I'm trying to sort an array of objects by a specific property value in Javascript. I...

Sum NA values in r


r,sorting,na
I am using a dataframe that has multiple NA values so I was thinking about sorting the attributes based on their NA values. I was trying to use a for loop and this is what I have so far: > data <- read.csv("C:/Users/Nikita/Desktop/first1k.csv") > for (i in 1:length(data) ) {...

Create mask from bwtraceboundary in Matlab


image,matlab,image-processing,mask,boundary
I'm trying to create a mask (or similar result) in order to erase pieces of a binary image that are not attached to the object surrounded by the boundary. I saw this thread (http://www.mathworks.com/matlabcentral/answers/120579-converting-boundary-to-mask) to do this from bwboundaries, but I'm having trouble making suitable changes to it. My goal...

WPF Listbox Collection custom sort


wpf,sorting,listbox,compare,collectionview
I have a listbox DropPrice MyPrice Price1 Price2 I want to sort it like this Price1 Price2 DropPrice MyPrice I mean, if there's an item that starts with the sequence "price", it gets priority, else the smallest string should get the priority. My source code: var lcv = (ListCollectionView)(CollectionViewSource.GetDefaultView(_itemsSource)); var...

Giving a string variable as a filename in matlab


string,matlab,filenames
I am using the below mentioned code to get the file names of images according to their id's from images_1 text file as strings and use them to read the images from their directory image_count=1; for image_count=1:6 file=fopen('D:\Academics\New folder\CUB_200_2011\images_1.txt','r'); C = textscan(file, '%s'); original_image=imread('D:\Academics\New folder\CUB_200_2011\images\%s','C{1}{2*(image_count)}'); imshow(original_image) end I am able...

Sorting vector of Pointers of Custom Class


c++,sorting,c++11,vector
I have vector<FPGA*> current_generation_, which I'd like to sort by FPGA member fitness_ using the sort_members function. Applicable code follows: bool sort_members (FPGA* fpga_first, FPGA* fpga_second) { return (fpga_first->fitness() < fpga_second->fitness()); }; fpga.hpp #include <vector> class FPGA { public: explicit FPGA(int input_gates, int output_gates, int normal_gates); const int fitness(); protected:...

Javascript sort array of objects in reverse chronological order


javascript,arrays,sorting
I have an array of objects which holds a list of jobs and I would like to sort them in reverse chronological order as they would appear on a resume for example. I came up with the below solution which 'seems' to work but I was wondering if there is...

Reading all the files in sequence in MATLAB


matlab,image-processing
I am trying to read all the images in the folder in MATLAB using this code flst=dir(str_Expfold); But it shows me output like this. which is not the sequence as i want. Can anyone please tell me how can i read all of them in sequence? for giving downmark, please...

Sort when values are None or empty strings python


python,list,sorting,null
I have a list with dictionaries in which I sort them on different values. I'm doing it with these lines of code: def orderBy(self, col, dir, objlist): if dir == 'asc': sorted_objects = sorted(objlist, key=lambda k: k[col]) else: sorted_objects = sorted(objlist, key=lambda k: k[col], reverse=True) return sorted_objects Now the problem...

Sort multiple columns of Excel in VBA given the top-left and lowest-right cell


excel,vba,excel-vba,sorting
I am trying to sort these three columns (Sort By Col-2) in excel using VBA. Top-left (Row number and Column number e.g. 1,1) and lowest-right cell (Row number and Column number e.g. 9,3) are known. Every cell contains the values of String type. Input: Col-1 Col-2 Col-3 P1 I1 XYZ...

Given an array/object of datetimes, how can I return an array/object of the times sorted by hour and times sorted by days of the week


php,arrays,sorting,datetime,laravel
PHP/Laravel, I'm getting an array of objects that includes date times for each record. I need to generate analytics on objects on an hourly basis of a day and daily basis of a week. So for example: For a date range of 1/1/2015 - 1/10/2015 I return 100 records all...

Animate through multiple 2D Matlab plots


matlab,plot
I have multiple 2D line plots in Matlab (they represent some wave moving through space). Each plot represents the wave at some time t. I want to animate through these plots (i.e. show the first plot for a fraction of a second, then show the next one, and the next,...

Why can't I calculate CostFunction J


matlab,machine-learning
This is my implementation of CostFunctionJ: function J = CostFunctionJ(X,y,theta) m = size(X,1); predictions = X*theta; sqrErrors =(predictions - y).^2; J = 1/(2*m)* sum(sqrErrors); But when I try to enter the command in MATLAB as: >> X = [1 1; 1 2; 1 3]; >> y = [1; 2; 3];...