unix,grep,cut , How to change horizontal input to vertical output


How to change horizontal input to vertical output

Question:

Tag: unix,grep,cut

I am using UNIX Korn shell. I am trying to create a program in Unix that will search a text file which is shown below:

Last Name:First Name:City:State:Class:Semester Enrolled:Year First Enrolled

Gilman:Randy:Manhattan:KS:Junior:Spring:2010
Denton:Jacob:Rochester:NY:Senoir:Fall:2009
Goodman:Joshua:Warren:MI:Freshman:Summer:2014
Davidson:Blair:Snohomish:WA:Sophmore:Fall:2013
Anderson:Neo:Seattle:WA:Senoir:Spring:2008
Beckman:John:Ft. Polk:LA:Freshman:Spring:2014

Then take that line and cut it out and display it vertically. So if I search Gilman. It would produce:

Gilman
Randy
Manhattan
KS
Junior
Spring
2010

However included in this I should also be able to produce the following layout:

Last Name: Gilman
First Name: Randy
City: Manhattan
State: KS
Class: Junior
Semester Enrolled: Spring
Year First Enrolled: 2010

Now I have figured out most of it which I will display in my code below:

cat<<MENULIST
                         A - Add Student Information 
                         D - Delete Student Information
                         M - Modify Student Information
                         I - Inquiry on a Student
                         X - Exit

MENULIST

    echo -en '\n'
    echo -en '\n'
    echo "                      Pleasa choose one of the following:                      "

    #take input from operation

    read choice

    case $choice in
        a|A) ;; 
        d|D) ;;
        m|M) ;;
        i|I)   
            #Create Inguiry Message
            clear
            echo "                                 Record Inquiry                                 "
            echo -en '\n'
            echo -en '\n'
            echo "What is the last name of the person"
            #Gather search parameter
            read last_name
            grep -i "$last_name" $1 | cut -f 1-7 -d ':' ;;


        x|X) echo "The program is ending" ; exit 0;;
        *) echo -en '\n' ;echo "Not an acceptable entry." ;echo "Please press enter to try again"; read JUNK | tee -a $2 ; continue
    esac

I am just really having trouble on redirecting input to the proper output. There is much more to the program but this is the relevant portion for this problem. Any help would be appreciated.

EDIT:

Ok the final answer is shown below:

while IFS=: read c1 c2 c3 c4 c5 c6 c7 rest ; do
                case "$c1" in
                    "$last_name" )
                        echo -e "Last Name: $c1\nFirst Name: $c2\nCity: $c3\nState: $c4\nClass: $c5\nSemester Enrolled: $c6\nYear First Enrolled: $c7\n\n"
                    ;;
                esac
            done < $1

This performed the intended outcome correctly thank you everyone for your help. I still have some other questions but I will make a new topic for that so it doesn't get to jumbled together.


Answer:

Replace your line

 grep -i "$last_name" $1 | cut -f 1-7 -d ':' ;;

with

  awk -F: -vnameMatch="$last_name" \
   '$1==nameMatch{
      printf("LastName:%s\nFirstName:%s\nCity:%s\nState:%s\nClass:%s\nSemester Enrolled:%s\nYear First Enrolled:%s\n\n", \
             $1, $2, $3, $4, $5, $6, $7)
    }' $1 
;;

It's pretty much the same idea in ksh.

while IFS=: read c1 c2 c3 c4 c5 c6 c7 rest ; do
  case "$c1" in 
    "$last_name" )
        printf "LastName:%s\nFirstName:%s\nCity:%s\nState:%s\nClass:%s\nSemester Enrolled:%s\nYear First Enrolled:%s\n\n",
             "$c1" "$c2" "$c3" "$c4" "$c5" "$c6" "$c7"
      ;;
   esac
  done < $1

I think I've got all the syntax right, but don't have the energy tonight to test :-/ ... If you can use this, and there are problems, post a comment and I'll clean it up.

IHTH.

EDIT:

#Case statement for conducting an inquiry of the file
        i|I)   
            #Create Inguiry Message
            clear
            echo "                                 Record Inquiry                                 "
            echo -en '\n'
            echo -en '\n'
            echo "What is the last name of the person:"
            #Gather search parameter
            read last_name
            while IFS=: read c1 c2 c3 c4 c5 c6 c7 rest ; do
                case "$c1" in
                    "$last_name" )
                        printf "Last Name:%s\nFirst Name:%s\nCity:%s\nState:%s\nClass:%s\nSemester Enrolled:%s\nYear First Enrolled:%s\n\n", "$c1", "$c2", "$c3", "$c4", "$c5", "$c6", "$c7"
                    ;;
                esac
            done < $2
        #Case statement for ending the program          
        x|X) echo "The program is ending" ; exit 0;;

The error message is:

./asg7s: line 26: syntax error at line 93: `)' unexpected

Line 93 is

x|X) echo "The program is ending" ; exit 0;;

Kind of weird because I didn't mess with that part of the program so I know it has to be something in the portion I changed.

RLG

Adding something extra so I can save RLG edit without other's "approval"


Related:


Redirect output from file to stdout


bash,shell,unix,stdout
I have a program that can output its results only to files, with -o option. This time I need to output it to console, i.e. stdout. Here is my first try: myprog -o /dev/stdout input_file But it says: /dev/ not writable I've found this question that's similar to mine, but...

Mounting GEOM_ELI Encrypted ZFS Pool as root


unix,encryption,freebsd,boot,zfs
I have a 3 Disk RAIDz1 Pool, encrypted with AES128 in GEOM_ELI, that I have been using in FreeNAS since version 8. There have been many zpool upgrades, and over all I am very happy with ZFS. Lately however I have been growing frustrated with FreeNAS. Largely many bugs that...

Using pure Python over grep?


python,grep,sublimetext3
I am not familiar with grep as I've always been on a Windows system so when someone suggested I add these lines to my code, I'm a little confused... grep = 'grep -E \'@import.*{}\' * -l'.format(name) proc = Popen(grep, shell=True, cwd=info['path'], stdout=PIPE, stderr=PIPE) From my understanding, this is trying to...

Split a column's values into multiple rows awk/unix/python


python,unix,awk
Here is a sample row of input data : header: id,indicator,{(pid,days_remaining)} row: id_558314,1,{(property_66021,7),(property_24444,1),(property_285395,6)} Expected output of data : header: id,indicator,pid,days_remaining row1: id_558314,1,property_66021,7 row2: id_558314,1,property_24444,1 row3: id_558314,1,property_285395,6 Is this something easily doable using awk/unix/python? Thanks in advance. Your time and help is appreciated....

What is wrong with these formats for Unix Shell Scripting Comparing Strings [duplicate]


bash,unix
This question already has an answer here: bash output command not found 4 answers I've been reading through StackExchanges posts on scripting and none of the things that I have tried are working as I would expect. Also this is my 1st shell script. # Script File Begin echo...

How to do the expansion of variable in shell script? [duplicate]


linux,bash,shell,unix,ksh
This question already has an answer here: Bash expand variable in a variable 4 answers I hava a column in database which contains the following data: sql_proc|test_sql.sql|/home/Desktop/myfile.txt|$IMP_FILES/myFolder|convert I have fetched this into a variable and I have used the cut command with "|" as delimiter and saved each field...

storing 'du' result in a variable [duplicate]


bash,unix,putty
This question already has an answer here: How to return the output of program in a variable? 4 answers I am using Putty with bash-4.2. Therein, I am outputting file size with: du -m myfile.csv which returns: 1.25 myfile.csv How do i store this line in a variable so...

gzip and pipe to output (performance consideration)


linux,unix,encryption,gzip,solaris
q1) Can i check if I do a gzip -c file | encrypt (some parameters) a) does gzip print out the output line by line and pipe it to the encrypt function or b) gzip will be perform 1st, then the output will be pipe all at once to the...

C++ Compile error on NetBSD '::system' has not been declared


c++,unix,boost,g++,netbsd
I have some C++ code that compiles nicely on Linux but so far I'm having trouble getting it to compile correctly on NetBSD. These are my includes: #include <fstream> #include <iostream> #include <cstdlib> #include <unistd.h> #include <iomanip> #include <boost/regex.hpp> I compile with this command: g++ -v -O2 -fno-strict-aliasing -I /usr/pkg/include...

How to replace newlines/linebreaks with a single space, but only if they are inside of start/end regexes?


regex,linux,shell,unix,replace
I have an extremely big (many GBs in size) file that looks like [x data1 data2 data3 data4 y] [a data5 data 6 data7 data 8 b> [x data y] ...and so on How to replace newlines (which might be surrounded by spaces) with a single space, but only if...

Calling find more than once on the same folder tree


linux,bash,shell,unix,find
I'm running a find command multiple times on the same group of files. The results of my find commands are usually disjoint sets, AKA I'm running find -mmin +35; find -mmin -25, and doing different things to the results. It seems sort of silly to search through the entire file...

grep string between two other strings as delimiters


regex,powershell,grep
I have to do a report on how many times a certain CSS class appears in the content of our pages (over 10k pages). The trouble is, the header and footer contains that class, so a grep returns every single page. So, how do I grep for content? EDIT: I...

Enumerating executable files in C


c,unix,directory,ansi
I am trying to enumerate only files with the execute (+x) bit set. My code seems to list all files. It also seems to enumerate directory and the above directory which I do not want. Example: .. should_not_be_executable.sh . Is there a way to filter '..' and '.' without strstr()?...

Multiple line search in a file using java or unix command


java,shell,unix,command
I want to find the following pattern in a file like subclass "Pool1" 11:22:33:44:55:66 { dynamic; } the above pattern I have to find in a file. How can I find either using java or Unix command?...

Unable to connect to mysql after automating mysql-server installation in bash script


mysql,linux,bash,unix
Generates a random string: passdb="$(< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c${1:-2};)" Feeds passdb to debconf i.e to set random passwords for mysql users: debconf-set-selections <<< 'mysql-server mysql-server/root_password password $passdb' debconf-set-selections <<< 'mysql-server mysql-server/root_password_again password $passdb' apt-get -y install mysql-server It get's installed successfully but I'm getting an error after...

Improving a sort program's efficiency; displaying column sort % completion in terminal


python,perl,sorting,unix
I have a big pipe-delimited input file approx 6 million lines as below: 24|BBG000SJFVB0|EQ0000000009296012|OI SA-ADR|OIBR/C|US|ADR|Equity 16|BBG002PHVB83|EQ0000000022353186|BLOOM SELECT INCOME FUND|BLB-U|CT|Closed-End Fund|Equity -50|BBG000V0TN75|EQ0000000010271114|MECHEL-PREF SPON ADR|MTL/P|US|ADR|Equity 20|BBG002S0ZR60|EQ0000000022739316|DIVIDEND 15 SPLIT CORP II-RT|DF-R|CT|Closed-End Fund|Equity -20|BBG001R3LGM8|EQ0000000017879513|ING FLOATING RATE SENIOR LOA|ISL/U|CT|Closed-End Fund|Equity 0|BBG006M6SXL2|EQ0000000006846232|AA...

find numbers divisible by 3 in csv file using shell script


bash,shell,unix,awk
I have csv file having content like below : 1|2|3 4|5|6 7|8|9 Now I would like to find the numbers which are divisible by 3 using shell scripting. I would like to use awk command for this. I am learning shell scripting. So could you please help me out to...

Identifying when a file is changed- Bash


bash,shell,unix
So in my bash shell script, I have it running through a for loop. Inside the for loop, I use find "$myarray[i]" >> $tmp to look for a certain directory each time through the loop. Sometimes, it finds the variable in myarray[i] and sometimes it doesn't. When it does find...

How to fix btrfs root inode errors


unix,fsck,btrfs
Running btrfsck, or more officially, btrfs check --repair gives this output stating that there are root inode errors. The repair command does not fix the issue and reruns will display the same output. The system is fully mountable and operational, but I cannot perform advanced operations on the partition (resizing)....

Why is Unix/Terminal faster than R?


r,bash,shell,unix,terminal
I'm new to Unix, however, I have recently realized that very simple Unix commands can do very simple things to large data set very very quickly. My question is why are these Unix commands so fast relative to R? Let's begin by assuming that the data is big, but not...

grep in R does not find rows when terms are ORed together


regex,r,grep
I am new to R. I am working on a very large dataset with 1M+ rows and 1000+ columns that needs serious data cleaning. I have managed to clean the data for most part, however that has left me with some messy repetitive code. I am trying to use grep...

Why can I view some Unix executable files in Mac OS X and not others?


git,bash,shell,unix,binary
I am on a Macbook Pro on Mac OS X 10.10 (Yosemite). When I go to /usr/bin, git is there as a unix executable file. When I open it up in Sublime Text, all I get is unreadable machine code. However, when I open up a different Unix executable fileā€”in...

Random password generate in shell script with one special character


bash,shell,unix,random,passwords
I have following code </dev/urandom tr -dc '[email protected]#$%&_+=' | head -c 16 which is randomly generating password perfectly. I want two changes: It should only contain 1 special symbols that i have listed there It should choose random length of the password I tried with length = $(($RANDOM%8+9)) then putting...

ImageMagick - Making 2 GIFs into side by side GIFs using IM convert


image,unix,imagemagick,imagemagick-convert
I have 2 GIFs that are the same length. I want to put the GIFs beside each other to have 1 GIF with both playing at the same time. I have tried to use the convert tool with: convert +append 1.gif1 2.gif output.gif However, this seems to blend all the...

Parse IP and Download-Total from mikrotik


regex,bash,sed,grep
I wanna extract IP and download-total from mikrotik command /queue simple print stat Here's some example : 0 name="101" target=192.168.10.101/32 rate=0bps/0bps total-rate=0bps packet-rate=0/0 total-packet-rate=0 queued-bytes=0/0 total-queued-bytes=0 queued-packets=0/0 total-queued-packets=0 bytes=17574842/389197663 total-bytes=0 packets=191226/308561 total-packets=0 dropped=9/5899 total-dropped=0 1 name="102" target=192.168.10.102/32 rate=0bps/0bps total-rate=0bps packet-rate=0/0 total-packet-rate=0 queued-bytes=0/0 total-queued-bytes=0...

How to remove characters before and including an underscore?


linux,string,bash,unix,awk
In bash scripting what's an efficient way to do the following please? var="fooo_barrrr" What is the best way to remove all characters before and including the '_' so that var becomes "barrrr" please?...

Remove specific (complex) line from MANY files (sed?)


regex,bash,sed,grep
Server was hacked, every php file on the server now starts with malicious code: <?php if(!isset($GLOBALS["\x61\156\x75\156\x61"])) { $ua=strtolower($_SERVER["\x48\124\x54\120\x5f\125\x53\105\x52\137\x41\107\x45\116\x54"]); if ((! strstr($ua,"\x6d\163\x69\145")) and (! strstr($ua,"\x72\166\x3a\61\x31"))) $GLOBALS["\x61\156\x75\156\x61"]=1; } ?><?php $zdsnpbzghe =...

How to extract single-/multiline regex-matching items from an unpredictably formatted file and put each one in a single line into output file?


linux,shell,unix,replace,grep
I have a very huge file which looks like this: <a>text</a>text blah <b>data1</b>abc<b>data2</b> <b>data3</b>blahblah <c>text</c> <d>text</d> <x>blahblah<b>data4 data5 data6</b> <b>data7 </x> That is, its formatting is unpredictable. I need to extract each <b>...</b> item (it might contain multiline text!) and put every one of them in a single separate line....

Capture tee's argument inside piped Perl execution


perl,unix
How to capture piped command's argument ? I use : perl my_script.pl -some_args | tee arg_filename How to get arg_filename 's value inside my_script.pl ? CONTEXT I need to send this filename in a mail which my_script.pl sends at the end. I need to use tee because we dump huge...

Extracting columns within a range AWK


unix,awk
I've been trying to extract lines where a number in one columns falls within a particular range. Lets say I have a file that looks as so: chrom prediction chrom1 0.75 chrom2 0.5 chrom4 0.76 If I wanted to print lines where the prediction value was in the range from...

How to copy matches from an extremely large file if it contains no newlines?


python,linux,bash,perl,grep
The problem is I cannot avoid working with extremely big files which contain no newlines in them: <a>text1</a>...gigabytes of data here, all in one single line...[a text to extract b> What should I do if I want to copy matches from this file (putting every match in a separate line,...

Use Unix Executable File to Run Shell Script and MPKG File


osx,shell,unix
I have 2 shell scripts and 2 mpkg installer, I am trying to use an unix excitable file to run them all. here is the script I have, but it always has error message "No such file or directory" ? #!/bin/sh # Find the absolute script current path path=$( cd...

Converting values from for loop to json format


linux,unix
Below is a snippet of a for loop where I sort txt file names. I am then trying to save the results in a json format file. However it results in an invalid json format due to the last , inserted in obj. How could i convert to json format...

Join Statement omitting entries


unix,join,hidden-characters
Using: Unix 2.6.18-194.el5 I am having an issue where this join statement is omitting values/indexes from the match. I found out the values are between 11-90 (out of about 3.5 Million entries) and I have tried to look for foreign characters but I may be overlooking something (Tried cat -v...

Doubts with for loop in Unix


bash,shell,unix,for-loop
Sorry if its sounds dumb but I'm currently studying for an exam to apply for a job (administrative one) and I learnt about java, php, python but not much about unix scripts. I can see it's very similar to php in structure and I have many questions about it ....

Delete some lines from text using Linux command


linux,shell,sed,grep,pattern-matching
I know how to match text using regex patterns but not how to manipulate them. I have used grep to match and extract lines from a text file, but I want to remove those lines from the text. How can I achieve this without having to write a python or...

Reading rsync source from file results in improper parsing of file names with white space


bash,shell,unix,scripting,rsync
I wrote a simple script that searches through a specific directory defined by the variable "SCOPE" producing a list of directories that were modified within the past 24 hours printing them to a temp file. The first line of the file is deleted (to exclude the root level of the...

How do I grep GZ files to extract PNG files?


grep,png,gz
Ok, so I have a tone of .GZ files in a folder, and I'm looking to recursively look through each one of them and extract all the PNG files into another destination folder. How would I do that? EDIT: I've been using this command from the terminal to find a...

how to check a directory exist and can write file


oracle,shell,unix,sql-loader
I need to check directory test_file exist, if yes then need to check directory test_file is having write permission so that files will be writen in the same. this functionality i needed in shell script(ksh). What would be return code from sqlldr for single table load and multi table...

Matching string inside file and returning result


regex,string,bash,shell,grep
I've got a few peculiar issues with trying to search for a string inside of a .db file. The way I tried was by using grep, which does apparently find the string(s), although this is the output: $ grep "ext" *.db Binary file enormous.db matches There are a couple problems...

Bash, Using grep, sed or awk to extract section of text and then match


bash,awk,sed,grep
I have a text file and want to extract all interfaces matching "blue" random text random text random text random text random text int 1 random text blue random text random text int 2 random text random text red random text int 3 random text random text random text blue...

how to deletes line from a text file that are taken from another file [duplicate]


shell,awk,sed,grep,sh
This question already has an answer here: Remove duplicates from text file based on second text file 4 answers I have a data.txt file with a lot of lines in it and a lines.txt that contains some lines. I want to delete all lines from data.txt that match any...

How to open a new terminal from my working terminal with same directory in Linux?


linux,unix,ssh
Is there any command to open a new terminal from my working terminal with same directory while working with ssh in Linux? e.g., I am working in the following terminal. My pwd is /home/work/Kayan01/test_run I want to open a new terminal directly from my working terminal, so that the pwd...

Selecting unique lines based on two columns


unix,awk
I've been trying to figure out a way to select lines that are unique based on the values of two columns. For example, here is a sample/example of my file: chr1 10 12 chr1 10 12 chr1 10 11 chr1 9 12 chr2 15 20 And this is what I...

pass enter key from Java to Shell script


java,unix,jsch
I am trying a Java program to run multiple commands in unix environment. I would need to pass 'ENTER' after each command. Is there some way to pass enter in the InputStream. JSch jsch=new JSch(); Session session=jsch.getSession("MYUSERNAME", "SERVER", 22); session.setPassword("MYPASSWORD"); Properties config = new Properties(); config.put("StrictHostKeyChecking", "no"); session.setConfig(config); session.connect(); Channel...

How to find average and maximum in an interval using Shell [closed]


linux,bash,shell,unix,awk
I would like to extract sum, mean and average in each 6 numbers interval from a column. I found many discussions related to this problem, but all those are for whole column. e.g. To compute sum of a column: awk '{sum+=$1} END { print sum}' To calculate Average: awk '{sum+=$1}...

SFTP Processbuilder


java,unix,sftp,processbuilder,ssh2-sftp
I have to invoke unix commands from Java and I have to use ProcessBuilder. I want to login through SFTP using key authentication and I want to SFTP the file from local directory to remote location. ProcessBuilder pb = new ProcessBuilder("sftp","-oIdentityFile=privateKey","-b","commands.txt","[email protected]"); Content in commands.txt is put localFileDirectory remoteDirectory exit Is...

How to pass url to wget


grep,pipe,wget
If I have a document with many links and I want to download especially one picture with the name: www.website.de/picture/example_2015-06-15.jpeg. How can I write a command that downloads me automatically exactly this one I extracted out of my document. My idea would be this but I'll get a failure message...

grep first n rows, return file name only


regex,linux,search,grep
I can do the following to search for what I need and return the file name: grep -l "mysearchstring" ./*.xml However the files I am searching are huge so this takes forever. The string I am searching will appear in the first 200 rows so how can I search only...