bash,sed,grep , How to use sed to replace a string within a string?


How to use sed to replace a string within a string?

Question:

Tag: bash,sed,grep

I read the following article "Using grep and sed to find and replace a string" but how can I extend it to chain multiple greps. For example I have the following directory/file structure

dir1/metadata.txt
dir2/metadata.txt

dir1/metadata.txt has

filename1 '= 1.0.0'
filename2 '= 1.0.0'

dir2/metadata.txt has

filename1     '= 1.0.0'
long_filename '= 1.0.0'

In other words, both dir1/metadata.txt and dir2/metadata.txt contain "filename '1.0.0'" but the spaces between the "filename" and the "'1.0.0'" in each file is different.

Now I want to replace filename1's associated version to '2.0.0' in ALL metadata.txt files so the resulting files look like...

dir1/metadata.txt has

filename1 '= 2.0.0'
filename2 '= 1.0.0'

dir2/metadata.txt has

filename1     '= 2.0.0'
long_filename '= 1.0.0'

I'm trying

find . -name metadata.txt | xargs grep filename1 | sed -i "s/1\.0\.0/2.0.0/g" <some option here>

but I know the "some option here" part. Any clues?


Answer:

Try the following:

Linux:

find . -name metadata.txt \
  -exec sed -i "s/^\(filename1[[:space:]]\{1,\}'= \)1\.0\.0/\12.0.0/" {} +

OSX / BSD:

find . -name metadata.txt \
  -exec sed -i '' "s/^\(filename1[[:space:]]\{1,\}'= \)1\.0\.0/\12.0.0/" {} +

Note: The only reason why platform-specific commands are required is that GNU sed and BSD sed interpret the nonstandard -i option, which specifies the suffix to use for an optional backup of the original file, differently: GNU sed considers the option-argument for -i optional, whereas BSD sed considers it mandatory, requiring an explicit argument to specify the empty string (indicating the desire not to create a backup file)

POSIX-compliance notes:


Related:


storing 'du' result in a variable [duplicate]


bash,unix,putty
This question already has an answer here: How to return the output of program in a variable? 4 answers I am using Putty with bash-4.2. Therein, I am outputting file size with: du -m myfile.csv which returns: 1.25 myfile.csv How do i store this line in a variable so...

How to change svn:externals from bash file non-interactive


bash,svn,svn-externals
I have multi externals need to be set within a file externals.txt and I attempt to change the svn:externals from a bash: svn pe svn:externals svn://hostname/branchname -F extenals.txt But the command throws out an error: svn: E205007: None of the environment variables SVN_EDITOR, VISUAL or EDITOR are set, and no...

using sed to replace a line with back slashes in a shell script


regex,bash,shell,ssh,sed
I am trying to replace the bottom one of these 2 lines with sed in a file. <rule>out_prefix=orderid ^1\\d\+ updatemtnotif/</rule>\n\ <rule>out_prefix=orderid ^2\\d\+ updatemtnotif/</rule>\n\ And the following command seems to do that when executed as a command at the bash prompt sed -i '[email protected]_prefix=orderid ^2\\\\d\\+ updatemtnotif/@out_prefix=orderid ^2\\\\d\\+ updatemtnotif_fr/@g' /opt/temp/rules.txt however, when...

[: : integer expression expected


bash,if-statement,integer
COUNTER=0 let COUNTER=COUNTER+1 count=`ssh -i /var/www/.ssh/id_rsa_root -o stricthostkeychecking=no $host $cmd` count1=`echo $count | awk '{print $4}'` printf "count1 : $count1\n" result1=${count1/.*} if [ "$result1" -ge "0" ]; then echo $host else echo $host exit fi If the value of $result1 is INTEGER and greater than zero, it'll goto IF loop...

AWK|BASH, use double FS and ternary operator


bash,awk
Is it possible? I was wondering how to do: Count fields differentiated by comma. Only the obtained first field of the previous step, count words differentiated by space. If there is more than 2 words, print NF, otherwise $0. Input cellular biol immunogenet, rosario escuela estadist, medellin medellin Expected output...

How to extract first letters of dashed separated words in a bash variable?


linux,string,bash,shell,variables
I would like to extract the first letter of dashed separated words value of my bash variable, like this: MY_TEXT=this-is-my-custom-text I would like to create a second variable like this: MY_INITIALS=timct...

Bash script that removes C source comments


bash
I need to write a bash script which copies all .c files from a directory $2 to another directory $1, but without comments. I only have to remove comments that begin with //, might have tabs/spaces before the comment, but not letters. Also, I need to do it with only...

Mac OSX - Allow for user input in shell script via GUI or Prompt


osx,bash,shell
first time post so please let me know how I could improve.. I created a shell script which requires a person to input their name and then generates a report. The script works as needed when chmod'd into an executable script and run from terminal. But, I would like to...

how to deletes line from a text file that are taken from another file [duplicate]


shell,awk,sed,grep,sh
This question already has an answer here: Remove duplicates from text file based on second text file 4 answers I have a data.txt file with a lot of lines in it and a lines.txt that contains some lines. I want to delete all lines from data.txt that match any...

Why does `sort file > file` result in an empty file? [duplicate]


bash
This question already has an answer here: bash redirect input from file back into same file 7 answers When you try to sort a file in-place with sort afile > afile you silently end up with afile being an empty file. Why is that? I'd expect either an error...

While loop in bash using variable from txt file


linux,bash,rhel
I am new to bash and writing a script to read variables that is stored on each line of a text file (there are thousands of these variables). So I tried to write a script that would read the lines and automatically output the solution to the screen and save...

Matching string inside file and returning result


regex,string,bash,shell,grep
I've got a few peculiar issues with trying to search for a string inside of a .db file. The way I tried was by using grep, which does apparently find the string(s), although this is the output: $ grep "ext" *.db Binary file enormous.db matches There are a couple problems...

Aggregate failure codes in bash


bash
I've got a script which has multiple stages, and at each stage it's possible to fail, but the script can carry on running. Concretely, I generate some json, and check if the diff is correct. The diff could be wrong, but it doesn't stop the next stage of json being...

How do I check whether a file or file directory exist in bash?


bash,if-statement
I currently have this bash script (which is located in my home directory, i.e., /home/fusion809/ and I am running it as root as it's necessary for the icon copying lines): cd /home/fusion809/Pictures/Icon* declare -a A={Arch,Debian,Fedora,Mageia,Manjaro,OpenSUSE} declare -a B={Adwaita,Faenza,gnome,Humanity} for i in $A; do for j in $B; do if test...

Extra backslash when storing grep in a value


linux,bash
In a bash script I have: Check="grep -e '"'\(-S mount\)'"' /etc/audit/audit.rules" set -x When you run it it shows it as: CHECK='grep -e '\''\(-S mount\)'\'' /etc/audit/audit.rules' Now it works exactly what I want but I want to understand it. Why is there 2 extra \'s?...

how to immediately login through ssh?


bash,ssh
Here is my problem. I need to run a command ./deploy.sh -u 1540 This will fetch version 1540 of deploy.sh on SVN When I do, the script access SVN and ask for a password. I'm using ssh. It will first ask me a password since it guesses my SVN login...

Assign and use of a variable in the same subshell


bash,scope,subshell
I was doing something very simple like: v=5 echo "$v" and expected it to print 5. However, it does not. The value that was just set is not available for the next command. I recently learnt that "In most shells, each command of a pipeline is executed in a separate...

How do I silence the HEAD of a curl request while using the silent flag?


bash,shell,curl,command-line,pipe
When I run the curl command and direct the data to a file, I get back the content of the site as expected. $ curl "www.site.com" > file.txt $ head file.txt Top of site ... However, this command shows a progress bar, which I do not want: % Total %...

Shell script to loop over files with same names but different extensions


linux,bash,shell
For example say I have: filename1.ext1 filename1.ext2 filename2.ext1 filename2.ext2 I need to write a shell script to feed these files into a program like so: program filename1.ext1 filename1.ext2 program filename2.ext1 filename2.ext2 Additionally the .ext1 files must be entered first and the .ext2 files second. Any help would be appreciated....

BASH - conditional sum of columns and rows in csv file


linux,bash,csv,awk
i have CSV file with some database benchmark results here is the example: Date;dbms;type;description;W;D;S;results;time;id Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;570;265;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;420;215;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;500;365;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;530;255;50 Mon Jun 15 14:22:20 CEST 2015;hsql;on-disk;text;2;1;1;870;265;99 Mon Jun 15 14:22:20 CEST 2015;hsql;on-disk;text;2;1;1;620;215;99...

AWK write to new column base on if else of other column


linux,bash,shell,awk,sed
I have a CSV file with columns A,B,C,D. Column D contains values on a scale of 0 to 1. I want to use AWK to write to a new column E base in values in column D. For example: if value in column D <0.7, value in column E =...

how to modify an array value with given index?


arrays,linux,bash
I want to modify an array cell, which I can do when I know the cell as a number. However here my cell position is given by $i. pomme[`${i}`]="" I tried without the `` and it doesn't work either? How am I suppose to do it?...

Permission denied in find: why do we need 2>&1?


bash,error-handling,find,io-redirection
I was recently searching for a way to filter out 'Permission denied' errors while searching for a file using the find command, and I found this link: How can I exclude all "permission denied" messages from "find"? Here's an the answer that worked for me from the link: find ....

AWK count number of times a term appear with respect to other columns


linux,shell,command-line,awk,sed
Given a CSV file: id, fruit, binary 1, apple, 1 2, orange, 0 3, pear, 1 4, apple, 0 5, peach, 0 6, apple, 1 How can i calculate for each unique values in fruit, the number of times the binary value =1 / number of occurences of that fruit...

Python: can't access newly defined environment variables


python,bash,environment-variables
I can't access my env var: import subprocess, os print os.environ.get('PATH') # Works well print os.environ.get('BONSAI') # doesn't work But the env var is well added in my /home/me/.bashrc: BONSAI=/home/me/Utils/bonsai_v3.2 export BONSAI And I can access this env var from a new terminal....

Identifying when a file is changed- Bash


bash,shell,unix
So in my bash shell script, I have it running through a for loop. Inside the for loop, I use find "$myarray[i]" >> $tmp to look for a certain directory each time through the loop. Sometimes, it finds the variable in myarray[i] and sometimes it doesn't. When it does find...

Bash script using sed acts differently when passing variable


bash,sed
I have a script that I am writing that checks a value and then based on the value modifies it. I am trying to understand why it works this one way and not the other. Based on the google and stackoverflow searches I did, nothing really fits what I am...

Why can I view some Unix executable files in Mac OS X and not others?


git,bash,shell,unix,binary
I am on a Macbook Pro on Mac OS X 10.10 (Yosemite). When I go to /usr/bin, git is there as a unix executable file. When I open it up in Sublime Text, all I get is unreadable machine code. However, when I open up a different Unix executable file—in...

linux - running a process and tailing a file simultaneously


bash,shell,tail
I want a run a long task on a remote machine (with python fabric using ssh). It logs to a file on the remote machine. What I want to do is to run that script and tail (actively display) the log file content until the script execution ends. The problem...

print filenames into scripts in bash


bash,printf,echo
How do you print each name of a file from a directory into a string and make make new scripts? to print each file name for i in `ls new_manifest*`; do echo $i; done but when I try and print the rest of the string with $i like this is...

sed and PHP tags


regex,linux,sed
I have a problem using SED. I have a php file whit this structure in the very first line: <?php echo 'first' ?><?php echo 'second' ?><?php echo 'third';?> I'm trying to remove the first two statements and have as a result: <?php echo 'third';?> I've tried this code: sed -i...

bash interactive script pass input


bash,command-line-arguments
I'm running the following interactive Jar. java -jar script.jar argument-line-here \n Now I'm creating a bash script which runs the jar file. How do I pass the argument line and the conformation "\n" to this interactive script? these are input lines for the script. This question has some answers, expect...

How to test if a command is a shell reserved word?


bash,shell
I am writing a bash script and I would like to verify if a string is a shell reserved word (like if, for, alias, etc...). How can I can do this?...

Macports switch PHP CLI version


php,bash,drupal,terminal,macports
I'm trying to switch my Terminal PHP version to 5.4 because I ran into some issues with Drush while updating my Drupal core. http://drupal.stackexchange.com/questions/112090/drush-command-errors The reason for these issues is my Terminal PHP version is different then my localhost. php -v in Terminal returns PHP 5.5.13 (cli) but my localhost...

How to append entry the end of a multi-line entry using any of stream editors like sed or awk


linux,bash,awk,sed,sh
I am trying to use a script to append the host name at the end of a multi-line entry of a specific Host_Alias field in sudoers file. The current sudoers file has an entry similar to : Host_Alias srv_linuxestate= \ host10,host12,host13,host1,host50,\ host16,host1,host2,host11,host15,host21,\ host3,host14 My required output would be something like...

shell script cut from variables


bash,shell,shellcode
The file is like this aaa&123 bbb&234 ccc&345 aaa&456 aaa$567 bbb&678 I want to output:(contain "aaa" and text after &) 123 456 I want to do in in shell script, Follow code be consider #!/bin/bash raw=$(grep 'aaa' 1.txt) var=$(cut -f2 -d"&" "$raw") echo $var It give me a error like...

Capitalize all files in a directory using Bash


osx,bash,rename
I have a directory called "Theme_1" and I want to capitalize all the filenames starting with "theme". The first letter of every file name is lowercase and I want it to be upcase. I've tried this but all the letters in the filename are upcase, which is not what I...

Delete some lines from text using Linux command


linux,shell,sed,grep,pattern-matching
I know how to match text using regex patterns but not how to manipulate them. I have used grep to match and extract lines from a text file, but I want to remove those lines from the text. How can I achieve this without having to write a python or...

Bash modify CSV to change a field


linux,bash,awk
I have a very big CSV file (aprox. 10.000 rows and 400 columns) and I need to modify certain columns (like 15, 156, 220) to change format from 20140321132233 to 2014-03-21 13:22:33. All fields that I need to modify are datetime. I saw some examples using awk but for math...

Replace [a-z],[a-z] with [a-z], [a-z] and keep the letters


bash,awk,sed
How can I replace [a-z],[a-z] with [a-z], [a-z] and keeping the letters? Input suny stony brook, stony brook,usa. Output suny stony brook, stony brook, usa. What I have tried sed 's/[a-z],[a-z]/[a-z], [a-z]/g' <<< "suny stony brook, stony brook,usa." sed 's/[a-z],[a-z]/, /g' <<< "suny stony brook, stony brook,usa." ...

Calling find more than once on the same folder tree


linux,bash,shell,unix,find
I'm running a find command multiple times on the same group of files. The results of my find commands are usually disjoint sets, AKA I'm running find -mmin +35; find -mmin -25, and doing different things to the results. It seems sort of silly to search through the entire file...

Finding the average of a column excluding certain rows using AWK


linux,bash,awk,scripting
I want to calculate the average of the 5th column (last column) excluding the rows with the value "9999". Would appreciate your feedback. 77.300 16 1 3.6112914285714268 9.4 77.300 16 2 -0.001737142857145102 20.0 77.300 16 3 5.1570742857142857 8.9 77.300 17 0 3.6112914285714268 8.9 77.300 17 1 2.9484342857142849 11.7 77.300 17...

sed string with special character New


linux,sed,special-characters
when i use this script to replace : mrm.fr.mycompany.com by 10.70.89.40:8081/artifactory sed -i -e "s/mrm.fr.mycompany.com/10.70.89.40:8081/artifactory/g" config.xml i have the error : sed: -e expression n°1, caractère 41: option inconnue pour `s' can anyone help me thanks in advance regard, Youssef...

shell script for counting replacements


bash,replace,count
Ten files located in a directory. Each file content has "apple" word. write a bash shell script to replace "apple" with "banana" in all ten files and Print the number of replacements for each file. Have tried in this way but dont know how to get number of replacement. can...

Bash alias function with predefined argument


bash
I have an alias gl which is an wrapper for git log. Basically like this function gitLog(){ if [ $# -eq 0 ] then git log else git log -n $1 } alias gl=gitLog I want to add an alias which just calls gitLog with an argument like this alias...