bash,unix,sh , Pass command line arguments from bash script to program verbatim, with escaping


Pass command line arguments from bash script to program verbatim, with escaping

Question:

Tag: bash,unix,sh

So, I have a bash script that needs to run X if --xyz flag is set, and Y otherwise. It also needs to pass all command line arguments through to the programs it runs. My problem with this is that I can't get it to pass the arguments exactly as they appear, instead it either escapes things that don't need escaped, or strips the escaping from things that should be escaped.

This is my test script:

#!/bin/sh
echo "[email protected]"
[email protected]
BASEDIR=$(dirname $0)
while test $# -gt 0
do
    case "$1" in
        --xyz) echo ${args/--xyz/}
            exit $?
            ;;
    esac
    shift
done

echo "$args"
printf %s "$args"
echo
echo "$(printf %s "$args")"
echo "$(printf %q "$args")"

I then test this like so:

[[email protected] X]$ ./test.sh -c Foo\ Bar
-c Foo Bar
-c Foo Bar
-c Foo Bar
-c Foo Bar
-c\ Foo\ Bar
[[email protected] X]$ echo -c Foo\ Bar
-c Foo Bar
[[email protected] X]$ echo "-c Foo\ Bar"
-c Foo\ Bar

I need it to be like the last command. Foo\ Bar retaining the escaping for the space, but no escaping for the space after -c.

I don't have any control over what arguments are passed into this script, so I can't do double escaping or anything like that.

How exactly can I reach my goal of passing command line arguments basically verbatim with escaping intact to another program?


Answer:

You can't (easily) convert an argument list to a string in a way that it can be converted back to a list. Fortunately, you rarely actually need to do that.

It's not clear to me exactly what your requirements are, but here's a simple script which seems to demonstrate the behaviour you expect:

# Function to execute if --xyz in argument list
X() { local -i i=0
      for arg in "[email protected]"; do echo X$((++i)): "$arg"; done
}
# Function to call otherwise
Y() { local -i i=0
      for arg in "[email protected]"; do echo Y$((++i)): "$arg"; done
}

# Select one or the other
x_or_y() {
  local arg
  # Scan arguments for --xyz (Not very precise)
  for arg in "[email protected]"; do
    case "$arg" in
      --xyz) X "[email protected]";
             return $?;;
      --)    break;;
    esac
  done
  # No --xyz found
  Y "[email protected]"
}

# Demonstrate that arguments are passed verbatim:
$ x_or_y something "a b c" 'a backslash\inside'
Y1: something
Y2: a b c
Y3: a backslash\inside

$ x_or_y something --xyz "a b c" 'a backslash\inside'
X1: something
X2: --xyz
X3: a b c
X4: a backslash\inside

With bash, if you need to remove the --xyz argument, you can use an array. (You could also use set --, but I think the array is clearer):

x_or_y() {
  local -i i=1
  local arg
  local todo=Y
  local args=()
  # Scan arguments for --xyz (Not very precise)
  for arg in "[email protected]"; do
    ((++i))
    case "$arg" in
      --xyz) todo=X; break;;
      --)    args+=("$arg"); break;;
      *)     args+=("$arg");;
    esac
  done
  # Call with the args scanned and the rest of the args:
  "$todo" "${args[@]}" "${@:i}"
}

Edit: As Etan Reisner notes, it is possible to do this without an array. The array is a useful technique, but for this problem you can do it with substring expressions:

x_or_y() {
  local -i i
  local -i skip=0
  local todo=Y
  # Scan arguments for --xyz (Not very precise)
  for ((i=1; i<=$#; ++i)); do
    case "${!i}" in
      --xyz) todo=X; skip=1; break;;
      --)    break;;
    esac
  done
  # Call with the args scanned and the rest of the args:
  "$todo" "${@:1:i-1}" "${@:i+skip}"
}

Related:


print filenames into scripts in bash


bash,printf,echo
How do you print each name of a file from a directory into a string and make make new scripts? to print each file name for i in `ls new_manifest*`; do echo $i; done but when I try and print the rest of the string with $i like this is...

Bash script that removes C source comments


bash
I need to write a bash script which copies all .c files from a directory $2 to another directory $1, but without comments. I only have to remove comments that begin with //, might have tabs/spaces before the comment, but not letters. Also, I need to do it with only...

Use Unix Executable File to Run Shell Script and MPKG File


osx,shell,unix
I have 2 shell scripts and 2 mpkg installer, I am trying to use an unix excitable file to run them all. here is the script I have, but it always has error message "No such file or directory" ? #!/bin/sh # Find the absolute script current path path=$( cd...

Permission denied in find: why do we need 2>&1?


bash,error-handling,find,io-redirection
I was recently searching for a way to filter out 'Permission denied' errors while searching for a file using the find command, and I found this link: How can I exclude all "permission denied" messages from "find"? Here's an the answer that worked for me from the link: find ....

How do I check whether a file or file directory exist in bash?


bash,if-statement
I currently have this bash script (which is located in my home directory, i.e., /home/fusion809/ and I am running it as root as it's necessary for the icon copying lines): cd /home/fusion809/Pictures/Icon* declare -a A={Arch,Debian,Fedora,Mageia,Manjaro,OpenSUSE} declare -a B={Adwaita,Faenza,gnome,Humanity} for i in $A; do for j in $B; do if test...

Extra backslash when storing grep in a value


linux,bash
In a bash script I have: Check="grep -e '"'\(-S mount\)'"' /etc/audit/audit.rules" set -x When you run it it shows it as: CHECK='grep -e '\''\(-S mount\)'\'' /etc/audit/audit.rules' Now it works exactly what I want but I want to understand it. Why is there 2 extra \'s?...

AWK|BASH, use double FS and ternary operator


bash,awk
Is it possible? I was wondering how to do: Count fields differentiated by comma. Only the obtained first field of the previous step, count words differentiated by space. If there is more than 2 words, print NF, otherwise $0. Input cellular biol immunogenet, rosario escuela estadist, medellin medellin Expected output...

shell script for counting replacements


bash,replace,count
Ten files located in a directory. Each file content has "apple" word. write a bash shell script to replace "apple" with "banana" in all ten files and Print the number of replacements for each file. Have tried in this way but dont know how to get number of replacement. can...

bash interactive script pass input


bash,command-line-arguments
I'm running the following interactive Jar. java -jar script.jar argument-line-here \n Now I'm creating a bash script which runs the jar file. How do I pass the argument line and the conformation "\n" to this interactive script? these are input lines for the script. This question has some answers, expect...

Aggregate failure codes in bash


bash
I've got a script which has multiple stages, and at each stage it's possible to fail, but the script can carry on running. Concretely, I generate some json, and check if the diff is correct. The diff could be wrong, but it doesn't stop the next stage of json being...

how to modify an array value with given index?


arrays,linux,bash
I want to modify an array cell, which I can do when I know the cell as a number. However here my cell position is given by $i. pomme[`${i}`]="" I tried without the `` and it doesn't work either? How am I suppose to do it?...

Identifying when a file is changed- Bash


bash,shell,unix
So in my bash shell script, I have it running through a for loop. Inside the for loop, I use find "$myarray[i]" >> $tmp to look for a certain directory each time through the loop. Sometimes, it finds the variable in myarray[i] and sometimes it doesn't. When it does find...

Replace [a-z],[a-z] with [a-z], [a-z] and keep the letters


bash,awk,sed
How can I replace [a-z],[a-z] with [a-z], [a-z] and keeping the letters? Input suny stony brook, stony brook,usa. Output suny stony brook, stony brook, usa. What I have tried sed 's/[a-z],[a-z]/[a-z], [a-z]/g' <<< "suny stony brook, stony brook,usa." sed 's/[a-z],[a-z]/, /g' <<< "suny stony brook, stony brook,usa." ...

How to extract first letters of dashed separated words in a bash variable?


linux,string,bash,shell,variables
I would like to extract the first letter of dashed separated words value of my bash variable, like this: MY_TEXT=this-is-my-custom-text I would like to create a second variable like this: MY_INITIALS=timct...

BASH - conditional sum of columns and rows in csv file


linux,bash,csv,awk
i have CSV file with some database benchmark results here is the example: Date;dbms;type;description;W;D;S;results;time;id Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;570;265;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;420;215;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;500;365;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;530;255;50 Mon Jun 15 14:22:20 CEST 2015;hsql;on-disk;text;2;1;1;870;265;99 Mon Jun 15 14:22:20 CEST 2015;hsql;on-disk;text;2;1;1;620;215;99...

pass enter key from Java to Shell script


java,unix,jsch
I am trying a Java program to run multiple commands in unix environment. I would need to pass 'ENTER' after each command. Is there some way to pass enter in the InputStream. JSch jsch=new JSch(); Session session=jsch.getSession("MYUSERNAME", "SERVER", 22); session.setPassword("MYPASSWORD"); Properties config = new Properties(); config.put("StrictHostKeyChecking", "no"); session.setConfig(config); session.connect(); Channel...

How to extract single-/multiline regex-matching items from an unpredictably formatted file and put each one in a single line into output file?


linux,shell,unix,replace,grep
I have a very huge file which looks like this: <a>text</a>text blah <b>data1</b>abc<b>data2</b> <b>data3</b>blahblah <c>text</c> <d>text</d> <x>blahblah<b>data4 data5 data6</b> <b>data7 </x> That is, its formatting is unpredictable. I need to extract each <b>...</b> item (it might contain multiline text!) and put every one of them in a single separate line....

Mounting GEOM_ELI Encrypted ZFS Pool as root


unix,encryption,freebsd,boot,zfs
I have a 3 Disk RAIDz1 Pool, encrypted with AES128 in GEOM_ELI, that I have been using in FreeNAS since version 8. There have been many zpool upgrades, and over all I am very happy with ZFS. Lately however I have been growing frustrated with FreeNAS. Largely many bugs that...

linux - running a process and tailing a file simultaneously


bash,shell,tail
I want a run a long task on a remote machine (with python fabric using ssh). It logs to a file on the remote machine. What I want to do is to run that script and tail (actively display) the log file content until the script execution ends. The problem...

Bash modify CSV to change a field


linux,bash,awk
I have a very big CSV file (aprox. 10.000 rows and 400 columns) and I need to modify certain columns (like 15, 156, 220) to change format from 20140321132233 to 2014-03-21 13:22:33. All fields that I need to modify are datetime. I saw some examples using awk but for math...

Shell script to loop over files with same names but different extensions


linux,bash,shell
For example say I have: filename1.ext1 filename1.ext2 filename2.ext1 filename2.ext2 I need to write a shell script to feed these files into a program like so: program filename1.ext1 filename1.ext2 program filename2.ext1 filename2.ext2 Additionally the .ext1 files must be entered first and the .ext2 files second. Any help would be appreciated....

Capitalize all files in a directory using Bash


osx,bash,rename
I have a directory called "Theme_1" and I want to capitalize all the filenames starting with "theme". The first letter of every file name is lowercase and I want it to be upcase. I've tried this but all the letters in the filename are upcase, which is not what I...

Calling find more than once on the same folder tree


linux,bash,shell,unix,find
I'm running a find command multiple times on the same group of files. The results of my find commands are usually disjoint sets, AKA I'm running find -mmin +35; find -mmin -25, and doing different things to the results. It seems sort of silly to search through the entire file...

Mac OSX - Allow for user input in shell script via GUI or Prompt


osx,bash,shell
first time post so please let me know how I could improve.. I created a shell script which requires a person to input their name and then generates a report. The script works as needed when chmod'd into an executable script and run from terminal. But, I would like to...

Extracting columns within a range AWK


unix,awk
I've been trying to extract lines where a number in one columns falls within a particular range. Lets say I have a file that looks as so: chrom prediction chrom1 0.75 chrom2 0.5 chrom4 0.76 If I wanted to print lines where the prediction value was in the range from...

how to immediately login through ssh?


bash,ssh
Here is my problem. I need to run a command ./deploy.sh -u 1540 This will fetch version 1540 of deploy.sh on SVN When I do, the script access SVN and ask for a password. I'm using ssh. It will first ask me a password since it guesses my SVN login...

How to change svn:externals from bash file non-interactive


bash,svn,svn-externals
I have multi externals need to be set within a file externals.txt and I attempt to change the svn:externals from a bash: svn pe svn:externals svn://hostname/branchname -F extenals.txt But the command throws out an error: svn: E205007: None of the environment variables SVN_EDITOR, VISUAL or EDITOR are set, and no...

Why does `sort file > file` result in an empty file? [duplicate]


bash
This question already has an answer here: bash redirect input from file back into same file 7 answers When you try to sort a file in-place with sort afile > afile you silently end up with afile being an empty file. Why is that? I'd expect either an error...

storing 'du' result in a variable [duplicate]


bash,unix,putty
This question already has an answer here: How to return the output of program in a variable? 4 answers I am using Putty with bash-4.2. Therein, I am outputting file size with: du -m myfile.csv which returns: 1.25 myfile.csv How do i store this line in a variable so...

Bash script using sed acts differently when passing variable


bash,sed
I have a script that I am writing that checks a value and then based on the value modifies it. I am trying to understand why it works this one way and not the other. Based on the google and stackoverflow searches I did, nothing really fits what I am...

How to test if a command is a shell reserved word?


bash,shell
I am writing a bash script and I would like to verify if a string is a shell reserved word (like if, for, alias, etc...). How can I can do this?...

using sed to replace a line with back slashes in a shell script


regex,bash,shell,ssh,sed
I am trying to replace the bottom one of these 2 lines with sed in a file. <rule>out_prefix=orderid ^1\\d\+ updatemtnotif/</rule>\n\ <rule>out_prefix=orderid ^2\\d\+ updatemtnotif/</rule>\n\ And the following command seems to do that when executed as a command at the bash prompt sed -i '[email protected]_prefix=orderid ^2\\\\d\\+ updatemtnotif/@out_prefix=orderid ^2\\\\d\\+ updatemtnotif_fr/@g' /opt/temp/rules.txt however, when...

Macports switch PHP CLI version


php,bash,drupal,terminal,macports
I'm trying to switch my Terminal PHP version to 5.4 because I ran into some issues with Drush while updating my Drupal core. http://drupal.stackexchange.com/questions/112090/drush-command-errors The reason for these issues is my Terminal PHP version is different then my localhost. php -v in Terminal returns PHP 5.5.13 (cli) but my localhost...

Why can I view some Unix executable files in Mac OS X and not others?


git,bash,shell,unix,binary
I am on a Macbook Pro on Mac OS X 10.10 (Yosemite). When I go to /usr/bin, git is there as a unix executable file. When I open it up in Sublime Text, all I get is unreadable machine code. However, when I open up a different Unix executable fileā€”in...

How to append entry the end of a multi-line entry using any of stream editors like sed or awk


linux,bash,awk,sed,sh
I am trying to use a script to append the host name at the end of a multi-line entry of a specific Host_Alias field in sudoers file. The current sudoers file has an entry similar to : Host_Alias srv_linuxestate= \ host10,host12,host13,host1,host50,\ host16,host1,host2,host11,host15,host21,\ host3,host14 My required output would be something like...

While loop in bash using variable from txt file


linux,bash,rhel
I am new to bash and writing a script to read variables that is stored on each line of a text file (there are thousands of these variables). So I tried to write a script that would read the lines and automatically output the solution to the screen and save...

Python: can't access newly defined environment variables


python,bash,environment-variables
I can't access my env var: import subprocess, os print os.environ.get('PATH') # Works well print os.environ.get('BONSAI') # doesn't work But the env var is well added in my /home/me/.bashrc: BONSAI=/home/me/Utils/bonsai_v3.2 export BONSAI And I can access this env var from a new terminal....

shell script cut from variables


bash,shell,shellcode
The file is like this aaa&123 bbb&234 ccc&345 aaa&456 aaa$567 bbb&678 I want to output:(contain "aaa" and text after &) 123 456 I want to do in in shell script, Follow code be consider #!/bin/bash raw=$(grep 'aaa' 1.txt) var=$(cut -f2 -d"&" "$raw") echo $var It give me a error like...

Capture tee's argument inside piped Perl execution


perl,unix
How to capture piped command's argument ? I use : perl my_script.pl -some_args | tee arg_filename How to get arg_filename 's value inside my_script.pl ? CONTEXT I need to send this filename in a mail which my_script.pl sends at the end. I need to use tee because we dump huge...

Assign and use of a variable in the same subshell


bash,scope,subshell
I was doing something very simple like: v=5 echo "$v" and expected it to print 5. However, it does not. The value that was just set is not available for the next command. I recently learnt that "In most shells, each command of a pipeline is executed in a separate...

Bash alias function with predefined argument


bash
I have an alias gl which is an wrapper for git log. Basically like this function gitLog(){ if [ $# -eq 0 ] then git log else git log -n $1 } alias gl=gitLog I want to add an alias which just calls gitLog with an argument like this alias...

[: : integer expression expected


bash,if-statement,integer
COUNTER=0 let COUNTER=COUNTER+1 count=`ssh -i /var/www/.ssh/id_rsa_root -o stricthostkeychecking=no $host $cmd` count1=`echo $count | awk '{print $4}'` printf "count1 : $count1\n" result1=${count1/.*} if [ "$result1" -ge "0" ]; then echo $host else echo $host exit fi If the value of $result1 is INTEGER and greater than zero, it'll goto IF loop...

Matching string inside file and returning result


regex,string,bash,shell,grep
I've got a few peculiar issues with trying to search for a string inside of a .db file. The way I tried was by using grep, which does apparently find the string(s), although this is the output: $ grep "ext" *.db Binary file enormous.db matches There are a couple problems...

AWK write to new column base on if else of other column


linux,bash,shell,awk,sed
I have a CSV file with columns A,B,C,D. Column D contains values on a scale of 0 to 1. I want to use AWK to write to a new column E base in values in column D. For example: if value in column D <0.7, value in column E =...

Finding the average of a column excluding certain rows using AWK


linux,bash,awk,scripting
I want to calculate the average of the 5th column (last column) excluding the rows with the value "9999". Would appreciate your feedback. 77.300 16 1 3.6112914285714268 9.4 77.300 16 2 -0.001737142857145102 20.0 77.300 16 3 5.1570742857142857 8.9 77.300 17 0 3.6112914285714268 8.9 77.300 17 1 2.9484342857142849 11.7 77.300 17...

How do I silence the HEAD of a curl request while using the silent flag?


bash,shell,curl,command-line,pipe
When I run the curl command and direct the data to a file, I get back the content of the site as expected. $ curl "www.site.com" > file.txt $ head file.txt Top of site ... However, this command shows a progress bar, which I do not want: % Total %...