linux,find,unzip , Pretty recursive directory and file print

Pretty recursive directory and file print


Tag: linux,find,unzip

I am building a JAVA app that saves the output of bash commands into a list.

@root ~/a $ zipinfo -1 

[email protected] ~/a $ find .

The idea is to compare files and directories from the zip to what is on the disk and remove any differences from the disk. In this example test/c.txt should only be removed.

As you can see the format is different. Which command do I need to have the same style as zipinfo -1?

I tried commands like:

ls -R

ls -LR | grep "" 

find . -exec ls -dl \{\} \; | awk '{print $9}'


One way to remove the . prefix is to use sed. For example:

find . ! -name . | sed -e 's|^\./||'

The ! -name . removes the . entry from the list. The sed part removes the ./ from the beginning of each line.

It should give you pretty much the same format as your zipinfo, though perhaps not the same order.

Note: the above suggestion is compatible with both Linux and other unix versions such as MacOS X. On Linux specifically you can achieve the same result with:

find . ! -name . -printf "%P\n"

The -printf predicate is specific to the Linux findutils, and the %P format prints each found file, removing the search directory from its beginning.


Extra backslash when storing grep in a value

In a bash script I have: Check="grep -e '"'\(-S mount\)'"' /etc/audit/audit.rules" set -x When you run it it shows it as: CHECK='grep -e '\''\(-S mount\)'\'' /etc/audit/audit.rules' Now it works exactly what I want but I want to understand it. Why is there 2 extra \'s?...

While loop in bash using variable from txt file

I am new to bash and writing a script to read variables that is stored on each line of a text file (there are thousands of these variables). So I tried to write a script that would read the lines and automatically output the solution to the screen and save...

Delete some lines from text using Linux command

I know how to match text using regex patterns but not how to manipulate them. I have used grep to match and extract lines from a text file, but I want to remove those lines from the text. How can I achieve this without having to write a python or...

Shell script to loop over files with same names but different extensions

For example say I have: filename1.ext1 filename1.ext2 filename2.ext1 filename2.ext2 I need to write a shell script to feed these files into a program like so: program filename1.ext1 filename1.ext2 program filename2.ext1 filename2.ext2 Additionally the .ext1 files must be entered first and the .ext2 files second. Any help would be appreciated....

Git post-receive hook is not executed

The following post-receive hook: #!/bin/bash echo "-> Post-receive test" is not executed when pushing to my remote repository. The remote is ssh://[email protected]:2222/home/git/repo.git (it's a VM) and works, as when I manually checkout it I see the modifications I've made. Some additional informations ... $ ls -al /home/git drwxr-xr-x 7 git...

How to append entry the end of a multi-line entry using any of stream editors like sed or awk

I am trying to use a script to append the host name at the end of a multi-line entry of a specific Host_Alias field in sudoers file. The current sudoers file has an entry similar to : Host_Alias srv_linuxestate= \ host10,host12,host13,host1,host50,\ host16,host1,host2,host11,host15,host21,\ host3,host14 My required output would be something like...

how to spy on linux binaries for testing of shell scripts

sorry if this is a duplicate, i had no idea what to search for... my use case is more complex, but can be narrowed down to the following problem: i want to run a bash script, which invokes all sorts of binaries, for example: grep. i want to assert that...

Ignore first few lines and last few lines in a file Linux

I have a file like this and would like to print $0 except the first two and last three lines in linux. Tried awk command but no luck, is there any options I am using the following command - I suppose I am doing something wrong, but not able to...

Get system startup time (without reading /proc/uptime)

I cannot open /proc/uptime due to open_basedir restriction. The command uptime is too old and doesn't have the -s flag support. How can I - in PHP - get time when the server started? My current code is this, but it does not work on the production server (for the...

make error during building webkitgtk

I use UBuntu 14.04 LTS. I need to build webkitgtk 2.8.3 Here is an example instruction which I have used: linuxfromscratch When I run sudo make -j8 I get following log: Scanning dependencies of target JavaScriptCore-4-gir Scanning dependencies of target fake-generated-webkitdom-headers [ 0%] Scanning dependencies of target WebKit2-fake-api-headers Scanning dependencies...

How to extract single-/multiline regex-matching items from an unpredictably formatted file and put each one in a single line into output file?

I have a very huge file which looks like this: <a>text</a>text blah <b>data1</b>abc<b>data2</b> <b>data3</b>blahblah <c>text</c> <d>text</d> <x>blahblah<b>data4 data5 data6</b> <b>data7 </x> That is, its formatting is unpredictable. I need to extract each <b>...</b> item (it might contain multiline text!) and put every one of them in a single separate line....

Syncing Vagrant VMs across different physical servers

I'm using Vagrant to deploy my VMs and my current setup looks like this: server1 = VM1, VM2, VM3 ( main production server ) server2 = VM1, VM2, VM3 ( backup server ) My questions is, can I somehow sync the VMs across the different physical servers in case one...

How to remove characters before and including an underscore?

In bash scripting what's an efficient way to do the following please? var="fooo_barrrr" What is the best way to remove all characters before and including the '_' so that var becomes "barrrr" please?...

Change a Script to a For Do Done Loop

I have a script that I need to turn into a loop, the script works exactly as I need it to (My Awk-fu is extremely weak, so be nice) when I put in the file name like the example below. #!/bin/bash awk -v FS='(<LastName>|<\/LastName>)' '{print $2}' 17822624.xml >> test.csv awk...

Java read bytes from Socket on Linux

I'm trying to send a file from my Windows machine to my Raspberry-Pi 2, and I have a client and a server. The client should be able to send a zip file over the network to my server on my linux machine. I know my client and server work on...

Django MySQLClient pip compile failure on Linux

Django documentation as of v1.8 recommends using mysqlclient connector for the framework. I'm attempting to pip install the package on Ubuntu 14.04 with Python 3.4 and running into a GCC error that I'm unable to find reference to. I'm not an expert on compiling software, so was hoping somebody can...

fread(), solaris to unix portability and use of uninitialised values

Valgrind found the following error and I, after reading the documentation, the code and other questions in here couldn't figure it out why. Valgrind: first warning ~$ valgrind --vgdb=yes --vgdb-error=0 --read-var-info=yes --leak-check=yes --track-origins=yes debitadmin* debitadmin ==20720== Conditional jump or move depends on uninitialised value(s) ==20720== at 0x4013BC6: initialise (dbg.c:199) ==20720==...

How to replace newlines/linebreaks with a single space, but only if they are inside of start/end regexes?

I have an extremely big (many GBs in size) file that looks like [x data1 data2 data3 data4 y] [a data5 data 6 data7 data 8 b> [x data y] ...and so on How to replace newlines (which might be surrounded by spaces) with a single space, but only if...

sed and PHP tags

I have a problem using SED. I have a php file whit this structure in the very first line: <?php echo 'first' ?><?php echo 'second' ?><?php echo 'third';?> I'm trying to remove the first two statements and have as a result: <?php echo 'third';?> I've tried this code: sed -i...

How to extract first letters of dashed separated words in a bash variable?

I would like to extract the first letter of dashed separated words value of my bash variable, like this: MY_TEXT=this-is-my-custom-text I would like to create a second variable like this: MY_INITIALS=timct...

Linux - sh script - download multiple files from FTP

I need script that can download files from ftp via my sh code I have use expect with ftp, but if I do for loop inside code, I got wrong # args: should be "for start test next command" while executing "for v in "a b c"" My code /usr/bin/expect...

Calling find more than once on the same folder tree

I'm running a find command multiple times on the same group of files. The results of my find commands are usually disjoint sets, AKA I'm running find -mmin +35; find -mmin -25, and doing different things to the results. It seems sort of silly to search through the entire file...

AWK count number of times a term appear with respect to other columns

Given a CSV file: id, fruit, binary 1, apple, 1 2, orange, 0 3, pear, 1 4, apple, 0 5, peach, 0 6, apple, 1 How can i calculate for each unique values in fruit, the number of times the binary value =1 / number of occurences of that fruit...

How can I resolve the “Could not fix timestamps in …” “…Error: The requested feature is not implemented.”

I have been trying to build a project in F# on Linux that I have located here on github. It's a basic kata project that I am working on as a demo. However on Linux (specifically Ubuntu 14.04 LTS Desktop) I haven't been able to get it to build yet...

What are correct permissions for Linux Apache2 PHP 5.3 log file?

I discovered the reason why I was not getting entries into my php_errors.log file related to permissions. Right now, I have set it to 666 (rw-rw-rw-) but surely this is a security weakness? Thus, my question. php.ini file: error_log /var/log/httpd/php_errors.log log_errors On # ls -ld /var/log /var/log/httpd /var/log/httpd/php_errors.log drwxr-xr-x 6...

How to check which symbols on my shared library have non-position independent code (PIC)?

I'm trying to build a .deb package with debuild -i -us -uc -b and in the end I see: Now running lintian... warning: the authors of lintian do not recommend running it with root privileges! W: libluajit-5.1-2: hardening-no-relro usr/lib/powerpc64le-linux-gnu/ E: libluajit-5.1-2: shlib-with-non-pic-code usr/lib/powerpc64le-linux-gnu/ W: luajit: hardening-no-relro usr/bin/luajit-2.1.0-alpha W: luajit: binary-without-manpage...

linux running command as root from c code that run as normal user

I have a c++ code and I need to running from it a command to adjust the system time. so I thought using system("su root -c date hh:mm") command from my c++ code. The problem is that when I write 'su root -c date hh:mm' in the terminal its requires...

How to make new line when using echo to write a file in C

hi ı am triying to take the data of files in a folder with system function this is the code char path[100],command[120]; scanf("%s",&path); sprintf(command,"echo $(ls %s) > something.txt",path); system(command); but when I look to the something.txt there is no new line. This is the output, all on one line with...

Force linux to use php as php55

Helllo, I have a little problem here. I have PHP 5.3 installed and it's accessible via php command, I also have php 5.5 accessible as php55 command. Now I need to force linux to use php55 when I write php command. Simple way: How I switch the versions of php?...

What does it indicate if /proc/PID/maps shows zero for all addresses?

I'm debugging a problem with a Linux DNS server. Curiously, when I look at /proc/PID/maps for the DNS server process, this is what I get: 00000000-00000000 r-xp 00000000 00:0e 2344 /usr/sbin/unbound 00000000-00000000 rw-p 00000000 00:0e 2344 /usr/sbin/unbound 00000000-00000000 ---p 00000000 00:00 0 00000000-00000000 rw-p 00000000 00:00 0 [heap] 00000000-00000000 rw-p...

Permission denied in find: why do we need 2>&1?

I was recently searching for a way to filter out 'Permission denied' errors while searching for a file using the find command, and I found this link: How can I exclude all "permission denied" messages from "find"? Here's an the answer that worked for me from the link: find ....

ret_from_syscall source code and when it is called

In the below call trace we see function called ret_from_syscall. Which function is this ? When it will called during system call ? Where is the corresponding code for this ? May 7 16:40:34.322086 warn TCU-0 kernel: [cf83ddc0] [00009751] 0x9751 (unreliable) May 7 16:40:34.322086 warn TCU-0 kernel: [cf83ddd0] [c00469ac] do_syslog+0x198/0x424...

How could I simulate a lack of file descriptor?

I would like to provoke a situation where there is no file descriptor remaining. I have been thinking about 2 possibilities: Opening thousands of files randomly until the result of open is -1 Set a very low number of file descriptor available (let's say stdin, stdout and stderr). How would...

shell script to shutdown/restart Linux system

Is there any suitable shell script for shutting down or restarting a Linux machine? I have tried a shell script for shutdown, but when I enter sudo shutdown it will ask for the password. How we can enter the password using the script?

Linux-wget command

I need a quick help on customizing my wget command in a shell script: The wget command looks something like this: wget http://infamvn:8081/nexus/content/groups/LDM_REPO_LIN64/com/infa/com.infa. products.ldm.ingestion.server.scala/ Here I'd like to add the in a variable, so I created a script something like this: n = wget...

How to look at the top 30 lines (or head) of all files inside a directory?

If I have 30+ files in a directory /a.cpp /b.cpp /c.cpp ... What's the fastest way to export the head (top n lines) of all files into one single document? Thanks....

AWK write to new column base on if else of other column

I have a CSV file with columns A,B,C,D. Column D contains values on a scale of 0 to 1. I want to use AWK to write to a new column E base in values in column D. For example: if value in column D <0.7, value in column E =...

how to modify an array value with given index?

I want to modify an array cell, which I can do when I know the cell as a number. However here my cell position is given by $i. pomme[`${i}`]="" I tried without the `` and it doesn't work either? How am I suppose to do it?...

Finding the average of a column excluding certain rows using AWK

I want to calculate the average of the 5th column (last column) excluding the rows with the value "9999". Would appreciate your feedback. 77.300 16 1 3.6112914285714268 9.4 77.300 16 2 -0.001737142857145102 20.0 77.300 16 3 5.1570742857142857 8.9 77.300 17 0 3.6112914285714268 8.9 77.300 17 1 2.9484342857142849 11.7 77.300 17...

Linux: Using split on limited space

I have a huge file on a linux machine. The file is ~20GB and the space on my box is ~25GB. I want to split the file into ~100mb parts. I know theres a 'split' command but that keeps the original file. I don't have enough space to keep the...

debian 8: deb command not found. How can i fix it? [closed]

I am trying to install graphics card Nvidia Geforce 660 on my desktop. Os: debian 8. Here is the wiki which gives guidelines : But as soon as I am on the first step: deb jessie main contrib non-free bash: deb: command not found How can I install...

BASH - conditional sum of columns and rows in csv file

i have CSV file with some database benchmark results here is the example: Date;dbms;type;description;W;D;S;results;time;id Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;570;265;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;420;215;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;500;365;50 Mon Jun 15 14:22:20 CEST 2015;sqlite;on-disk;text;2;1;1;530;255;50 Mon Jun 15 14:22:20 CEST 2015;hsql;on-disk;text;2;1;1;870;265;99 Mon Jun 15 14:22:20 CEST 2015;hsql;on-disk;text;2;1;1;620;215;99...

Bash modify CSV to change a field

I have a very big CSV file (aprox. 10.000 rows and 400 columns) and I need to modify certain columns (like 15, 156, 220) to change format from 20140321132233 to 2014-03-21 13:22:33. All fields that I need to modify are datetime. I saw some examples using awk but for math...

Extract minor version from kernel to bash variable

I am new to bash and writing a script that needs to compare the minor version of the kernel to see if it is greater than or equal to 10, and exit if it is not. Currently I have something like: KERNEL = $(uname -r) declare -i MINOR_VERSION=$(echo $KERNEL |...

How does the kernel separate threads from processes

Suppose I have a browser process like Firefox, that has pid = 123. Firefox has 5 opened tabs each running in a separate thread, so in total it has 5 threads. So I want to know in depth, how the kernel will separate the process into the thread to execute...

How to open a new terminal from my working terminal with same directory in Linux?

Is there any command to open a new terminal from my working terminal with same directory while working with ssh in Linux? e.g., I am working in the following terminal. My pwd is /home/work/Kayan01/test_run I want to open a new terminal directly from my working terminal, so that the pwd...

NASM: copying a pointer from a register to a buffer in .data

I am new to asm. I am trying to copy a pointer from a register to a .data variable using NASM, on linux 64-bit. Concider this program: section .data ptr: dq 0 section .text global _start _start: mov [ptr], rsp mov rax, 60 mov rdi, 0 syscall Here I try...

Using an ad-hoc libc with a tool which is an argument of another tool

I need to use a particular libc to run a tool (cp). The problem is that this tool has to be used as argument of another tool (for example timeout) and I don't want to use the modified libc with this one. I tried to do: timeout 10 LD_LIBRARY_PATH=/path/to/mod/libc/ cp...

sed string with special character New

when i use this script to replace : by sed -i -e "s/" config.xml i have the error : sed: -e expression n°1, caractère 41: option inconnue pour `s' can anyone help me thanks in advance regard, Youssef...