perl,unix , Capture tee's argument inside piped Perl execution

Capture tee's argument inside piped Perl execution


Tag: perl,unix

How to capture piped command's argument ?

I use :

perl -some_args  |  tee arg_filename

How to get arg_filename 's value inside ?

I need to send this filename in a mail which sends at the end.
I need to use tee because we dump huge output in the standard logging which we use inside but we print more relevant things on STDOUT and STDERR.
This script imports and uses additional proprietary libraries which could not be edited. But they too generate logs.


The short answer is - you can't.

tee is a separate process with it's own arguments. There is no way to access these arguments from that process. (well, I suppose you could run ps or something).

The point of tee is to take STDOUT write some of it to a log file, and pass through the rest of it down the 'pipe'. (Which results in printing it if nothing else).

What you could probably do instead is implement some manner of logging within your perl script, where the core functionality of tee is replicated by printing a line to both STDOUT and a designated log file.


Selecting unique lines based on two columns

I've been trying to figure out a way to select lines that are unique based on the values of two columns. For example, here is a sample/example of my file: chr1 10 12 chr1 10 12 chr1 10 11 chr1 9 12 chr2 15 20 And this is what I...

Extracting columns within a range AWK

I've been trying to extract lines where a number in one columns falls within a particular range. Lets say I have a file that looks as so: chrom prediction chrom1 0.75 chrom2 0.5 chrom4 0.76 If I wanted to print lines where the prediction value was in the range from...

Mounting GEOM_ELI Encrypted ZFS Pool as root

I have a 3 Disk RAIDz1 Pool, encrypted with AES128 in GEOM_ELI, that I have been using in FreeNAS since version 8. There have been many zpool upgrades, and over all I am very happy with ZFS. Lately however I have been growing frustrated with FreeNAS. Largely many bugs that...

Identifying when a file is changed- Bash

So in my bash shell script, I have it running through a for loop. Inside the for loop, I use find "$myarray[i]" >> $tmp to look for a certain directory each time through the loop. Sometimes, it finds the variable in myarray[i] and sometimes it doesn't. When it does find...

Creating a sequence of unique random digits

I have the following code use strict; use warnings; use 5.22.0; # Generating random seed using # Programming Perl p. 955 srand( time() ^ ($$ + ($$ << 15 ) ) ); # Generating code that could have duplicates my @code = ( (int(rand(9)) + 1), (int(rand(9)) + 1), (int(rand(9))...

Looping variables

I'm working with perl to make a script that will work with Dot products/assorted vector math. I've got a working script ( Still very much in progress/needs refinement ) that will do what I ask. #!/usr/bin/perl use strict; use warnings; use diagnostics; use Math::Vector::Real; use 5.010; use Math::Trig; my $source...

Why Filter::Indent::HereDoc complain when blank line in middle of HereDoc

I am trying Filter::Indent::HereDoc which allows one to indent the HereDocument. This is very useful, to be able to have HereDoc that flows with the code logic. From the above link When a 'here document' is used, the document text and the termination string must be flush with the left...

find numbers divisible by 3 in csv file using shell script

I have csv file having content like below : 1|2|3 4|5|6 7|8|9 Now I would like to find the numbers which are divisible by 3 using shell scripting. I would like to use awk command for this. I am learning shell scripting. So could you please help me out to...

Create unicode character with pack

I am trying to understand how Perl handles unicode. use feature qw(say); use strict; use warnings; use Encode qw(encode); say unpack "H*", pack("U", 0xff); say unpack "H*", encode( 'UTF-8', chr 0xff ); Output: ff c3bf Why do I get ff and not c3bf when using pack ?...

Check for decimal point and add it at the end if its not there using awk/perl

I have test.dat file with values given below: 20150202,abc,,,,3625.300000,,,,,-5,,,,,,,,,,,,,,,,,,,,,, 20150202,def,,,,32.585,,,,,0,,,,,,,,,,,,,,,,,,,,,, 20150202,xyz,,,,12,,,,,0.004167,,,,,,,,,,,,,,,,,,,,,, My expected output is shown below: 20150202,abc,,,,3625.300000,,,,,-5.,,,,,,,,,,,,,,,,,,,,,, ^. added here 20150202,def,,,,32.585,,,,,0.,,,,,,,,,,,,,,,,,,,,,, ^. added here 20150202,xyz,,,,12.,,,,,0.004167,,,,,,,,,,,,,,,,,,,,,, ^. added here So if column 6 and 11 doesn't have decimal point in it, then we should add '.' at the end of...

How to replace newlines/linebreaks with a single space, but only if they are inside of start/end regexes?

I have an extremely big (many GBs in size) file that looks like [x data1 data2 data3 data4 y] [a data5 data 6 data7 data 8 b> [x data y] ...and so on How to replace newlines (which might be surrounded by spaces) with a single space, but only if...

Windows/Linux child process STDIN differences

I built a simple text processing script at work to be used by another program. When I was done, someone remembered that the script needs to not block STDIN/STDOUT for the tool using it to work right, and modified the script accordingly. The script opens *nix's cat in a subprocess...

Taking multiple header (rows matching condition) and convert into a column

Hello I have a file that has multiple Headers in it that I need to have turned into column values. The file looks like this: Day1 1,Smith,London 2,Bruce,Seattle 5,Will,Dallas Day2 1,Mike,Frisco 4,James,LA I would like the file to end up looking like this: Day1,1,Smith,London Day1,2,Bruce,Seattle Day1,5,Will,Dallas Day2,1,Mike,Frisco Day2,4,James,LA The file...

Get ISO DateTime with only core modules in Perl?

I would like to get a date-time string such as 2015-06-17 10:20:34 with only core modules. The reason of this is that cpan install DateTime takes ages on my machine because of the tests and the fetch of all the dependencies. If I give my Program to my colleagues. They...

How to fix btrfs root inode errors

Running btrfsck, or more officially, btrfs check --repair gives this output stating that there are root inode errors. The repair command does not fix the issue and reruns will display the same output. The system is fully mountable and operational, but I cannot perform advanced operations on the partition (resizing)....

-M Script start time minus file modification time, in days

I would expect -M $_ to be negative, but it is zero for: perl -E 'qx(touch $_), sleep(5), say -M for "/tmp/file"' Does perldoc mentions such behavior?...

Perl: Using Text::CSV to print AoH

I have an array of hashes (AoH) which looks like this: $VAR1 = [ { 'Unit' => 'M', 'Size' => '321', 'User' => 'test' } { 'Unit' => 'M' 'Size' => '0.24' 'User' => 'test1' } ... ]; How do I write my AoH to a CSV file with separators,...

Split a column's values into multiple rows awk/unix/python

Here is a sample row of input data : header: id,indicator,{(pid,days_remaining)} row: id_558314,1,{(property_66021,7),(property_24444,1),(property_285395,6)} Expected output of data : header: id,indicator,pid,days_remaining row1: id_558314,1,property_66021,7 row2: id_558314,1,property_24444,1 row3: id_558314,1,property_285395,6 Is this something easily doable using awk/unix/python? Thanks in advance. Your time and help is appreciated....

Why this exclusion not working for long sentences?

Command perl -ne 'print unless /.[240,]/' input.txt > output.txt which includes some sentences which are longer than 240 letters. Why? Example data Development of World Funny Society program on young people who are working hard for the sport and social life such that they have time to go pizzeria every...

Reading from DATA file handle

My perl module needs to use a look up table that's about 309,000 lines long. Currently the part that loads the table into an array looks (roughly) like this: use strict; use warnings; # load all the data from below my @ref_data; while (<DATA>) { push @ref_data, $_ } close...

How to find average and maximum in an interval using Shell [closed]

I would like to extract sum, mean and average in each 6 numbers interval from a column. I found many discussions related to this problem, but all those are for whole column. e.g. To compute sum of a column: awk '{sum+=$1} END { print sum}' To calculate Average: awk '{sum+=$1}...

Use Unix Executable File to Run Shell Script and MPKG File

I have 2 shell scripts and 2 mpkg installer, I am trying to use an unix excitable file to run them all. here is the script I have, but it always has error message "No such file or directory" ? #!/bin/sh # Find the absolute script current path path=$( cd...

How to extract single-/multiline regex-matching items from an unpredictably formatted file and put each one in a single line into output file?

I have a very huge file which looks like this: <a>text</a>text blah <b>data1</b>abc<b>data2</b> <b>data3</b>blahblah <c>text</c> <d>text</d> <x>blahblah<b>data4 data5 data6</b> <b>data7 </x> That is, its formatting is unpredictable. I need to extract each <b>...</b> item (it might contain multiline text!) and put every one of them in a single separate line....

Calling find more than once on the same folder tree

I'm running a find command multiple times on the same group of files. The results of my find commands are usually disjoint sets, AKA I'm running find -mmin +35; find -mmin -25, and doing different things to the results. It seems sort of silly to search through the entire file...

Command line arguments in Perl

I am working on an open source project for GSoC and I have this piece of Perl code with me. I need to create another Perl file for a similar task. However, I am having trouble understanding 3 lines of this file. More specifically, I am not able to understand...

How to copy matches from an extremely large file if it contains no newlines?

The problem is I cannot avoid working with extremely big files which contain no newlines in them: <a>text1</a>...gigabytes of data here, all in one single line...[a text to extract b> What should I do if I want to copy matches from this file (putting every match in a separate line,...

How to remove characters before and including an underscore?

In bash scripting what's an efficient way to do the following please? var="fooo_barrrr" What is the best way to remove all characters before and including the '_' so that var becomes "barrrr" please?...

Counting occurrences of a word in a string in Perl

I am trying to find out the number of occurrences of "The/the". Below is the code I tried" print ("Enter the String.\n"); $inputline = <STDIN>; chop($inputline); $regex="\[Tt\]he"; if($inputline ne "") { @splitarr= split(/$regex/,$inputline); } [email protected]; print $scalar; The string is : Hello the how are you the wanna work on...

Redirect output from file to stdout

I have a program that can output its results only to files, with -o option. This time I need to output it to console, i.e. stdout. Here is my first try: myprog -o /dev/stdout input_file But it says: /dev/ not writable I've found this question that's similar to mine, but...

Perl : Display perl variable awk sed echo

When I am using below command directly its working fine but when I am trying to put this in perl script its giving lots of error. my $calculate = `echo "$value" | awk -F "SP=" '{print $2}' | awk -F ";" '{print $1}' | awk -F ":" '{print $2}' |...

How to match and remove the content preceding it from a file in unix [closed]

I have a mysql dump file, and i want to remove the content of the file after "-- Final view structure for view view_oss_user" using sed/perl. The input file is something like this : Content : rom `target` */; /*!50001 SET character_set_client = @saved_cs_client */; /*!50001 SET character_set_results = @saved_cs_results...

calling cgi script from other cgi script

I found a puzzling behavior using perl cgi for which -- I guess -- there is a perfectly valid explanation but I couldn't find one. There is this question on stackoverflow, but what it describes seems to fail for me. The situation: I have two perl cgi scripts. One of...

unable to understand qr interpolation

I was reading Programming Perl where I learned qr interpolation of strings as regex as : $re = qr/my.STRING/is; print $re; # prints (?si-xm:my.STRING) and it says The /s and /i modifiers were enabled in the pattern because they were supplied to qr//. The /x and /m, however, are disabled...

Perl Debugging Using Flags

So my goal is to find an easy way to turn on print statements in Perl a flag. In C/C++ you can use a #define to choose if certain code is run and it is a way to turn on and off debug print statements. Where if a #define DEBUG...

Find numbers in a file and change their value with perl

I have a file with some data in it but there are a bunch of annoying numbers that are less than one which I wanted to just change to 1 instead of manually doing it. I was wondering how you would do this in perl. I tried using something like...

How to pass a hash as optional argument to -M in command line

I know that when we need to pass some arguments to the use keyword after a package name we can pass them in the command line after the -M parameter. For example: use feature 'say'; say 'hello!'; can be invoked from the command line with >perl -Mfeature=say -e"say 'hello!'" But...

Multiple line search in a file using java or unix command

I want to find the following pattern in a file like subclass "Pool1" 11:22:33:44:55:66 { dynamic; } the above pattern I have to find in a file. How can I find either using java or Unix command?...

storing 'du' result in a variable [duplicate]

This question already has an answer here: How to return the output of program in a variable? 4 answers I am using Putty with bash-4.2. Therein, I am outputting file size with: du -m myfile.csv which returns: 1.25 myfile.csv How do i store this line in a variable so...

Capture tee's argument inside piped Perl execution

How to capture piped command's argument ? I use : perl -some_args | tee arg_filename How to get arg_filename 's value inside ? CONTEXT I need to send this filename in a mail which sends at the end. I need to use tee because we dump huge...

Join Statement omitting entries

Using: Unix 2.6.18-194.el5 I am having an issue where this join statement is omitting values/indexes from the match. I found out the values are between 11-90 (out of about 3.5 Million entries) and I have tried to look for foreign characters but I may be overlooking something (Tried cat -v...

What does this horribly ugly (yet somehow secretly beautiful) Perl code do?

I found this code in a Powerpoint presentation about Perl scripting and this was on a page demonstrating how ugly Perl code can get if you really wanted to make it that way. The presentation says nothing about what this code does. Honestly I am just very curious to know......

Why can I view some Unix executable files in Mac OS X and not others?

I am on a Macbook Pro on Mac OS X 10.10 (Yosemite). When I go to /usr/bin, git is there as a unix executable file. When I open it up in Sublime Text, all I get is unreadable machine code. However, when I open up a different Unix executable fileā€”in...

Version-dependent fallback code

I have a script that needs to run on multiple servers, however, each server may not have the same version of Perl available and may have differing features. Perl v5.14 introduced the /r modifier for regular expressions which returns the result of a substitution and leaves the original text alone....

Perl: Multiply loops, 1 hash and regex

I got stuck with logic behind loops (while & foreach) and AoH. I have basic knowledge about loops and arrays of hashes, but I can't quite understand how to combine them into 1 single and simple solution. My task is to check regular user's password age, if it is older...

Regex in Perl Uninitialized $1

My string looks like this: <File `../Path/To/My_File.gif'> I want to extract just "Path/To/My_File.gif". Here is the check I have: if ($row =~ /(?<=File `..\/).*(?=')/) { print "Found it!\n"; print "$1\n"; } I see "Found it!" printed to the console but also get an error saying that $1 is uninitialized. What...

pass enter key from Java to Shell script

I am trying a Java program to run multiple commands in unix environment. I would need to pass 'ENTER' after each command. Is there some way to pass enter in the InputStream. JSch jsch=new JSch(); Session session=jsch.getSession("MYUSERNAME", "SERVER", 22); session.setPassword("MYPASSWORD"); Properties config = new Properties(); config.put("StrictHostKeyChecking", "no"); session.setConfig(config); session.connect(); Channel...

Plain text emails displayed as attachment on some email clients

The email can be viewed normally using some email clients (Evolution, Thunderbird), but with other clients (e.g., GMX) the body of the message remains empty and an attachment containing the body of the message is sent. I would like to know how I can prevent this from happening, since the...

Opening multiple files in perl array

I have a perl script where by I assigned all the files with a .log extension to an array called @allfiles. How do I run my script for the files stored in each array? My idea is something like open(my $fn, '<', @allfiles) or die "Could not open file '@files':...

How to open a new terminal from my working terminal with same directory in Linux?

Is there any command to open a new terminal from my working terminal with same directory while working with ssh in Linux? e.g., I am working in the following terminal. My pwd is /home/work/Kayan01/test_run I want to open a new terminal directly from my working terminal, so that the pwd...