perl,pipe , Multiple forks and IO:Pipe


Multiple forks and IO:Pipe

Question:

Tag: perl,pipe

I am trying to build a program wich creates some forks and writes the results of the forks back to the main program. Thereby I try to use IO::Pipe

sub ForkRequests {

  my $pipe = IO::Pipe->new();
  my $pid;

  foreach my $feature ( @features ) {

    if ( $pid = fork() ) {
      $pipe->reader();
      while ( <$pipe> ) {
        print $_. "\n";
      }
    }
    elsif ( defined $pid ) {

      #child
      $pipe->writer();

      #somecalculations [...]
      print $pipe $calcresults;
    }
  }
}

I got my code for doing a pipe from the module's documentation.

If i now try to execute, I get an error message

Can't locate object method "reader" via package "IO::Pipe::End" at lmtest3.pl line 56.
Can't locate object method "writer" via package "IO::Pipe::End" at lmtest3.pl line 63.
Can't locate object method "reader" via package "IO::Pipe::End" at lmtest3.pl line 56, <GEN0> line 1.
Can't locate object method "writer" via package "IO::Pipe::End" at lmtest3.pl line 63, <GEN0> line 1.

So, my code does not seem to initiate a pipe object, but an IO::Pipe::End. So my question is, can anybody see the mistake in there? Why does it return the wrong object, and how would this be done correctly?

EDIT

I have some requests to some servers (most of the time 1 request to 7 ervers). Those request names are saved in @features, and will be executed at the point of #somecalculations.

Because the server response is pretty slow, I want those requests to start in parallel. They all have to get back to the main program and print the reply to the console.

I tried this code

sub ForkRequests {

  my $i = 0;
  my @pipes;
  my $pid;

  foreach my $feature ( @features ) {

    @pipes[$i] = IO::Pipe->new();
    if ( $pid = fork() ) {
      @pipes[$i]->reader();
    }
    elsif ( defined $pid ) {

      #child
      @pipes[$i]->writer();

      # calculations
      my $w = @pipes[$i];
      print $w $calc;
      print $w "end\n";
    }

    $i++;
  }
}

if ( $pid == 1 ) {
  while ( 1 ) {
    foreach my $pipe ( @pipes ) {
      while ( <$pipe> ) {
        unless ( $_ == "end" ) {
          print $_. "\n";
        }
        else { last; }
      }
    }
  }
}
else {
  exit;
}

}

as said, to save those pipes, but I still got a problem in reading them, as the program exits before it gets answers.


Answer:

The problem is that you are forking multiple child processes but trying to use the same pipe for all of them.

The reader method converts $pipe into an IO::Pipe::End object that you can read data from, so the first child is connected correctly. But you then call reader again on the same $pipe, and the error is thrown because it is no longer an object of the right class.

You simply need to create a new pipe for each child process:

sub fork_requests {

    for my $feature ( @features ) {

        my $pipe = IO::Pipe->new;
        my $pid;

        if ( $pid = fork ) {
            $pipe->reader;
            print while <$pipe>;
        }
        elsif ( defined $pid ) {
            $pipe->writer;
            # some calculations . . .
            print $pipe $calcresults;
            exit;
        }
    }

}

Update

Okay I think I understand what it is you need. This complete program should show you.

I have written fork_requests so that it expects a list of features as parameters, and I have written the child code so that it sleeps for two seconds to emulate the processing time and then simply prints the name of the feature.

The parent code stores all the pipes in an array, as I suggested, and prints the output from each of them in the order they were queued. All five child processes complete after two seconds, so the parent is suspended for that time and then prints the features originally passed in.

use strict;
use warnings;

use IO::Pipe;

STDOUT->autoflush;

fork_requests('A' .. 'E');

sub fork_requests {

    my @pipes;

    for my $feature ( @_ ) {

        my $pipe = IO::Pipe->new;
        my $pid;

        if ( $pid = fork ) {
            $pipe->reader;
            push @pipes, $pipe;
        }
        elsif ( defined $pid ) {
            $pipe->writer;
            select $pipe;
            # some calculations . . .
            sleep 2;
            my $calcresults = $feature;
            print $calcresults, "\n";
            exit;
        }
    }

    for my $pipe ( @pipes ) {
      print while <$pipe>;
    }

}

output

A
B
C
D
E

Related:


How do I silence the HEAD of a curl request while using the silent flag?


bash,shell,curl,command-line,pipe
When I run the curl command and direct the data to a file, I get back the content of the site as expected. $ curl "www.site.com" > file.txt $ head file.txt Top of site ... However, this command shows a progress bar, which I do not want: % Total %...

Perl Debugging Using Flags


perl,debugging,script-debugging
So my goal is to find an easy way to turn on print statements in Perl a flag. In C/C++ you can use a #define to choose if certain code is run and it is a way to turn on and off debug print statements. Where if a #define DEBUG...

Windows/Linux child process STDIN differences


linux,windows,perl,process,stdin
I built a simple text processing script at work to be used by another program. When I was done, someone remembered that the script needs to not block STDIN/STDOUT for the tool using it to work right, and modified the script accordingly. The script opens *nix's cat in a subprocess...

Difficulties initializing an array in Perl


arrays,perl,hash
I have the following code: print Dumper($dec_res->{repositories}[0]); print Dumper($dec_res->{repositories}[1]); my @repos = ($dec_res->{repositories}); print scalar @repos . "\n"; and the output is the following: $VAR1 = { 'status' => 'OK', 'name' => 'apir', 'svnUrl' => 'https://url.whatever/svn/apir', 'id' => 39, 'viewvcUrl' => 'https://url.whatever/viewvc/apir/' }; $VAR1 = { 'status' => 'OK', 'name'...

calling cgi script from other cgi script


perl,cgi
I found a puzzling behavior using perl cgi for which -- I guess -- there is a perfectly valid explanation but I couldn't find one. There is this question on stackoverflow, but what it describes seems to fail for me. The situation: I have two perl cgi scripts. One of...

How to match and remove the content preceding it from a file in unix [closed]


mysql,perl,sed,solaris
I have a mysql dump file, and i want to remove the content of the file after "-- Final view structure for view view_oss_user" using sed/perl. The input file is something like this : Content : rom `target` */; /*!50001 SET character_set_client = @saved_cs_client */; /*!50001 SET character_set_results = @saved_cs_results...

Taking multiple header (rows matching condition) and convert into a column


bash,perl,command-line,awk,sed
Hello I have a file that has multiple Headers in it that I need to have turned into column values. The file looks like this: Day1 1,Smith,London 2,Bruce,Seattle 5,Will,Dallas Day2 1,Mike,Frisco 4,James,LA I would like the file to end up looking like this: Day1,1,Smith,London Day1,2,Bruce,Seattle Day1,5,Will,Dallas Day2,1,Mike,Frisco Day2,4,James,LA The file...

Deleting upto a line


bash,perl,shell,sed,scripting
I have a line that looks like: foo cat dog = -48.34277635 foo(horse->0) = -60.34277635 and I only want the last set of numbers: -60.34277635 The line is formatted with that exact spacing. I've looked everywhere for a simpler solution, but I can't find anything without chopping the file piece...

How to extract some text from an HTML doc using Web::Query


perl
I'm trying to extract the subject (between the h3 tags) in the following example using Web::Query. Find 'h3' returns the author text, but I want the h3 in the subject class instead. I tried .subject.div.h3 but it returns undef. #!/usr/bin/perl use strict; use warnings; use Web::Query; # libweb-query-perl use Data::Dumper;...

unable to understand qr interpolation


regex,perl
I was reading Programming Perl where I learned qr interpolation of strings as regex as : $re = qr/my.STRING/is; print $re; # prints (?si-xm:my.STRING) and it says The /s and /i modifiers were enabled in the pattern because they were supplied to qr//. The /x and /m, however, are disabled...

How to tell Perl to always run with use v5.010?


perl
When I write small programs or oneliners I cannot use say. I always need to put: #!/usr/bin/env perl use strict; use warnings; use 5.010; In oneliners I can simply do perl -E "say 'toto'" but in regular programs I don't know how to do it... Any idea?...

Counting occurrences of a word in a string in Perl


regex,perl
I am trying to find out the number of occurrences of "The/the". Below is the code I tried" print ("Enter the String.\n"); $inputline = <STDIN>; chop($inputline); $regex="\[Tt\]he"; if($inputline ne "") { @splitarr= split(/$regex/,$inputline); } [email protected]; print $scalar; The string is : Hello the how are you the wanna work on...

Perl & Regex within Windows CMD Line


regex,windows,perl
Is there anyway to accomplish matching + storing all in one cmd line? So instead of saving the matches to an array: i.e. ($matches) = $filecontents =~ m/.../g ...the matches would save to a *.txt file? I have been experimenting for a couple of days now, and believe that I...

How to copy matches from an extremely large file if it contains no newlines?


python,linux,bash,perl,grep
The problem is I cannot avoid working with extremely big files which contain no newlines in them: <a>text1</a>...gigabytes of data here, all in one single line...[a text to extract b> What should I do if I want to copy matches from this file (putting every match in a separate line,...

Capture tee's argument inside piped Perl execution


perl,unix
How to capture piped command's argument ? I use : perl my_script.pl -some_args | tee arg_filename How to get arg_filename 's value inside my_script.pl ? CONTEXT I need to send this filename in a mail which my_script.pl sends at the end. I need to use tee because we dump huge...

Perl would I use fc over uc?


perl
When would you ever need to use fc(), when would uc() ever fail? Perl fc documentation...

Creating a sequence of unique random digits


arrays,perl,foreach,unique
I have the following code use strict; use warnings; use 5.22.0; # Generating random seed using # Programming Perl p. 955 srand( time() ^ ($$ + ($$ << 15 ) ) ); # Generating code that could have duplicates my @code = ( (int(rand(9)) + 1), (int(rand(9)) + 1), (int(rand(9))...

Perl: Multiply loops, 1 hash and regex


arrays,regex,perl,hash,perl-data-structures
I got stuck with logic behind loops (while & foreach) and AoH. I have basic knowledge about loops and arrays of hashes, but I can't quite understand how to combine them into 1 single and simple solution. My task is to check regular user's password age, if it is older...

Continuous data stream from linux command line into python script


python,linux,command-line,tcp,pipe
I need to get a continuous stream of data flowing from a peripheral device, to a raspberry pi, and lastly to my computer (mac). So far, the data is can make its way continuously (in real time) from the peripheral, to the command line of my raspberry pi. Note that...

Perl XML-RPC output format/schema


php,xml,perl,rpc,xml-rpc
I'm writing a script in Perl to use two XML-RPC APIs. With one of them it works fine, with the other one i always get an empty result. After contacting the vendor of the second API, they sent me a PHP sample script to operate with the API. I couldn't...

How to pass a hash as optional argument to -M in command line


perl,hash,package,command-line-interface
I know that when we need to pass some arguments to the use keyword after a package name we can pass them in the command line after the -M parameter. For example: use feature 'say'; say 'hello!'; can be invoked from the command line with >perl -Mfeature=say -e"say 'hello!'" But...

Get ISO DateTime with only core modules in Perl?


perl
I would like to get a date-time string such as 2015-06-17 10:20:34 with only core modules. The reason of this is that cpan install DateTime takes ages on my machine because of the tests and the fetch of all the dependencies. If I give my Program to my colleagues. They...

Why does my value change when I am not resetting it?


perl,reference
I have the following example exhibiting the problem I'm struggling to resolve. In the toy example, I have an array @actors with two levels. I also have an array of hashes @people which I am using to 'look up' properties of the people in @actors. The output of the program...

Check for decimal point and add it at the end if its not there using awk/perl


regex,perl,shell,awk
I have test.dat file with values given below: 20150202,abc,,,,3625.300000,,,,,-5,,,,,,,,,,,,,,,,,,,,,, 20150202,def,,,,32.585,,,,,0,,,,,,,,,,,,,,,,,,,,,, 20150202,xyz,,,,12,,,,,0.004167,,,,,,,,,,,,,,,,,,,,,, My expected output is shown below: 20150202,abc,,,,3625.300000,,,,,-5.,,,,,,,,,,,,,,,,,,,,,, ^. added here 20150202,def,,,,32.585,,,,,0.,,,,,,,,,,,,,,,,,,,,,, ^. added here 20150202,xyz,,,,12.,,,,,0.004167,,,,,,,,,,,,,,,,,,,,,, ^. added here So if column 6 and 11 doesn't have decimal point in it, then we should add '.' at the end of...

Perl : Display perl variable awk sed echo


perl
When I am using below command directly its working fine but when I am trying to put this in perl script its giving lots of error. my $calculate = `echo "$value" | awk -F "SP=" '{print $2}' | awk -F ";" '{print $1}' | awk -F ":" '{print $2}' |...

How to write/read multiple times to a pipe


c,pipe,fork,ipc
I'm trying to make two child processes communicate with each other via pipe.First child have to write data 1 and data 2, and then second child writes data 3 and data 4. Then childrens read each other data and print them. This is the code that i have so far....

Batch script terminates in case of error when using pipe operator


batch-file,error-handling,pipe
I need to perferm error handling (check ERRORLEVEL) on an operation involving the pipe operator, but instead of the script continuing with a non-zero ERRORLEVEL, it terminates immediately. How can I avoid this behavior? Consider the following example. (Note that is a simplified constructed example to illustrate the problem -...

-M Script start time minus file modification time, in days


perl,perldoc
I would expect -M $_ to be negative, but it is zero for: perl -E 'qx(touch $_), sleep(5), say -M for "/tmp/file"' Does perldoc mentions such behavior?...

What does this horribly ugly (yet somehow secretly beautiful) Perl code do?


perl,formatting,deobfuscation
I found this code in a Powerpoint presentation about Perl scripting and this was on a page demonstrating how ugly Perl code can get if you really wanted to make it that way. The presentation says nothing about what this code does. Honestly I am just very curious to know......

Perl: Using Text::CSV to print AoH


arrays,perl,csv
I have an array of hashes (AoH) which looks like this: $VAR1 = [ { 'Unit' => 'M', 'Size' => '321', 'User' => 'test' } { 'Unit' => 'M' 'Size' => '0.24' 'User' => 'test1' } ... ]; How do I write my AoH to a CSV file with separators,...

How to pass url to wget


grep,pipe,wget
If I have a document with many links and I want to download especially one picture with the name: www.website.de/picture/example_2015-06-15.jpeg. How can I write a command that downloads me automatically exactly this one I extracted out of my document. My idea would be this but I'll get a failure message...

Pipe Feeding Anomaly


bash,pipe,fifo
I have a gzipped file that I've split into 3 separate files: xaa, xab, xac. I make a fifo mkfifo p1 and reassemble the files by reading from it, also calculating a checksum and unzipping the file in a pipe: cat p1 p1 p1 | tee >(sha1sum > sha1sum_new.txt) |...

Find numbers in a file and change their value with perl


regex,perl
I have a file with some data in it but there are a bunch of annoying numbers that are less than one which I wanted to just change to 1 instead of manually doing it. I was wondering how you would do this in perl. I tried using something like...

Version-dependent fallback code


perl
I have a script that needs to run on multiple servers, however, each server may not have the same version of Perl available and may have differing features. Perl v5.14 introduced the /r modifier for regular expressions which returns the result of a substitution and leaves the original text alone....

Why this exclusion not working for long sentences?


text-processing,perl
Command perl -ne 'print unless /.[240,]/' input.txt > output.txt which includes some sentences which are longer than 240 letters. Why? Example data Development of World Funny Society program on young people who are working hard for the sport and social life such that they have time to go pizzeria every...

Plain text emails displayed as attachment on some email clients


perl,email,attachment,mime,plaintext
The email can be viewed normally using some email clients (Evolution, Thunderbird), but with other clients (e.g., GMX) the body of the message remains empty and an attachment containing the body of the message is sent. I would like to know how I can prevent this from happening, since the...

problems copying shared hash in perl threads


multithreading,perl
I encountered what I feel is strange behavior of shared hash in perl and needed some help understanding it. The actual problem is in a far larger code-base and I have tried reducing it to smaller reproducible script. So essentially the problem I'm facing is I have a shared variable...

Create unicode character with pack


perl,unicode
I am trying to understand how Perl handles unicode. use feature qw(say); use strict; use warnings; use Encode qw(encode); say unpack "H*", pack("U", 0xff); say unpack "H*", encode( 'UTF-8', chr 0xff ); Output: ff c3bf Why do I get ff and not c3bf when using pack ?...

Why Filter::Indent::HereDoc complain when blank line in middle of HereDoc


perl,heredoc
I am trying Filter::Indent::HereDoc which allows one to indent the HereDocument. This is very useful, to be able to have HereDoc that flows with the code logic. From the above link When a 'here document' is used, the document text and the termination string must be flush with the left...

Reading from DATA file handle


performance,perl
My perl module needs to use a look up table that's about 309,000 lines long. Currently the part that loads the table into an array looks (roughly) like this: use strict; use warnings; # load all the data from below my @ref_data; while (<DATA>) { push @ref_data, $_ } close...

Print all non-matching items in Perl


regex,perl,if-statement,matching
I have a perl script that compares text from two files. The files on most days should be exactly the same, but if for some reason some of the text in the file changes I want the script to print out all the text that is not exactly the same...

Reducing code verbosity and efficiency


perl
I came across the below where some heavy stipulations were done, finally we got a number of @hits and we need to return just one: if ($#hits > 0) { my $highestScore = 0; my $chosenMatch = ""; for $hit (@hits) { my $currScore = 0; foreach $k (keys %{$hit})...

Regex in Perl Uninitialized $1


regex,perl
My string looks like this: <File `../Path/To/My_File.gif'> I want to extract just "Path/To/My_File.gif". Here is the check I have: if ($row =~ /(?<=File `..\/).*(?=')/) { print "Found it!\n"; print "$1\n"; } I see "Found it!" printed to the console but also get an error saying that $1 is uninitialized. What...

Perl - an array content


arrays,perl
Can you explain me how to check if an element belongs to array? My script needs to know whether the element has wanted extension to make a shortcut and copy it to another directory. Here is an example: my @array = qw(avi mp4 mov); my $dir = "E:\Downloads"; opendir (my...

Looping variables


perl,scripting
I'm working with perl to make a script that will work with Dot products/assorted vector math. I've got a working script ( Still very much in progress/needs refinement ) that will do what I ask. #!/usr/bin/perl use strict; use warnings; use diagnostics; use Math::Vector::Real; use 5.010; use Math::Trig; my $source...

Command line arguments in Perl


perl
I am working on an open source project for GSoC and I have this piece of Perl code with me. I need to create another Perl file for a similar task. However, I am having trouble understanding 3 lines of this file. More specifically, I am not able to understand...

C++ - back to start of loop without checking the condition


c++,perl,loops,redo
Programming on perl, we can use a smart function named 'redo' - we can go back to the start of the loop, without looking at the condition. It's useful when, for example, we create a table in which we must to set a expected values/characters (e.g. "a-b-c", nothing else). I...

Can't locate module(s) using Mojo::DOM


perl,dom,mojolicious,mojo
I'm new to Mojolicious. I am sure this is probably a setup problem, but it's eaten up an entire day of my time. I'm trying to run this simple test code #!/usr/bin/perl use strict; use warnings; use Mojo::DOM; use Mojo::UserAgent; my $ua = Mojo::UserAgent->new(); $ua->get('http://stackoverflow.com/questions/26353298/find-links-containing-bold-text-using-wwwmechanize')->res->dom('a div')->ancestors('div.spacer')->each( sub { say $_->all_text...

how to print all the lines of a text file from hash - perl


perl
I am reading a text file which is having multiple lines in it. I have been assigned a task to print the lines using hash. "\n" is the delimiter we can use. Here is what I tried and got stuck : code : use strict; use warnings; my %hash =...

Opening multiple files in perl array


arrays,perl
I have a perl script where by I assigned all the files with a .log extension to an array called @allfiles. How do I run my script for the files stored in each array? My idea is something like open(my $fn, '<', @allfiles) or die "Could not open file '@files':...