bin/ppll

filter: Statementstmt 95.4% 21 / 22 Branchbran n/a Conditioncond n/a MC/DCmcdc n/a Subroutinesub 100% 7 / 7 Podpod n/a totaltotal 96.5% 28 / 29 CC 1 SCAR 0.0
CC
1
Cov
95%
BEGIN
1.0
BEGIN
1.0
BEGIN
1.0
CRAP 1.0
« blib/lib/App/ppll.pm ↑ Summary
1 #!/usr/bin/env perl
2  
3 =encoding utf8
4  
5 =head1 NAME
6  
7 ppll
8  
9 =cut
10  
11 5 ▶use autodie;
Subroutine BEGIN: called CC=1 SCAR=0.0
12 5 ▶use strict;
Subroutine BEGIN: called CC=1 SCAR=0.0
13 5 ▶use utf8::all;
Subroutine BEGIN: called CC=1 SCAR=0.0
14 5 ▶use v5.20;
Subroutine BEGIN: called CC=1 SCAR=0.0
15 5 ▶use warnings;
Subroutine BEGIN: called CC=1 SCAR=0.0
16  
17 5 ▶use FindBin;
Subroutine BEGIN: called CC=1 SCAR=0.0
18  
19 5 ▶use lib "$FindBin::Bin/../lib";
Subroutine BEGIN: called CC=1 SCAR=0.0
20  
21 5 require App::ppll;
22  
23 0 exit App::ppll->new->call;
24  
25 =head1 SYNOPSIS
26  
27 ppll [OPTIONS] COMMAND [ARGS]
28 ppll [OPTIONS] -c COMMAND [PARAMETERS]
29 ppll --help
30 ppll --version
31  
32 In the first form, all arguments after I<COMMAND> are considered arguments to
33 I<COMMAND>, not options to C<ppll>, even if they begin with a hyphen.
34  
35 =head1 DESCRIPTION
36  
37 C<ppll> is a tool to control the execution of commands. It can run commands in
38 parallel, construct commands from lists of parameters, and more.
39  
40 It handles the output of commands and can prefix lines with which command
41 produced it, print timestamps, etc.
42  
43 C<ppll> has functionality similar to C<xargs> and C<parallel>.
44  
45 =head2 PARAMETERS AND JOBS
46  
47 ‘Parameters’ is what makes C<ppll> run multiple commands. In one way or
48 another C<ppll> can be given a number of parameters, and it will create and run
49 one ‘job’ for each parameter.
50  
51 If no parameter-related option is used C<ppll> will read the parameters from
52 STDIN (if it’s not a TTY). Ordinarily, each line read becomes one parameter, but
53 there are options to control this.
54  
55 The other important input to C<ppll> is the ‘command’. The command-line
56 arguments after all options make up the command and its arguments.
57  
58 Each job will execute the command. All occurrences of C<{}> in any part of the
59 command (and its arguments) will be replaced with the job’s parameter. If there
60 are no such occurrences the parameter will be appended as an additional
61 argument.
62  
63 For example:
64  
65 ppll echo {} <<EOF
66 cow
67 pig
68 sheep
69 EOF
70  
71 Will create jobs that execute C<echo cow>, C<echo pig> and so on. An other way
72 to do the same thing would be:
73  
74 ppll echo <<EOF
75 cow
76 pig
77 sheep
78 EOF
79  
80 C<{}> becomes useful when it’s not supposed to be the last argument, e.g:
81  
82 ppll cp {} farm/ <<EOF
83 cow
84 pig
85 sheep
86 EOF
87  
88 Or when it’s supposed to be just a part of an argument, or show up more than
89 once, e.g:
90  
91 ppll curl -o photo-of-a-{}.jpg http://example.com/img/{}.jpg <<EOF
92 cow
93 pig
94 sheep
95 EOF
96  
97 The above would run
98 C<curl -o photo-of-a-cow.jpg http://example.com/img/cow.jpg>, etc.
99  
100 =head3 JOB CONTROL
101  
102 The default is to run several such jobs in parallel. Once a job is finished
103 another one is started, until there are no more jobs to run.
104  
105 The number of jobs run at the same time is controlled by the B<--jobs> option.
106 The default is one per CPU (but not necessarily one on each CPU).
107  
108 If a job fails (exits with a code other than 0), no more jobs will be started.
109 C<ppll> I<attempts to> stop other running jobs.
110  
111 The B<--force> option tells C<ppll> to not care about failed jobs and always
112 keep starting new ones. Any failed jobs are still noted and will still affect
113 C<ppll>’s own exit code.
114  
115 =head3 SERIAL LINES, PARALLEL FIELDS
116  
117 This feature deserves its own section.
118  
119 It can be used to parallelise heavy tasks, while also making sure certain tasks
120 are run in a particular order.
121  
122 Each line of input will create a ‘group’ of parameters, that will be run before
123 the next group starts. Each line is split up into individual parameters (using
124 the C<IFS> environment variable), that will be run at the same time.
125  
126 An example:
127  
128 ppll --slpf wake-up.sh <<EOF
129 rooster
130 hens chickens
131 cows pigs sheep
132 cat
133 EOF
134  
135 The above would run C<wake-up.sh> first for C<rooster>. I<Then> it would run it
136 for C<hens> and C<chickens>, at the same time, but after the C<rooster> job has
137 completed. Then C<cows>, C<pigs> and C<sheep> in parallel, and so on.
138  
139 =head1 OPTIONS
140  
141 =head2 GENERAL
142  
143 =over
144  
145 =item B<--help>
146  
147 =item B<-h>
148  
149 =item B<-?>
150  
151 Print (this) help and exit.
152  
153 =item B<--version>
154  
155 =item B<-V>
156  
157 Print B<ppll>’s version and exit.
158  
159 =back
160  
161 =head2 PARAMETERS AND JOBS
162  
163 =over
164  
165 =item B<--command> I<COMMAND>
166  
167 =item B<-C> I<COMMAND>
168  
169 Run I<COMMAND> as a shell command with each parameter passed as an argument.
170 This allows for shell constructs, such as piping, redirecting, etc.
171  
172 If used then the rest of the command-line arguments are used as parameters,
173 instead of a command.
174  
175 More or less the same as:
176  
177 ppll $SHELL -c <COMMAND> -
178  
179 (C<$SHELL> defaults to C</bin/sh>.)
180  
181 =item B<--delimiter> I<DELIMITER>
182  
183 =item B<-d> I<DELIMITER>
184  
185 Use I<DELIMITER>, as a regex, to split lines into fields. Implies B<--fields>.
186  
187 =item B<--empty>
188  
189 Include empty parameters. These are otherwise discarded.
190  
191 =item B<--fields>
192  
193 =item B<-f>
194  
195 Alias for C<--mode fields>.
196  
197 =item B<--force>
198  
199 =item B<-k>
200  
201 Keep starting new jobs even if a job has failed.
202  
203 =item B<--jobs> I<N>
204  
205 =item B<-j> I<N>
206  
207 Run (up to) this many jobs in parallel.
208  
209 =item B<--lines>
210  
211 =item B<-l>
212  
213 Alias for C<--mode lines>.
214  
215 =item B<--mode> I<MODE>
216  
217 How input should be turned into parameters. I<MODE> is one of:
218  
219 =over
220  
221 =item C<auto> (default)
222  
223 If there’s only one line of input, behave as if C<fields>, otherwise as if
224 C<lines>.
225  
226 =item C<fields>
227  
228 Split lines into fields and use each field as a parameter.
229 Splitting is done using a delimiter set with B<--delimiter>, or the C<IFS>
230 environment variable.
231  
232 =item C<lines>
233  
234 Use each line as a parameter.
235  
236 =item C<slpf>
237  
238 Split each line into parameters. All parameters from a single line will be run
239 in parallel, but lines will be run serially.
240  
241 =back
242  
243 See also: B<--fields>, B<--lines>, B<--serial-lines-parallel-fields>
244  
245 =item B<--random>
246  
247 =item B<-R>
248  
249 Shuffle the parameters.
250  
251 =item B<--replace-string> I<STRING>
252  
253 =item B<-replstr> I<STRING>
254  
255 =item B<-I=s> I<STRING>
256  
257 Use I<STRING> instead of C<{}> as a placeholder for the parameter.
258  
259 =item B<--reverse>
260  
261 =item B<-r>
262  
263 Use the parameters in reverse order.
264  
265 =item B<--sequence> I<SPEC>
266  
267 =item B<-s> I<SPEC>
268  
269 Use I<SPEC> as parameters.
270  
271 I<SPEC> is in one of the forms:
272  
273 =over
274  
275 =item C<N..M>
276  
277 Parameters are from I<N> to I<M>, inclusive.
278  
279 =item C<M>
280  
281 Parameters are from 1 to I<M>, inclusive.
282  
283 =back
284  
285 =item B<--serial-lines-parallel-fields>
286  
287 =item B<-slpf>
288  
289 Alias for C<--mode slpf>.
290  
291 =back
292  
293 =head2 OUTPUT
294  
295 =over
296  
297 =item B<--colour>
298  
299 =item B<--color>
300  
301 =item B<--no-colour>
302  
303 =item B<--no-color>
304  
305 Turn on or off ANSI colours in the output.
306  
307 The default is to enable colours if STDOUT is a TTY.
308  
309 =item B<--markers>
310  
311 =item B<--no-markers>
312  
313 Prefix (the default), or don’t prefix, jobs’ STDOUT and STDERR output with
314 C<E<gt>> or C<≫>, respectively.
315  
316 =item B<--quiet>
317  
318 =item B<-q>
319  
320 Print less. Can be repeated for less and less.
321  
322 =item B<--summary>
323  
324 =item B<-S>
325  
326 =item B<--no-summary>
327  
328 Print (the default if there’s more than one parameter), or don’t print, a
329 summary of how many jobs ran and which of them failed.
330  
331 =item B<--timestamp-format> I<FORMAT>
332  
333 Format timestamps using I<FORMAT>. See L<DateTime::strftime|DateTime::strftime>.
334  
335 The default is C<%T.%3N>.
336  
337 Implies B<--timestamps>.
338  
339 =item B<--timestamps>
340  
341 =item B<-t>
342  
343 Prefix lines from jobs’ output with a timestamp. The time printed will be the
344 I<approximate> time the line was printed by the job.
345  
346 =item B<--verbose>
347  
348 =item B<-v>
349  
350 Print more. Can be repeated for more and more.
351  
352 =back
353  
354 =head1 DIAGNOSTICS
355  
356 =head1 CONFIGURATION AND ENVIRONMENT
357  
358 C<ppll> uses some environment variables in some cases:
359  
360 =over
361  
362 =item C<COLUMNS>
363  
364 If set then used when printing things across the whole width of the terminal. If
365 not set then C<ppll> will try to work out the terminal width in other ways,
366 falling back to 80.
367  
368 =item C<IFS>
369  
370 Used for splitting lines into fields, e.g. with C<--fields> or
371 C<--serial-lines-parallel-fields>.
372  
373 =item C<SHELL>
374  
375 Used with C<--commmand> (C<-c>).
376  
377 =back
378  
379 =head1 EXAMPLES
380  
381 ppll -t make
382  
383 Run a long-running command, e.g. C<make>, and prefix the output with timestamps.
384  
385 ppll -j 10 -s 1..1000 gobbledygook --with-rigmarole --number
386  
387 For each number from 1 to 1,000 (inclusive), run the C<gobbledygook> command
388 with some arguments. Add the a number as the last argument, e.g. C<gobbledygook
389 --with-rigmarole --number 42>.
390  
391 Run no more than 10 such commands at a time.
392  
393 ppll curl -o photo-of-a-{}.jpg http://example.com/img/{}.jpg <<EOF
394 cow
395 pig
396 sheep
397 EOF
398  
399 Download three files in parallel. Note the use of C<{}> to use each parameter in
400 two places in each command.
401  
402 =head1 SEE ALSO
403  
404 =over
405  
406 =item
407 L<xargs|https://pubs.opengroup.org/onlinepubs/9699919799/utilities/xargs.html>
408  
409 POSIX, and thus available almost everywhere, and can run multiple commands based
410 on a list of parameters, much like C<ppll>.
411  
412 =item L<GNU Parallel|https://www.gnu.org/software/parallel/>
413  
414 Also runs commands in parallel, but with an interface quite different to
415 C<ppll>’s.
416  
417 =item L<moreutil|https://joeyh.name/code/moreutils/>’s C<parallel> and C<ts>
418  
419 moreutils contains a C<parallel> tool similar yet different to GNU Parallel.
420  
421 It also has a separate tool for adding timestamps to output.
422  
423 =back
424  
425 =head1 AUTHOR
426  
427 Theo -q Willows, C<< <theo@willows.se> >>
428  
429 =head1 BUGS AND LIMITATIONS
430  
431 Please report any bugs or feature requests through the web interface at
432 L<https://gitlab.com/munkei-software/ppll/issues>.
433  
434 =head1 SUPPORT
435  
436 You can find documentation for this module with the perldoc command.
437  
438 perldoc ppll
439  
440 Or:
441  
442 ppll --help
443  
444 You can also look for information at:
445  
446 =over
447  
448 =item * MetaCPAN
449  
450 L<https://metacpan.org/pod/App::ppll>
451  
452 =item * GitLab
453  
454 L<https://gitlab.com/munkei-software/ppll>
455  
456 =back
457  
458 =head1 LICENSE AND COPYRIGHT
459  
460 Copyright 2019 Theo Willows.
461  
462 This program is free software; you can redistribute it and/or modify it under
463 the terms of either: the GNU General Public License as published by the Free
464 Software Foundation; or the Artistic License.
465  
466 See L<http://dev.perl.org/licenses/> for more information.
467  
468 =cut
« blib/lib/App/ppll.pm ↑ Summary