blib/lib/Util/Con.pm

filter: Statementstmt 58.6% 211 / 360 Branchbran 47.4% 93 / 196 Conditioncond 27.9% 26 / 93 MC/DCmcdc 21.5% 11 / 51 Subroutinesub 68.5% 24 / 35 Podpod 100% 10 / 10 totaltotal 50.3% 375 / 745 CC 146 SCAR 47.7
CC
146
Cov
51%
_structure_describe
506.0
_structure_render
272.0
_structure_kind
182.0
CRAP 118.0
↑ Summary
1 # ABSTRACT: Small everyday helpers: directory listing, shell commands, JSON files, number formatting, and data-structure shapes
2 75 ▶use strict;
Subroutine BEGIN: called CC=1 SCAR=0.0
3 75 ▶use feature 'say';
Subroutine BEGIN: called CC=1 SCAR=0.0
4 75 ▶use warnings FATAL => 'all';
Subroutine BEGIN: called CC=1 SCAR=0.0
5 # Devel::Confess is loaded for its side effect, which is process-wide: it
6 # installs $SIG{__DIE__} and $SIG{__WARN__} so that every die and warn in the
7 # program carries a stack trace. `use Util::Con` has always given the caller
8 # this, so it stays at the top, where loading the module is enough.
9 75 ▶use Devel::Confess;
Subroutine BEGIN: called CC=1 SCAR=0.0
10  
11 package Util::Con;
12 75 require 5.010001;
13 75 our $VERSION = 0.01;
14 75 require Exporter;
15 75 our @ISA = ('Exporter');
16 75 our @EXPORT = qw(dir execute file2string format_commas json_file_to_ref random_key random_value ref_to_json_file st structure);
17  
18 # Startup cost. Everything other than the pragmas, Exporter and
19 # Devel::Confess is loaded on first use, not at `use Util::Con`. Loading DDP,
20 # autodie, Capture::Tiny and JSON::MaybeXS eagerly cost ~40 ms of interpreter
21 # startup on every script that touched this module, even ones that only called
22 # format_commas().
23 #
24 # The rules that decide what can be deferred:
25 # - PRAGMAS (warnings, feature, autodie) act on the *compiler* for the
26 # enclosing lexical scope, so they can never be deferred. autodie was the
27 # single most expensive line here (~15 ms); this file does not use it, and
28 # the bare open()s it was covering carry explicit `or die`.
29 # - Exporter must exist at compile time (it supplies import()), but it is
30 # free -- it is already in memory as part of the core.
31 # - Everything else is a plain module, so `require` inside the sub that
32 # needs it works. `require` is idempotent and a repeat call is one %INC
33 # hash lookup, so putting it in the hot sub costs nothing measurable.
34 # - A runtime `require` does NOT run import(), so the imported short names
35 # (find, capture, decode_json, colored, blessed) are not available. Each
36 # call site below is therefore fully qualified. Calling a not-yet-loaded
37 # sub with a block prototype is also unparseable at compile time, which is
38 # why Capture::Tiny::capture is handed an explicit sub{} instead of a bare
39 # {}.
40  
41 # ---------------------------------------------------------------------------
42 # `use Util::Con` hands the calling package three things besides @EXPORT:
43 # autodie, and DDP's p() and np().
44 #
45 # autodie is LEXICAL, and that is the hard case. It is Fatal underneath, and
46 # Fatal::import does a bare `caller()` with no $Level support: calling
47 # autodie->import() directly from in here makes Fatal believe the importing
48 # file is Util/Con.pm. Its runtime leak-guard then sees calls arriving from a
49 # different file and correctly declines to apply -- open() in the caller just
50 # returns undef again, silently. Verified: the naive version does nothing at
51 # all.
52 #
53 # So the import has to be performed in a context where caller() reports the
54 # caller's package and file. A string eval carrying a #line directive does
55 # exactly that, and is the same trick Import::Into uses (not depended on here
56 # because it is not a core module). $line/$file come from our own caller, so
57 # autodie's scope starts at the `use Util::Con` line, which is where a
58 # hand-written `use autodie` would have started it too.
59 #
60 # The eval only compiles a closure, and the import runs when that closure is
61 # called after the eval has returned, which is also what Import::Into does.
62 # Perls up to at least 5.12 keep a string eval's own %^H in force while the
63 # eval runs, and autodie 2.06 (core in 5.10.1 and 5.12.5) keeps the guard that
64 # un-installs it at the end of the scope in %^H: imported inside the eval,
65 # autodie was un-installed again as the eval returned, and a missing file
66 # opened in the caller returned false. t/00-load.t checks this.
67 #
68 # p() and np() are installed by _install_ddp_stubs(), below.
69 #
70 # NB: `use Util::Con ();` skips import() entirely, per normal Perl semantics,
71 # so that is the escape hatch for a caller that wants none of this.
72 # ---------------------------------------------------------------------------
73 75 ▶sub import {
Subroutine import: called CC=2 SCAR=7.0
74 75 my $class = shift;
75 75 $class->export_to_level(1, $class, @_); # the @EXPORT subs, one level up
76  
77 75 my ($pkg, $file, $line) = caller;
78 75 _install_ddp_stubs($pkg);
79 75 require autodie;
80 75 ▶ my $autodie = eval qq{package $pkg;\n#line $line "$file"\nsub { autodie->import(':default') }}
Branch: 1 branch
BranchTrueFalseTotal
my $autodie = eval qq[package $pkg;\n#line $line "$file"\nsub { autodie->import(':default') }] or die "Util::Con: could not propagate autodie to ${file}: $@" 75 0 75
81 or die "Util::Con: could not propagate autodie to $file: $@";
82 75 $autodie->();
83 75 return;
84 }
85  
86 # Scripts rely on `use Util::Con` giving them DDP's p() and np() without a
87 # `use DDP` of their own. These stubs keep that convenience without paying
88 # DDP's ~16 ms unless something actually prints. They are installed at the
89 # caller's `use` line, which is compile time for the rest of the caller, and
90 # with DDP's own prototype, because that is what makes `p @files` pass \@files
91 # rather than a flat list. On first call, DDP->import() is run as if from the
92 # caller's package, so it overwrites the stub there with the real thing.
93 #
94 # A p() or np() the caller already has -- its own, or DDP's from an earlier
95 # `use DDP` -- is left alone. Util.pm, which this module was before it was
96 # Util::Con, compiled the stubs into main:: whenever it loaded, which redefined
97 # DDP's p() under its FATAL warnings when a script said `use DDP` first.
98 75 my %DDP_LOADED; # package => true once DDP's real p() and np() are installed there
99  
100 75 ▶sub _install_ddp_stubs {
Subroutine _install_ddp_stubs: called CC=3 SCAR=11.4
101 75 my ($pkg) = @_;
102 75 ▶ no strict 'refs';
Subroutine BEGIN: called CC=1 SCAR=0.0
103 75 for my $name ('p', 'np') {
104 150 my $glob = "${pkg}::$name";
105 150 ▶ next if defined &{$glob};
Branch: 1 branch
BranchTrueFalseTotal
if defined &{$glob;} 0 150 150
106 150 ▶ *{$glob} = sub (\[@$%&];%) {
Subroutine __ANON__: not called CC=1 SCAR=3.5
107 0 _load_ddp($pkg);
108 0 goto &{$glob};
109 150 };
110 }
111 75 return;
112 }
113  
114 0 ▶sub _load_ddp {
Subroutine _load_ddp: not called CC=3 SCAR=24.8
115 0 my ($pkg) = @_;
116 0 ▶ return if $DDP_LOADED{$pkg}++;
Branch: 1 branch
BranchTrueFalseTotal
if $DDP_LOADED{$pkg}++ 0 0 0
117 0 require DDP;
118 0 ▶ local $SIG{__WARN__} = sub {
Subroutine __ANON__: not called CC=2 SCAR=17.9
119 0 ▶ warn @_ unless $_[0] =~ m{\ASubroutine \Q$pkg\E::n?p redefined};
Branch: 1 branch
BranchTrueFalseTotal
unless $_[0] =~ /\ASubroutine \Q$pkg\E::n?p redefined/ 0 0 0
120 0 };
121 0 my $ok = eval qq{package $pkg;\nDDP->import({output => 'STDOUT', array_max => 10, show_memsize => 1, show_dualvar => 'off'}); 1};
122 0 ▶ die "Util::Con: could not load DDP into $pkg: $@" unless $ok;
Branch: 1 branch
BranchTrueFalseTotal
unless $ok 0 0 0
123 0 return;
124 }
125  
126 1050 ▶sub dir {
Subroutine dir: called CC=19 SCAR=29.5
Pod documented
127 1050 my %args = (
128 tests => 'f',
129 dir => '.',
130 regex => '.',
131 recursive => 0,
132 on_match => undef, # optional callback: streams results, keeps RAM flat
133 cpu => 1, # worker processes for a recursive walk; 1 = no forking
134 @_,
135 );
136  
137 1050 my %allowed = map { $_ => 1 } qw(tests dir regex recursive on_match cpu);
138 1050 ▶ if (my @bad = grep { !$allowed{$_} } keys %args) {
Branch: 1 branch
BranchTrueFalseTotal
if (my(@bad) = grep({not $allowed{$_};} keys %args)) 1 1049 1050
139 1 die "Unrecognized arguments: @bad\nAccepted args: "
140 . join(', ', sort keys %allowed);
141 }
142 # Added 's' (non-zero size), removed 't' (tty test - meaningless on a path)
143 die "Unrecognized characters in tests: $args{tests}"
144 1049 ▶ if $args{tests} =~ /[^rwxoRWXOezsfdlpSbcugkTB]/;
Branch: 1 branch
BranchTrueFalseTotal
if $args{"tests"} =~ /[^rwxoRWXOezsfdlpSbcugkTB]/ 1 1048 1049
145 die "cpu must be a positive integer, got '$args{cpu}'"
146 1048 ▶ unless $args{cpu} =~ /\A[1-9][0-9]*\z/;
Branch: 1 branch
BranchTrueFalseTotal
unless $args{"cpu"} =~ /\A[1-9][0-9]*\z/ 2 1046 1048
147 1046 ▶ my $re = eval { qr/$args{regex}/ }
Branch: 1 branch
BranchTrueFalseTotal
my $re = eval { do { qr/$args{"regex"}/ } } or die qq[Invalid regex '$args{"regex"}': $@] 1045 1 1046
148 or die "Invalid regex '$args{regex}': $@";
149 # Compile the tests once. The first test stats the path; every later test
150 # reuses the stat buffer via the special '_' filehandle, so N tests cost
151 # one stat(2) syscall instead of N (~2.4x faster for 4 tests).
152 # -l needs lstat, so it is hoisted to the front, and the test after it
153 # must re-stat the real path (otherwise '_' holds lstat data, or -l _
154 # would be a fatal error).
155 1045 my @t = split //, $args{tests};
156 1045 @t = ((grep { $_ eq 'l' } @t), (grep { $_ ne 'l' } @t));
157 1045 my @parts;
158 1045 my $fresh = 1; # 1 = the next test must stat the path itself rather than reuse '_'
159 1045 for my $t (@t) {
160 857 ▶ if ($t eq 'l') { push @parts, '-l $_[0]'; $fresh = 1 }
Branch: 2 branches
BranchTrueFalseTotal
if ($t eq 'l') { } 0 857 857
elsif ($fresh) { } 857 0 857
161 857 elsif ($fresh) { push @parts, "-$t \$_[0]"; $fresh = 0 }
162 0 else { push @parts, "-$t _" }
163 }
164 my $tester = @parts
165 ? (eval 'sub { ' . join(' && ', @parts) . ' }'
166 or die "Failed to compile tests: $@")
167 1045 ▶ : sub { 1 }; # empty tests string now means "no filtering", not "nothing matches"
Subroutine __ANON__: called CC=1 SCAR=0.0
Branch: 1 branch
BranchTrueFalseTotal
@parts ? : 857 188 1045
Condition50%1/2
eval 'sub { ' . join(" && ", @parts) . ' }' or die "Failed to compile tests: $@"
l!l
8570
MC/DC0%0/1
eval 'sub { ' . join(" && ", @parts) . ' }' or die "Failed to compile tests: $@"
eval 'sub { ' . join(" && ", @parts) . ' }'
Truth table50%1/2
A
A= eval 'sub { ' . join(" && ", @parts) . ' }'
Aresult
11
00
168  
169 1045 my $target_dir = $args{dir};
170 1045 $target_dir =~ s{(?<=[^/])/+\z}{}; # strip trailing slashes, but keep "/" intact
171 # "." is left off the front of results; "/" must not become "//etc".
172 6696 ▶ my $join = sub {
Subroutine __ANON__: called CC=3 SCAR=11.0
173 6696 my ($d, $item) = @_;
174 6696 ▶ $d eq '.' ? $item : $d eq '/' ? "/$item" : "$d/$item"
Branch: 2 branches
BranchTrueFalseTotal
$d eq '.' ? : 117 6579 6696
$d eq '/' ? : 1449 5130 6579
175 1045 };
176  
177 1045 my $cb = $args{on_match};
178 1045 my @items;
179 1045 ▶ my $emit = $cb // sub { push @items, $_[0] };
Subroutine __ANON__: called CC=1 SCAR=0.1
Condition66%2/3
$cb // sub { push @items, $_[0]; }
l!l&&r!l&&!r
909550
MC/DC0%0/2
$cb // sub { push @items, $_[0]; }
$cb sub { push @items, $_[0]; }
Truth table66%2/3
A // B
A= $cbB= sub { push @items, $_[0]; }
ABresult
1X1
011
000
180  
181 1045 ▶ if ($args{recursive}) {
Branch: 1 branch
BranchTrueFalseTotal
if ($args{'recursive'}) { } 630 415 1045
182 630 require File::Find; # ~1.6 ms, and only the recursive branch needs it
183 # Walk everything strictly below $start, handing each match to $sink.
184 402 ▶ my $find = sub {
Subroutine __ANON__: called CC=1 SCAR=0.0
185 402 my ($start, $sink) = @_;
186 402 my $strip = $start eq '.';
187 File::Find::find({
188 no_chdir => 1,
189 15726 ▶ wanted => sub {
Subroutine __ANON__: called CC=5 SCAR=16.1
190 15726 ▶ return if $_ eq $start; # $start itself is not "below" it
Branch: 1 branch
BranchTrueFalseTotal
if $_ eq $start 402 15324 15726
191 15324 my ($file) = m{([^/]+)\z}s; # basename, much cheaper than File::Spec
192 15324 ▶ if ($file =~ $re && $tester->($_)) {
Branch: 1 branch
BranchTrueFalseTotal
if ($file =~ /$re/ and &$tester($_)) 8821 6503 15324
Condition100%3/3
$file =~ /$re/ and &$tester($_)
!ll&&!rl&&r
228542188821
MC/DC100%2/2
$file =~ /$re/ and &$tester($_)
$file =~ /$re/ &$tester($_)
Truth table100%3/3
A and B
A= $file =~ /$re/B= &$tester($_)
ABresult
0X0
100
111
193 8821 my $path = $_;
194 8821 ▶ $path =~ s{^\./}{} if $strip;
Branch: 1 branch
BranchTrueFalseTotal
if $strip 324 8497 8821
195 8821 $sink->($path);
196 }
197 },
198 402 }, $start);
199 630 };
200 630 ▶ if ($args{cpu} == 1) {
Branch: 1 branch
BranchTrueFalseTotal
if ($args{'cpu'} == 1) { } 306 324 630
201 306 $find->($target_dir, $emit);
202 } else {
203 324 _dir_parallel($target_dir, $args{cpu}, $re, $tester, $join, $find, $emit);
204 }
205 } else {
206 # cpu is ignored here on purpose: one readdir plus one stat per entry
207 # is ~6 us warm, far below the cost of forking and shipping results back.
208 415 ▶ opendir my $dh, $target_dir
Branch: 1 branch
BranchTrueFalseTotal
opendir my $dh, $target_dir or die "Cannot open directory ${target_dir}: $!" 414 1 415
209 or die "Cannot open directory $target_dir: $!";
210 414 ▶ while (defined(my $item = readdir $dh)) { # defined(): a file named "0" is falsy
Branch: 1 branch
BranchTrueFalseTotal
if (defined(my $item = readdir $dh)) 4002 414 4416
211 4002 ▶ next if $item =~ /\A\.\.?\z/; # \A..\z: "$" would match before a trailing \n
Branch: 1 branch
BranchTrueFalseTotal
if $item =~ /\A\.\.?\z/ 828 3174 4002
212 3174 ▶ next if $item !~ $re;
Branch: 1 branch
BranchTrueFalseTotal
unless $item =~ /$re/ 690 2484 3174
213 2484 my $path = $join->($target_dir, $item);
214 2484 ▶ $emit->($path) if $tester->($path);
Branch: 1 branch
BranchTrueFalseTotal
if &$tester($path) 2070 414 2484
215 }
216 414 closedir $dh;
217 }
218 976 ▶ return $cb ? () : @items;
Branch: 1 branch
BranchTrueFalseTotal
$cb ? : 78 898 976
219 }
220  
221 # Jobs handed out per worker. Parallel::ForkManager starts a new job whenever
222 # a worker finishes, so more jobs than workers is what evens out a tree whose
223 # subdirectories differ wildly in size; each job costs one fork plus one
224 # Storable round-trip through a temp file, so too many jobs loses again.
225 # Measured 2026-09-23, 20-core NVMe box, warm cache, best of 3, seconds:
226 #
227 # tree (files) serial k=1 k=2 k=4 k=8 k=16
228 # ~/perl5 (150k), cpu=8 1.23 0.95 0.25 0.29 0.36 0.47
229 # ~/perl5 (150k), cpu=4 1.23 1.03 0.82 0.39 0.45 0.61
230 # perlbrew/build (120k), cpu=8 0.75 0.19 0.24 0.25 0.33 0.44
231 #
232 # k=1 and k=2 collapse on the skewed tree; 4 is the smallest value that does
233 # not. Re-run: edit the value below, then time
234 # perl -Ilib -MUtil::Con -e 'dir(dir => $ARGV[0], recursive => 1, cpu => 8)' TREE
235 75 my $DIR_JOBS_PER_CPU = 4;
236  
237 # Recursive dir() across $cpu worker processes. Results come back to this
238 # process and go through $emit here, so an on_match closure still sees and
239 # mutates the caller's variables -- it just runs after a whole job's batch
240 # arrives rather than per file, and match order differs from the serial walk.
241 324 ▶sub _dir_parallel {
Subroutine _dir_parallel: called CC=18 SCAR=29.8
242 324 my ($root, $cpu, $re, $tester, $join, $find, $emit) = @_;
243 324 require Parallel::ForkManager; # not core, and only cpu > 1 needs it
244  
245 # One subtree per job would starve the pool whenever a single directory
246 # holds most of the tree (~/perl5: one of 3 top-level dirs has 99% of the
247 # files, and a top-level split gave no speedup at any cpu). So walk
248 # breadth-first here until a level is wide enough to share out, testing
249 # the entries of every level walked; the final level's directories are
250 # already tested and become the jobs.
251 324 my $want = $cpu * $DIR_JOBS_PER_CPU;
252 324 my @level = ($root);
253 324 ▶ while (@level && @level < $want) {
Branch: 1 branch
BranchTrueFalseTotal
if (@level and @level < $want) 324 324 648
Condition66%2/3
@level and @level < $want
!ll&&!rl&&r
0324324
MC/DC50%1/2
@level and @level < $want
@level @level < $want
Truth table66%2/3
A and B
A= @levelB= @level < $want
ABresult
0X0
100
111
254 324 my @next;
255 324 for my $d (@level) {
256 324 ▶ opendir my $dh, $d or die "Cannot open directory $d: $!";
Branch: 1 branch
BranchTrueFalseTotal
opendir my $dh, $d or die "Cannot open directory ${d}: $!" 324 0 324
257 324 ▶ while (defined(my $item = readdir $dh)) {
Branch: 1 branch
BranchTrueFalseTotal
if (defined(my $item = readdir $dh)) 4860 324 5184
258 4860 ▶ next if $item =~ /\A\.\.?\z/;
Branch: 1 branch
BranchTrueFalseTotal
if $item =~ /\A\.\.?\z/ 648 4212 4860
259 4212 my $path = $join->($d, $item);
260 4212 ▶ $emit->($path) if $item =~ $re && $tester->($path);
Branch: 1 branch
BranchTrueFalseTotal
if $item =~ /$re/ and &$tester($path) 1755 2457 4212
Condition100%3/3
$item =~ /$re/ and &$tester($path)
!ll&&!rl&&r
117012871755
MC/DC100%2/2
$item =~ /$re/ and &$tester($path)
$item =~ /$re/ &$tester($path)
Truth table100%3/3
A and B
A= $item =~ /$re/B= &$tester($path)
ABresult
0X0
100
111
261 4212 ▶ push @next, $path if !-l $path && -d _; # lstat: File::Find does not follow symlinks either
Branch: 1 branch
BranchTrueFalseTotal
if not -l $path and -d _ 3888 324 4212
Condition66%2/3
not -l $path and -d _
!ll&&!rl&&r
32403888
MC/DC50%1/2
not -l $path and -d _
not -l $path -d _
Truth table66%2/3
not(A) and B
A= not -l $pathB= -d _
ABresult
0X0
100
111
262 }
263 324 closedir $dh;
264 }
265 324 @level = @next;
266 }
267 324 ▶ return unless @level;
Branch: 1 branch
BranchTrueFalseTotal
unless @level 0 324 324
268  
269 324 my @jobs;
270 324 push @{ $jobs[$_ % $want] }, $level[$_] for 0 .. $#level;
271  
272 324 my @errors;
273 324 my $pm = Parallel::ForkManager->new($cpu);
274 # The default 1 s poll in wait_all_children added a flat 1.01 s to every
275 # call, measured; 0 makes it block in waitpid instead.
276 324 $pm->set_waitpid_blocking_sleep(0);
277 2276 ▶ $pm->run_on_finish(sub {
Subroutine __ANON__: called CC=6 SCAR=22.8
278 2276 my ($pid, $exit, $id, $signal, $core, $data) = @_;
279 2276 ▶ if ($signal || $exit || !$data) {
Branch: 1 branch
BranchTrueFalseTotal
if ($signal or $exit or not $data) 0 2276 2276
Condition33%2/6
$signal or $exit or not $data
l!l&&r!l&&!r
002276
$signal or $exit
l!l&&r!l&&!r
002276
MC/DC0%0/3
$signal or $exit or not $data
$signal $exit not $data
Truth table25%1/4
A or B or not(C)
A= $signalB= $exitC= not $data
ABCresult
1XX1
01X1
0011
0000
280 0 push @errors, "dir: worker $pid died (exit $exit, signal $signal)";
281 0 return;
282 }
283 2276 ▶ push @errors, $data->{error} if defined $data->{error};
Branch: 1 branch
BranchTrueFalseTotal
if defined $data->{"error"} 0 2276 2276
284 2276 $emit->($_) for @{ $data->{paths} };
285 324 });
286 324 for my $job (@jobs) {
287 2414 ▶ $pm->start and next;
Branch: 1 branch
BranchTrueFalseTotal
$pm->start and next 2346 68 2414
288 68 my @found;
289 68 ▶ my $ok = eval { $find->($_, sub { push @found, $_[0] }) for @$job; 1 };
Subroutine __ANON__: called CC=1 SCAR=0.0
290 68 ▶ $pm->finish(0, { paths => \@found, error => $ok ? undef : "$@" });
Branch: 1 branch
BranchTrueFalseTotal
$ok ? : 68 0 68
291 }
292 256 $pm->wait_all_children;
293 256 ▶ die join "\n", @errors if @errors;
Branch: 1 branch
BranchTrueFalseTotal
if @errors 0 256 256
294 256 return;
295 }
296  
297 8 ▶sub execute {
Subroutine execute: called CC=15 SCAR=27.8
Pod documented
298 8 my ($cmd, $return, $die) = @_;
299 8 ▶ $return = 'exit' unless defined $return;
Branch: 1 branch
BranchTrueFalseTotal
unless defined $return 2 6 8
300 8 ▶ $die = 1 unless defined $die;
Branch: 1 branch
BranchTrueFalseTotal
unless defined $die 7 1 8
301 8 ▶ if ($return !~ m/^(exit|stdout|stderr|all)$/) {
Branch: 1 branch
BranchTrueFalseTotal
unless ($return =~ /^(exit|stdout|stderr|all)$/) 1 7 8
302 1 die "you gave \$return = \"$return\", while this subroutine only accepts ^(exit|stdout|stderr|all)\$";
303 }
304 7 require Capture::Tiny; # ~7.5 ms
305 # Capture::Tiny redirects fd 1 and fd 2 and nothing else, so the child still
306 # inherits the parent's fd 0. A child that prompts -- rm on a
307 # write-protected file, cp -i, apt -- prompts only when fd 0 is a tty, then
308 # writes its question into the captured stderr where nobody can see it and
309 # blocks on the terminal for an answer the user does not know is wanted.
310 # (Observed 2026-09-13: `rm -r` on read-only cl.tex hung jobsearch/tex/tar.pl
311 # indefinitely.) Pointing fd 0 at /dev/null turns the prompt into an instant
312 # EOF, so such a child declines and exits instead of hanging.
313 # Reopening STDIN is the point, not `local *STDIN`: the child inherits the
314 # descriptor, and only reopening the handle perl holds on fd 0 -- which frees
315 # fd 0 and so reclaims it as the lowest available -- puts /dev/null there.
316 7 require File::Spec;
317 7 my $saved_stdin;
318 7 ▶ open $saved_stdin, '<&', \*STDIN or die "cannot save STDIN: $!"
Branch: 1 branch
BranchTrueFalseTotal
if defined fileno STDIN 7 0 7
Condition40%2/5
defined fileno STDIN && (open($saved_stdin, '<&', \*STDIN) || die("cannot save STDIN: $!"))
!ll&&!rl&&r
070
open $saved_stdin, "<&", \*STDIN or die "cannot save STDIN: $!"
l!l
70
MC/DC0%0/2
defined fileno STDIN && (open($saved_stdin, '<&', \*STDIN) || die("cannot save STDIN: $!"))
defined fileno STDIN open $saved_stdin, "<&", \*STDIN
Truth table0%0/3
A && B
A= defined fileno STDINB= open $saved_stdin, "<&", \*STDIN
ABresult
0X0
100
111
319 if defined fileno STDIN; # a caller may legitimately have closed it
320 7 ▶ open STDIN, '<', File::Spec->devnull or die "cannot reopen STDIN on devnull: $!";
Branch: 1 branch
BranchTrueFalseTotal
open STDIN, "<", "File::Spec"->devnull or die "cannot reopen STDIN on devnull: $!" 7 0 7
321 # An explicit sub{} rather than capture { ... }: the prototype that makes
322 # the block form legal is not visible at compile time under a runtime
323 # require, so perl would try to parse the block as an anonymous hash.
324 7 ▶ my ($stdout, $stderr, $exit) = Capture::Tiny::capture(sub {
Subroutine __ANON__: called CC=1 SCAR=0.0
325 7 system( $cmd )
326 7 });
327 7 ▶ if (defined $saved_stdin) {
Branch: 1 branch
BranchTrueFalseTotal
if (defined $saved_stdin) { } 7 0 7
328 7 ▶ open STDIN, '<&', $saved_stdin or die "cannot restore STDIN: $!";
Branch: 1 branch
BranchTrueFalseTotal
open STDIN, "<&", $saved_stdin or die "cannot restore STDIN: $!" 7 0 7
329 } else {
330 0 close STDIN;
331 }
332 7 ▶ if (($die == 1) && ($exit != 0)) {
Branch: 1 branch
BranchTrueFalseTotal
if ($die == 1 and $exit != 0) 1 6 7
Condition100%3/3
$die == 1 and $exit != 0
!ll&&!rl&&r
151
MC/DC100%2/2
$die == 1 and $exit != 0
$die == 1 $exit != 0
Truth table100%3/3
A and B
A= $die == 1B= $exit != 0
ABresult
0X0
100
111
333 1 say STDERR "exit = $exit";
334 1 say STDERR "STDOUT = $stdout";
335 1 say STDERR "STDERR = $stderr";
336 1 die "$cmd\n failed";
337 }
338 6 ▶ if ($return eq 'exit') {
Branch: 4 branches
BranchTrueFalseTotal
if ($return eq 'exit') { } 2 4 6
elsif ($return eq 'stderr') { } 1 3 4
elsif ($return eq 'stdout') { } 2 1 3
elsif ($return eq 'all') { } 1 0 1
339 2 return $exit
340 } elsif ($return eq 'stderr') {
341 1 chomp $stderr;
342 1 return $stderr
343 } elsif ($return eq 'stdout') {
344 2 chomp $stdout;
345 2 return $stdout
346 } elsif ($return eq 'all') {
347 1 chomp $stdout;
348 1 chomp $stderr;
349 return {
350 1 exit => $exit,
351 stdout => $stdout,
352 stderr => $stderr
353 }
354 } else {
355 0 die "$return broke pigeonholes"
356 }
357 0 return $stdout
358 }
359  
360 3 ▶sub file2string {
Subroutine file2string: called CC=2 SCAR=6.9
Pod documented
361 3 my ($file) = @_;
362 3 ▶ open my $fh, '<', $file or die "can't read $file: $!"; # was autodie
Branch: 1 branch
BranchTrueFalseTotal
open my $fh, "<", $file or die "can't read ${file}: $!" 2 1 3
363 2 return do { local $/; <$fh> };
364 }
365  
366 21 ▶sub format_commas { # https://stackoverflow.com/questions/33442240/perl-printf-to-use-commas-as-thousands-separator
Subroutine format_commas: called CC=6 SCAR=17.9
Pod documented
367 21 my ($n, $format) = @_;
368 21 ▶ $format = '.%02d' unless defined $format; # '%.0u' for integers
Branch: 1 branch
BranchTrueFalseTotal
unless defined $format 17 4 21
369 21 my $neg = $n < 0;
370 21 my $abs = abs $n;
371 21 my $int = int $abs;
372 # int(100 * (.005 + frac)) is the original expression, and is exactly
373 # round-half-up to hundredths -- .005 * 100 is the +0.5 before truncation.
374 # It is kept verbatim so no already-correct result shifts by a cent.
375 21 my $cents = int(100 * (.005 + ($abs - $int)));
376 # ...but it can land on 100 (.999 rounds up to 1.00), which used to be
377 # printed straight into the fraction: 999999.999 came out "999,999.100".
378 # The carry has to go into the integer part instead.
379 21 ▶ if ($cents >= 100) {
Branch: 1 branch
BranchTrueFalseTotal
if ($cents >= 100) 6 15 21
380 6 $int++;
381 6 $cents -= 100;
382 }
383 # Split off the sign and work on the magnitude: the old code fed a negative
384 # through int()/sprintf and produced "-1,234.-49" for -1234.5, and lost the
385 # sign entirely for -0.5. A value that rounds to zero stays unsigned, so
386 # -0.001 is "0.00" rather than "-0.00".
387 21 ▶ my $sign = ($neg && ($int || $cents)) ? '-' : '';
Branch: 1 branch
BranchTrueFalseTotal
$neg && ($int || $cents) ? : 4 17 21
Condition100%6/6
$neg and $int || $cents
!ll&&!rl&&r
1614
$int or $cents
l!l&&r!l&&!r
311
MC/DC100%3/3
$neg and $int || $cents
$neg $int $cents
Truth table100%4/4
A and B or C
A= $negB= $intC= $cents
ABCresult
0XX0
1000
11X1
1011
388 # '%.0f' rather than interpolating $int: a large magnitude stringifies in
389 # scientific notation, and the reverse/unpack grouping shredded that into
390 # "1e,+20" for 1e20.
391 21 my $digits = sprintf '%.0f', $int;
392 21 return $sign . reverse(join(',', unpack('(A3)*', reverse $digits))) . sprintf($format, $cents);
393 }
394  
395 4 ▶sub json_file_to_ref {
Subroutine json_file_to_ref: called CC=4 SCAR=13.9
Pod documented
396 4 my ($json_filename) = @_;
397 4 ▶ die "$json_filename doesn't exist or isn't a file" unless -f $json_filename;
Branch: 1 branch
BranchTrueFalseTotal
unless -f $json_filename 1 3 4
398 3 ▶ die "$json_filename has 0 size" if -s $json_filename == 0;
Branch: 1 branch
BranchTrueFalseTotal
if -s $json_filename == 0 1 2 3
399 2 ▶ open my $fh, '<:raw', $json_filename or die "can't read $json_filename: $!";
Branch: 1 branch
BranchTrueFalseTotal
open my $fh, "<:raw", $json_filename or die "can't read ${json_filename}: $!" 2 0 2
400 2 my $json = do { local $/; <$fh> }; # slurp; $/ restored immediately
401 2 close $fh;
402 # Python/NumPy emit bare NaN / Infinity / -Infinity, which are not valid
403 # JSON. Rewrite them to null (-> undef) only when they sit in JSON value
404 # position (after '[', ',' or ':'), so literal "NaN" inside strings is safe.
405 2 $json =~ s/[\[,:]\s*\K-?(?:NaN|Infinity)(?=\s*[,\]}])/null/g;
406 2 require JSON::MaybeXS; # ~4.3 ms
407 2 return JSON::MaybeXS::decode_json($json);
408 }
409  
410 # These return *some* key/value from the hash, not a random one -- whichever
411 # pair the hash's internal iterator yields first.
412 #
413 # They used to be `foreach my $key (keys %{$hash}) { return $key }`, which is
414 # O(n): `keys` in list context materializes EVERY key into a temporary list,
415 # and then the loop throws all but the first away. On a 50,000-key hash that
416 # was 1.48 ms per call; via each() it is 0.15 us, ~10,000x faster.
417 #
418 # The bare `keys` in void context resets the iterator, so repeated calls keep
419 # returning the first pair instead of walking off the end of the hash. That
420 # is the same side effect the old `keys`-in-list-context version had, so a
421 # caller in the middle of its own each() loop is no worse off than before.
422 #
423 # Falling off a foreach also returned '' rather than undef in scalar context;
424 # each() on an empty hash returns an empty list, so these now yield a true
425 # undef, which is what t/random.t always asserted.
426 3 ▶sub random_key {
Subroutine random_key: called CC=1 SCAR=0.0
Pod documented
427 3 my ($hash) = @_;
428 3 keys %{ $hash }; # reset the iterator
429 3 my ($key) = each %{ $hash };
430 3 return $key;
431 }
432  
433 3 ▶sub random_value {
Subroutine random_value: called CC=1 SCAR=0.0
Pod documented
434 3 my ($hash) = @_;
435 3 keys %{ $hash }; # reset the iterator
436 3 my (undef, $value) = each %{ $hash };
437 3 return $value;
438 }
439  
440 2 ▶sub ref_to_json_file {
Subroutine ref_to_json_file: called CC=3 SCAR=11.0
Pod documented
441 2 my ($ref, $json_filename) = @_;
442 2 my $ref_json_filename = ref $json_filename;
443 2 ▶ unless ($ref_json_filename eq '') {
Branch: 1 branch
BranchTrueFalseTotal
unless ($ref_json_filename eq "") 1 1 2
444 1 die "$json_filename isn't a scalar/string";
445 }
446 1 require JSON::MaybeXS;
447 1 require Term::ANSIColor;
448 1 ▶ open my $fh, '>:raw', $json_filename # Write it unmangled
Branch: 1 branch
BranchTrueFalseTotal
open my $fh, ">:raw", $json_filename or die "can't write ${json_filename}: $!" 1 0 1
449 or die "can't write $json_filename: $!"; # was autodie
450 1 say $fh JSON::MaybeXS::encode_json($ref);
451 1 say 'Wrote ' . Term::ANSIColor::colored(['blue on_red'], $json_filename);
452 1 return $json_filename;
453 }
454  
455 # ----------------------------------------------------------------------------
456 # structure(): print the *shape* of a data structure instead of its values.
457 #
458 # p() from Data::Printer shows you every value; when a 50,000-element array
459 # holds the same kind of record over and over, that is the wrong tool. This
460 # merges all elements of an array into ONE representative node, so you see the
461 # schema once. Keys absent from some of the merged records are flagged '?';
462 # values that disagree about their type become a UNION.
463 #
464 # Two entry points, mirroring DDP's own p()/np() split:
465 # st() - prototyped, so `st %hash` works like `p %hash`. Use this
466 # interactively.
467 # structure() - plain sub, takes a reference: structure(\%hash), or the
468 # return value of another call. Use this from other code.
469 #
470 # Scalar::Util and Term::ANSIColor are require'd in structure(), which st()
471 # also funnels through, so the per-value helpers below can call them fully
472 # qualified without re-checking %INC on every node. Scalar::Util is not loaded
473 # at `use` time because it drags in List::Util's XS bootstrap (~4 ms) and only
474 # this family ever touches it.
475  
476 75 my %STRUCTURE_COLOR = (
477 key => 'bold', # hash key names
478 opt => 'red', # the '?' optional marker
479 HASH => 'bold cyan',
480 ARRAY => 'bold yellow',
481 UNION => 'bold magenta',
482 elem => 'bright_black', # the '[*]' / '<alt>' / '(empty)' markers
483 class => 'bold white', # blessed package names
484 number => 'green',
485 string => 'bright_blue',
486 bool => 'bright_magenta',
487 null => 'bright_black',
488 CODE => 'yellow',
489 other => 'red', # anything unexpected
490 );
491  
492 0 ▶sub _structure_color {
Subroutine _structure_color: not called CC=3 SCAR=24.8
493 0 my ($name, $text, $use_color) = @_;
494 0 ▶ return $text unless $use_color;
Branch: 1 branch
BranchTrueFalseTotal
unless $use_color 0 0 0
495 0 ▶ return Term::ANSIColor::colored([$STRUCTURE_COLOR{$name} // $STRUCTURE_COLOR{other}], $text);
Condition0%0/3
$STRUCTURE_COLOR{$name} // $STRUCTURE_COLOR{"other"}
l!l&&r!l&&!r
000
MC/DC0%0/2
$STRUCTURE_COLOR{$name} // $STRUCTURE_COLOR{"other"}
$STRUCTURE_COLOR{$name} $STRUCTURE_COLOR{"other"}
Truth table0%0/3
A // B
A= $STRUCTURE_COLOR{$name}B= $STRUCTURE_COLOR{"other"}
ABresult
1X1
011
000
496 }
497  
498 # Classify one value. Blessed containers keep their package name so that
499 # Foo=HASH and a plain HASH don't get merged into the same node.
500 0 ▶sub _structure_kind {
Subroutine _structure_kind: not called CC=13 SCAR=52.0
501 0 my ($v) = @_;
502 0 ▶ return 'null' unless defined $v;
Branch: 1 branch
BranchTrueFalseTotal
unless defined $v 0 0 0
503 0 my $r = ref $v;
504 0 ▶ return (Scalar::Util::looks_like_number($v) ? 'number' : 'string') if $r eq '';
Branch: 2 branches
BranchTrueFalseTotal
if $r eq "" 0 0 0
&Scalar::Util::looks_like_number($v) ? : 0 0 0
505 0 ▶ if (my $class = Scalar::Util::blessed($v)) {
Branch: 1 branch
BranchTrueFalseTotal
if (my $class = builtin::blessed($v)) 0 0 0
506 0 ▶ return 'bool' if $class eq 'JSON::PP::Boolean' || $class eq 'Types::Serialiser::Boolean';
Branch: 1 branch
BranchTrueFalseTotal
if $class eq "JSON::PP::Boolean" or $class eq "Types::Serialiser::Boolean" 0 0 0
Condition0%0/3
$class eq "JSON::PP::Boolean" or $class eq "Types::Serialiser::Boolean"
l!l&&r!l&&!r
000
MC/DC0%0/2
$class eq "JSON::PP::Boolean" or $class eq "Types::Serialiser::Boolean"
$class eq "JSON::PP::Boolean" $class eq "Types::Serialiser::Boolean"
Truth table0%0/3
A or B
A= $class eq "JSON::PP::Boolean"B= $class eq "Types::Serialiser::Boolean"
ABresult
1X1
011
000
507 0 ▶ return 'Regexp' if $class eq 'Regexp';
Branch: 1 branch
BranchTrueFalseTotal
if $class eq "Regexp" 0 0 0
508 0 ▶ my $rt = Scalar::Util::reftype($v) // '';
Condition0%0/2
builtin::reftype($v) // ""
l!l
00
MC/DC0%0/1
builtin::reftype($v) // ""
builtin::reftype($v)
Truth table0%0/2
A
A= builtin::reftype($v)
Aresult
11
00
509 0 ▶ return "$class=$rt" if $rt eq 'HASH' || $rt eq 'ARRAY' || $rt eq 'SCALAR' || $rt eq 'REF';
Branch: 1 branch
BranchTrueFalseTotal
if $rt eq "HASH" or $rt eq "ARRAY" or $rt eq "SCALAR" or $rt eq "REF" 0 0 0
Condition0%0/9
$rt eq "HASH" or $rt eq "ARRAY" or $rt eq "SCALAR" or $rt eq "REF"
l!l&&r!l&&!r
000
$rt eq "HASH" or $rt eq "ARRAY" or $rt eq "SCALAR"
l!l&&r!l&&!r
000
$rt eq "HASH" or $rt eq "ARRAY"
l!l&&r!l&&!r
000
MC/DC0%0/4
$rt eq "HASH" or $rt eq "ARRAY" or $rt eq "SCALAR" or $rt eq "REF"
$rt eq "HASH" $rt eq "ARRAY" $rt eq "SCALAR" $rt eq "REF"
Truth table0%0/5
A or B or C or D
A= $rt eq "HASH"B= $rt eq "ARRAY"C= $rt eq "SCALAR"D= $rt eq "REF"
ABCDresult
1XXX1
01XX1
001X1
00011
00000
510 0 return $class;
511 }
512 0 return $r; # HASH ARRAY SCALAR REF CODE GLOB
513 }
514  
515 # Merge a list of same-position values into a single structural node:
516 # { kind => 'hash', keys => { k => { node => .., optional => 0|1 } }, n, class }
517 # { kind => 'array', node => .., n => #instances, lens => [lengths], class }
518 # { kind => 'sref', node => .. } # \$x and \\$x
519 # { kind => 'leaf', types => { typename => 1, ... } }
520 # { kind => 'union', alts => { kindname => node } }
521 # $stack holds the refaddrs of the containers we are currently inside, so a
522 # self-referential structure terminates instead of recursing forever.
523 0 ▶sub _structure_describe {
Subroutine _structure_describe: not called CC=22 SCAR=62.3
524 0 my ($values, $opt, $depth, $stack) = @_;
525 0 my %by_kind;
526 0 push @{ $by_kind{ _structure_kind($_) } }, $_ for @{ $values };
527 0 my @kinds = sort keys %by_kind;
528  
529 0 ▶ if (@kinds > 1) {
Branch: 1 branch
BranchTrueFalseTotal
if (@kinds > 1) 0 0 0
530 return {
531 kind => 'union',
532 0 alts => { map { $_ => _structure_describe($by_kind{$_}, $opt, $depth, $stack) } @kinds },
533 };
534 }
535 0 ▶ my $k = $kinds[0] // 'null';
Condition0%0/2
$kinds[0] // "null"
l!l
00
MC/DC0%0/1
$kinds[0] // "null"
$kinds[0]
Truth table0%0/2
A
A= $kinds[0]
Aresult
11
00
536 0 my ($class, $reftype) = $k =~ /\A(.+)=(HASH|ARRAY|SCALAR|REF)\z/;
537 0 ▶ $reftype //= $k;
Condition0%0/3
$reftype //= $k
l!l&&r!l&&!r
000
MC/DC0%0/2
$reftype //= $k
$reftype $k
Truth table0%0/3
A //= B
A= $reftypeB= $k
ABresult
1X1
011
000
538  
539 0 ▶ if ($reftype eq 'HASH' || $reftype eq 'ARRAY' || $reftype eq 'SCALAR' || $reftype eq 'REF') {
Branch: 1 branch
BranchTrueFalseTotal
if ($reftype eq "HASH" or $reftype eq "ARRAY" or $reftype eq "SCALAR" or $reftype eq "REF") 0 0 0
Condition0%0/9
$reftype eq "HASH" or $reftype eq "ARRAY" or $reftype eq "SCALAR" or $reftype eq "REF"
l!l&&r!l&&!r
000
$reftype eq "HASH" or $reftype eq "ARRAY" or $reftype eq "SCALAR"
l!l&&r!l&&!r
000
$reftype eq "HASH" or $reftype eq "ARRAY"
l!l&&r!l&&!r
000
MC/DC0%0/4
$reftype eq "HASH" or $reftype eq "ARRAY" or $reftype eq "SCALAR" or $reftype eq "REF"
$reftype eq "HASH" $reftype eq "ARRAY" $reftype eq "SCALAR" $reftype eq "REF"
Truth table0%0/5
A or B or C or D
A= $reftype eq "HASH"B= $reftype eq "ARRAY"C= $reftype eq "SCALAR"D= $reftype eq "REF"
ABCDresult
1XXX1
01XX1
001X1
00011
00000
540 # Drop anything we're already inside: its shape is an ancestor's shape.
541 0 my @fresh = grep { !$stack->{ Scalar::Util::refaddr($_) } } @{ $values };
542 0 ▶ return { kind => 'leaf', types => { '(cycle)' => 1 } } unless @fresh;
Branch: 1 branch
BranchTrueFalseTotal
unless @fresh 0 0 0
543 0 ▶ if ($opt->{max_depth} && $depth >= $opt->{max_depth}) {
Branch: 1 branch
BranchTrueFalseTotal
if ($opt->{"max_depth"} and $depth >= $opt->{"max_depth"}) 0 0 0
Condition0%0/3
$opt->{"max_depth"} and $depth >= $opt->{"max_depth"}
!ll&&!rl&&r
000
MC/DC0%0/2
$opt->{"max_depth"} and $depth >= $opt->{"max_depth"}
$opt->{"max_depth"} $depth >= $opt->{"max_depth"}
Truth table0%0/3
A and B
A= $opt->{"max_depth"}B= $depth >= $opt->{"max_depth"}
ABresult
0X0
100
111
544 0 return { kind => 'leaf', types => { '(...)' => 1 } };
545 }
546 0 local @{ $stack }{ map { Scalar::Util::refaddr($_) } @fresh } = (1) x @fresh;
547  
548 0 ▶ if ($reftype eq 'HASH') {
Branch: 2 branches
BranchTrueFalseTotal
if ($reftype eq 'HASH') { } 0 0 0
elsif ($reftype eq 'ARRAY') { } 0 0 0
549 0 my %keys; # key => { vals => [...], seen => n }
550 0 for my $h (@fresh) {
551 0 for my $key (keys %{ $h }) {
552 0 push @{ $keys{$key}{vals} }, $h->{$key};
553 0 $keys{$key}{seen}++;
554 }
555 }
556 # A wide hash used as a dictionary has no interesting key names:
557 # merge every value into one representative entry instead.
558 0 ▶ if ($opt->{hash_max} && keys %keys > $opt->{hash_max}) {
Branch: 1 branch
BranchTrueFalseTotal
if ($opt->{"hash_max"} and keys %keys > $opt->{"hash_max"}) 0 0 0
Condition0%0/3
$opt->{"hash_max"} and keys %keys > $opt->{"hash_max"}
!ll&&!rl&&r
000
MC/DC0%0/2
$opt->{"hash_max"} and keys %keys > $opt->{"hash_max"}
$opt->{"hash_max"} keys %keys > $opt->{"hash_max"}
Truth table0%0/3
A and B
A= $opt->{"hash_max"}B= keys %keys > $opt->{"hash_max"}
ABresult
0X0
100
111
559 0 my @all = map { @{ $keys{$_}{vals} } } keys %keys;
560 return {
561 0 kind => 'hash',
562 class => $class,
563 n => scalar @fresh,
564 wide => scalar keys %keys,
565 keys => { '*' => {
566 node => _structure_describe(\@all, $opt, $depth + 1, $stack),
567 optional => 0,
568 }},
569 };
570 }
571 0 my %out;
572 0 for my $key (keys %keys) {
573 $out{$key} = {
574 node => _structure_describe($keys{$key}{vals}, $opt, $depth + 1, $stack),
575 0 ▶ optional => ($keys{$key}{seen} < @fresh ? 1 : 0),
Branch: 1 branch
BranchTrueFalseTotal
$keys{$key}{'seen'} < @fresh ? : 0 0 0
576 };
577 }
578 0 return { kind => 'hash', class => $class, n => scalar @fresh, keys => \%out };
579 }
580 elsif ($reftype eq 'ARRAY') {
581 0 my (@elems, @lens);
582 0 for my $a (@fresh) {
583 0 push @lens, scalar @{ $a };
584 0 push @elems, @{ $a };
585 }
586 return {
587 0 ▶ kind => 'array',
Branch: 1 branch
BranchTrueFalseTotal
@elems ? : 0 0 0
588 class => $class,
589 node => (@elems ? _structure_describe(\@elems, $opt, $depth + 1, $stack) : undef),
590 n => scalar @fresh,
591 lens => \@lens,
592 };
593 }
594 return { # SCALAR / REF
595 kind => 'sref',
596 class => $class,
597 0 node => _structure_describe([map { ${ $_ } } @fresh], $opt, $depth + 1, $stack),
598 };
599 }
600 0 return { kind => 'leaf', types => { $k => 1 } };
601 }
602  
603 0 ▶sub _structure_array_desc {
Subroutine _structure_array_desc: not called CC=6 SCAR=37.4
604 0 my ($node, $c) = @_;
605 0 my @lens = @{ $node->{lens} };
606 0 my ($min, $max) = ($lens[0], $lens[0]);
607 0 ▶ for (@lens) { $min = $_ if $_ < $min; $max = $_ if $_ > $max }
Branch: 2 branches
BranchTrueFalseTotal
if $_ < $min 0 0 0
if $_ > $max 0 0 0
608 0 ▶ my $range = $min == $max ? $min : "$min-$max";
Branch: 1 branch
BranchTrueFalseTotal
$min == $max ? : 0 0 0
609 0 my $out = $c->('ARRAY', "ARRAY[$range]");
610 0 ▶ $out .= " (from $node->{n} instances)" if $node->{n} > 1;
Branch: 1 branch
BranchTrueFalseTotal
if $node->{"n"} > 1 0 0 0
611 0 return $out;
612 }
613  
614 0 ▶sub _structure_render {
Subroutine _structure_render: not called CC=16 SCAR=56.1
615 0 my ($label, $node, $indent, $opt, $c, $lines) = @_;
616 0 ▶ my $pad = (' ' x $indent) . ($label // '');
Condition0%0/2
$label // ""
l!l
00
MC/DC0%0/1
$label // ""
$label
Truth table0%0/2
A
A= $label
Aresult
11
00
617 0 ▶ my $class = defined $node->{class} ? $c->('class', "$node->{class}=") : '';
Branch: 1 branch
BranchTrueFalseTotal
defined $node->{'class'} ? : 0 0 0
618  
619 0 ▶ if ($node->{kind} eq 'hash') {
Branch: 4 branches
BranchTrueFalseTotal
if ($node->{'kind'} eq 'hash') { } 0 0 0
elsif ($node->{'kind'} eq 'array') { } 0 0 0
elsif ($node->{'kind'} eq 'sref') { } 0 0 0
elsif ($node->{'kind'} eq 'union') { } 0 0 0
620 0 my @k = sort keys %{ $node->{keys} };
621 0 ▶ my $count = $node->{wide} // scalar @k;
Condition0%0/2
$node->{"wide"} // scalar @k
l!l
00
MC/DC0%0/1
$node->{"wide"} // scalar @k
$node->{"wide"}
Truth table0%0/2
A
A= $node->{"wide"}
Aresult
11
00
622 0 ▶ push @{ $lines }, $pad . $class . $c->('HASH', "HASH{$count}")
Branch: 2 branches
BranchTrueFalseTotal
$node->{'n'} > 1 ? : 0 0 0
$node->{'wide'} ? : 0 0 0
623 . ($node->{n} > 1 ? " (from $node->{n} instances)" : '')
624 0 . ($node->{wide} ? ' ' . $c->('elem', '(keys merged)') : '');
625 0 for my $key (@k) {
626 0 my $entry = $node->{keys}{$key};
627 my $name = ($key eq '*' && $node->{wide})
628 ? $c->('elem', '<*>') . ': '
629 0 ▶ : $c->('key', $key) . ($entry->{optional} ? $c->('opt', '?') : '') . ': ';
Branch: 2 branches
BranchTrueFalseTotal
$key eq '*' && $node->{'wide'} ? : 0 0 0
$entry->{'optional'} ? : 0 0 0
Condition0%0/3
$key eq "*" and $node->{"wide"}
!ll&&!rl&&r
000
MC/DC0%0/2
$key eq "*" and $node->{"wide"}
$key eq "*" $node->{"wide"}
Truth table0%0/3
A and B
A= $key eq "*"B= $node->{"wide"}
ABresult
0X0
100
111
630 0 _structure_render($name, $entry->{node}, $indent + 1, $opt, $c, $lines);
631 }
632 }
633 elsif ($node->{kind} eq 'array') {
634 0 push @{ $lines }, $pad . $class . _structure_array_desc($node, $c);
635 0 ▶ if ($node->{node}) {
Branch: 1 branch
BranchTrueFalseTotal
if ($node->{'node'}) { } 0 0 0
636 0 _structure_render($c->('elem', '[*]') . ': ', $node->{node}, $indent + 1, $opt, $c, $lines);
637 } else {
638 0 push @{ $lines }, (' ' x ($indent + 1)) . $c->('elem', '(empty)');
639 }
640 }
641 elsif ($node->{kind} eq 'sref') { # \$x: stay on the same line, no extra indent level
642 0 _structure_render($pad . $class . $c->('elem', '\\') . ' ', $node->{node}, 0, $opt, $c, $lines);
643 }
644 elsif ($node->{kind} eq 'union') {
645 0 push @{ $lines }, $pad . $c->('UNION', 'UNION');
646 0 for my $alt (sort keys %{ $node->{alts} }) {
647 0 _structure_render($c->('elem', "<$alt>") . ' ', $node->{alts}{$alt}, $indent + 1, $opt, $c, $lines);
648 }
649 }
650 else { # leaf
651 0 push @{ $lines }, $pad . join($c->('elem', '|'),
652 0 map { $c->($_, $_) } sort keys %{ $node->{types} });
653 }
654 0 return;
655 }
656  
657 0 ▶sub structure {
Subroutine structure: not called CC=12 SCAR=50.5
Pod documented
658 0 my ($ref, %args) = @_;
659 0 my %opt = (
660 output => 'stdout', # 'stdout' | 'stderr' | 'string' (string: return it, print nothing)
661 color => 'auto', # 'auto' | 1 | 0
662 max_depth => 0, # 0 = unlimited
663 hash_max => 32, # hashes wider than this collapse to one '<*>' entry; 0 = never
664 as => undef, # optional header line, like DDP's "as"
665 %args,
666 );
667 0 my %allowed = map { $_ => 1 } qw(output color max_depth hash_max as);
668 0 ▶ if (my @bad = grep { !$allowed{$_} } keys %opt) {
Branch: 1 branch
BranchTrueFalseTotal
if (my(@bad) = grep({not $allowed{$_};} keys %opt)) 0 0 0
669 0 die "Unrecognized arguments: @bad\nAccepted args: " . join(', ', sort keys %allowed);
670 }
671 0 ▶ die "output must be one of stdout, stderr, string" if $opt{output} !~ m/\A(stdout|stderr|string)\z/;
Branch: 1 branch
BranchTrueFalseTotal
unless $opt{"output"} =~ /\A(stdout|stderr|string)\z/ 0 0 0
672  
673 0 require Scalar::Util; # ~4 ms (List::Util XS bootstrap); see header note
674  
675 0 ▶ my $fh = $opt{output} eq 'stderr' ? \*STDERR : \*STDOUT;
Branch: 1 branch
BranchTrueFalseTotal
$opt{'output'} eq 'stderr' ? : 0 0 0
676 my $use_color = $opt{color} eq 'auto'
677 ? ((($ENV{CLICOLOR_FORCE} || ($opt{output} ne 'string' && -t $fh)) && !$ENV{NO_COLOR}) ? 1 : 0)
678 0 ▶ : $opt{color};
Branch: 2 branches
BranchTrueFalseTotal
$opt{'color'} eq 'auto' ? : 0 0 0
($ENV{'CLICOLOR_FORCE'} || $opt{'output'} ne 'string' && -t $fh) && !$ENV{'NO_COLOR'} ? : 0 0 0
Condition0%0/9
$ENV{"CLICOLOR_FORCE"} || $opt{"output"} ne "string" && -t $fh and not $ENV{"NO_COLOR"}
!ll&&!rl&&r
000
$ENV{"CLICOLOR_FORCE"} or $opt{"output"} ne "string" and -t $fh
l!l&&r!l&&!r
000
$opt{"output"} ne "string" and -t $fh
!ll&&!rl&&r
000
MC/DC0%0/4
$ENV{"CLICOLOR_FORCE"} || $opt{"output"} ne "string" && -t $fh and not $ENV{"NO_COLOR"}
$ENV{"CLICOLOR_FORCE"} $opt{"output"} ne "string" -t $fh not $ENV{"NO_COLOR"}
Truth table0%0/6
A or B and C and not(D)
A= $ENV{"CLICOLOR_FORCE"}B= $opt{"output"} ne "string"C= -t $fhD= not $ENV{"NO_COLOR"}
ABCDresult
00XX0
010X0
1XX00
01100
1XX11
01111
679 0 ▶ require Term::ANSIColor if $use_color; # ~0.8 ms, skipped entirely when piping
Branch: 1 branch
BranchTrueFalseTotal
if $use_color 0 0 0
680 0 ▶ my $c = sub { _structure_color($_[0], $_[1], $use_color) };
Subroutine __ANON__: not called CC=1 SCAR=6.9
681  
682 0 my @lines;
683 0 ▶ push @lines, $c->('key', $opt{as}) if defined $opt{as};
Branch: 1 branch
BranchTrueFalseTotal
if defined $opt{"as"} 0 0 0
684 0 _structure_render(undef, _structure_describe([$ref], \%opt, 0, {}), 0, \%opt, $c, \@lines);
685 0 my $out = join("\n", @lines) . "\n";
686 0 ▶ print {$fh} $out unless $opt{output} eq 'string';
Branch: 1 branch
BranchTrueFalseTotal
unless $opt{"output"} eq "string" 0 0 0
687 0 return $out;
688 }
689  
690 # st(): structure() with DDP's calling convention, so you can write
691 #
692 # st %hash; st @array; st $ref; st %hash, max_depth => 2;
693 #
694 # instead of backslashing by hand. The \[@$%&] prototype makes perl pass a
695 # reference to the caller's variable, which is the whole trick -- but it also
696 # means the argument must BE a variable: st(\%h), st({...}) and
697 # st(func()) are compile-time errors, exactly as they are for DDP's p().
698 # Call structure() directly in those cases.
699 #
700 # NOTE: the obvious short name 's' is impossible. perl's tokenizer resolves a
701 # bare 's' to the substitution operator before it ever looks for a sub of that
702 # name, so `s $ref` dies with "Substitution pattern not terminated" and even
703 # `s($ref)` is parsed as s(pat)(repl). Only `&s($ref)` would work, which
704 # defeats the point. Hence 'st'.
705 0 ▶sub st (\[@$%&];%) {
Subroutine st: not called CC=3 SCAR=24.8
Pod documented
706 0 my ($item, @args) = @_;
707 # `st %h` / `st @a` / `st &c` hand us the HASH/ARRAY/CODE ref directly, but
708 # `st $x` hands us a ref to the scalar $x -- one layer more than we want.
709 # The prototype rejects everything except a plain variable, so a SCALAR/REF
710 # here is always that added layer: peel exactly one. `st $hashref` then
711 # describes the hash, and `my $x = 42; st $x` prints 'number', not '\ number'.
712 0 ▶ my $ref = (ref($item) eq 'SCALAR' || ref($item) eq 'REF') ? ${ $item } : $item;
Branch: 1 branch
BranchTrueFalseTotal
ref $item eq 'SCALAR' || ref $item eq 'REF' ? : 0 0 0
Condition0%0/3
ref $item eq "SCALAR" or ref $item eq "REF"
l!l&&r!l&&!r
000
MC/DC0%0/2
ref $item eq "SCALAR" or ref $item eq "REF"
ref $item eq "SCALAR" ref $item eq "REF"
Truth table0%0/3
A or B
A= ref $item eq "SCALAR"B= ref $item eq "REF"
ABresult
1X1
011
000
713 0 return structure($ref, @args);
714 }
715  
716 75 1;
717 # from md2pod.pl πατερ ημων ο εν τοις ουρανοις, ἁγιασθήτω τὸ ὄνομά σου
718 =encoding utf8
719  
720 =head1 NAME
721  
722 Util::Con - Small everyday helpers: directory listing, shell commands, JSON files, number formatting, and data-structure shapes
723  
724 =head1 Synopsis
725  
726 use Util::Con;
727
728 my @scripts = dir(dir => '.', regex => '\.pl$', recursive => 1);
729 my $branch = execute('git rev-parse --abbrev-ref HEAD', 'stdout');
730 my $text = file2string('notes.txt');
731 print format_commas(1234567.891), "\n"; # 1,234,567.89
732
733 my $data = json_file_to_ref('results.json');
734 ref_to_json_file($data, 'copy.json');
735
736 st $data; # print the shape of $data, not its values
737 p $data; # Data::Printer's p(), loaded on first use
738  
739 =head1 Description
740  
741 C<Util::Con> is a collection of the functions I reach for in nearly every script: listing a directory with file tests and a regex, running a shell command and keeping its output, slurping a file, reading and writing JSON, putting commas in a number, and seeing the I<shape> of a large data structure without scrolling through every value in it.
742  
743 Loading it is meant to be cheap. Everything that is not needed to compile the caller is loaded the first time a function needs it, so a script that only calls C<format_commas> does not pay for C<JSON::MaybeXS>, C<Capture::Tiny> or C<Data::Printer>.
744  
745 =head2 What C<use Util::Con> does
746  
747 Besides exporting the ten functions below, C<use Util::Con> does three things to the file and package that loads it:
748  
749 =over
750  
751 =item * It turns on C<autodie ':default'> in that file, from the C<use> line onward, so a bare C<open> or C<close> that fails dies with a message rather than returning false.
752  
753 =item * It gives that package C<p> and C<np> from C<Data::Printer> (C<DDP>). They are stubs until first called, so C<DDP> is loaded only by a script that prints something. A C<p> or C<np> the package already has, such as one from an earlier C<use DDP>, is left alone.
754  
755 =item * It loads C<Devel::Confess>, which is process-wide: every C<die> and C<warn> in the program carries a stack trace.
756  
757 =back
758  
759 C<use Util::Con ();> imports nothing and skips the first two; C<Devel::Confess> is still loaded. C<use Util::Con 'format_commas';> imports only the names given, and still does all three.
760  
761 =head1 Functions
762  
763 All functions are exported by default.
764  
765 =head2 dir
766  
767 my @files = dir(dir => '/data', regex => '\.csv$');
768 my @tree = dir(dir => '/data', regex => '\.csv$', recursive => 1, cpu => 8);
769 dir(dir => '/data', recursive => 1, on_match => sub { say $_[0] });
770  
771 Returns the paths under C<dir> whose base name matches C<regex> and that pass every file test in C<tests>. The paths are C<dir> joined to the entry with C</>; with the default C<< dir =E<gt> '.' >>, the leading C<./> is left off. C<.> and C<..> are never returned.
772  
773 It takes these named arguments:
774  
775 =over
776  
777 =item * C<dir> (default C<.>): The directory to list. Trailing slashes are removed, but C</> stays C</>.
778  
779 =item * C<regex> (default C<.>): A pattern the base name must match, given as a string or a C<qr//>. An invalid pattern dies, naming it.
780  
781 =item * C<tests> (default C<f>): One or more of perl's file tests, as letters: C<rwxoRWXOezsfdlpSbcugkTB>. Every one must pass. C<'d'> lists directories; C<''> applies no test at all.
782  
783 =item * C<recursive> (default C<0>): C<1> descends into subdirectories, through C<File::Find>, without following symbolic links.
784  
785 =item * C<cpu> (default C<1>): For a recursive listing, the number of worker processes (through C<Parallel::ForkManager>). Ignored when C<recursive> is C<0>.
786  
787 =item * C<on_match> (no default): A code reference called with each path as it is found; C<dir> then returns an empty list, so a very large tree is never held in memory.
788  
789 =back
790  
791 The file tests share one C<stat> per path, so asking for four of them costs no more than asking for one. C<-l> is checked first, with C<lstat>.
792  
793 With C<cpu> greater than 1, C<dir> walks the tree breadth-first until one level holds four directories per worker, then shares those directories out. The results are the same as a serial walk, in a different order. C<on_match> still runs in the calling process, so it can change the caller's variables, but it is called once a worker's whole batch arrives rather than as each file is found.
794  
795 An unknown argument, a letter that is not a file test, a C<cpu> that is not a positive integer, and a directory that cannot be opened all die.
796  
797 =head2 execute
798  
799 my $exit = execute('make test'); # 0, or dies
800 my $out = execute('git log -1 --format=%H', 'stdout');
801 my $err = execute('perl -c script.pl', 'stderr');
802 my $all = execute('ls /nonexistent', 'all', 0); # does not die
803 # $all = { exit => 512, stdout => '', stderr => "ls: cannot access ..." }
804  
805 Runs C<$cmd> through the shell with C<system>, capturing its standard output and standard error with C<Capture::Tiny>. The second argument says what to return:
806  
807 =over
808  
809 =item * C<exit> (the default): the wait status, C<$?> -- 0 for success, 256 for C<exit 1>
810  
811 =item * C<stdout>: standard output, chomped
812  
813 =item * C<stderr>: standard error, chomped
814  
815 =item * C<all>: a hash reference with C<exit>, C<stdout> and C<stderr>
816  
817 =back
818  
819 The third argument, C<$die>, is 1 by default: a command that exits non-zero prints its exit status, standard output and standard error to C<STDERR> and dies. Pass 0 to have the status returned instead.
820  
821 The command's standard input is the null device (C</dev/null>, or C<nul> on Windows). A command that would prompt -- C<rm> on a write-protected file, C<cp -i> -- reads end-of-file and declines rather than waiting, unseen, for an answer. The caller's C<STDIN> is restored afterwards.
822  
823 =head2 file2string
824  
825 my $text = file2string('notes.txt');
826  
827 Returns the whole of a file as one string, read without any decoding layer. A file that cannot be opened dies, naming it.
828  
829 =head2 format_commas
830  
831 format_commas(1234567.891); # '1,234,567.89'
832 format_commas(1234567, '%.0u'); # '1,234,567'
833 format_commas(999999.999); # '1,000,000.00'
834 format_commas(-0.5); # '-0.50'
835 format_commas(1e20); # '100,000,000,000,000,000,000.00'
836  
837 Puts a comma between each group of three digits of the integer part and appends the hundredths, rounded half up. The second argument is the C<sprintf> format the hundredths are printed with, C<'.%02d'> by default; C<'%.0u'> prints nothing, for a whole number. Rounding that carries into the integer part does so (C<0.999> is C<1.00>), a negative number keeps its sign, a number that rounds to zero has none, and a large number is grouped as digits rather than printed in exponent form.
838  
839 =head2 json_file_to_ref
840  
841 my $ref = json_file_to_ref('results.json');
842  
843 Reads a JSON file and returns the decoded data, through C<JSON::MaybeXS>. Python's C<json> module and NumPy write C<NaN>, C<Infinity> and C<-Infinity> as bare words, which are not JSON; where one of those stands as a value it is read as C<null>, so it becomes C<undef>. The same words inside a string are left alone. A file that does not exist, or is empty, dies.
844  
845 =head2 ref_to_json_file
846  
847 ref_to_json_file($data, 'results.json'); # prints "Wrote results.json"
848  
849 Writes a data structure to a file as JSON, through C<JSON::MaybeXS>, prints C<Wrote> and the file name, and returns the file name. The file name must be a plain string; a reference dies.
850  
851 =head2 random_key, random_value
852  
853 my $key = random_key(\%hash);
854 my $value = random_value(\%hash);
855  
856 Return one key, or one value, of a hash -- whichever pair perl's hash iterator yields first, which is quick no matter how big the hash is. This is I<any> key, not a randomly chosen one: the same hash returns the same pair each time. It is useful for looking at one record of a hash of records. An empty hash returns C<undef>.
857  
858 Both reset the hash's iterator, so do not call them on a hash in the middle of an C<each> loop.
859  
860 =head2 structure
861  
862 my $text = structure($ref, output => 'string');
863 structure(\%config, max_depth => 2, as => 'config');
864  
865 Prints the I<shape> of a data structure rather than its values: C<p> shows every element of a 50,000-element array of records, while C<structure> merges them into one and shows the record once.
866  
867 my @people = (
868 { name => 'Ann', age => 31, langs => ['perl', 'R'] },
869 { name => 'Bo', age => 28, langs => [], email => undef },
870 );
871 st @people;
872  
873 prints
874  
875 ARRAY[2]
876 [*]: HASH{4} (from 2 instances)
877 age: number
878 email?: null
879 langs: ARRAY[0-2] (from 2 instances)
880 [*]: string
881 name: string
882  
883 A key that only some of the merged hashes have is marked C<?>. Values in the same place that are of different kinds are shown as a C<UNION> of each kind. Arrays show the range of their lengths. Blessed references show their class. A reference back to a containing structure is shown as C<(cycle)> rather than followed.
884  
885 It takes these options after the reference:
886  
887 =over
888  
889 =item * C<output> (default C<stdout>): C<stdout> or C<stderr> to print there; C<string> to print nothing. The text is returned in every case.
890  
891 =item * C<color> (default C<auto>): C<1> or C<0>. C<auto> colours the output when it goes to a terminal, or when C<CLICOLOR_FORCE> is set, unless C<NO_COLOR> is set.
892  
893 =item * C<max_depth> (default C<0>): How many levels to descend before printing C<(...)>; C<0> is no limit.
894  
895 =item * C<hash_max> (default C<32>): A hash with more keys than this is taken to be a lookup table, and its values are merged into one entry, C<< E<lt>*E<gt> >>, rather than listed by key. C<0> lists every key.
896  
897 =item * C<as> (no default): A line to print above the structure.
898  
899 =back
900  
901 =head2 st
902  
903 st %hash;
904 st @array;
905 st $ref, max_depth => 2;
906  
907 C<structure>, called the way C<p> is called: the argument is a variable, not a reference to one, and the options follow it. As with C<p>, it must be a variable -- C<st(\%h)> and C<st(func())> do not compile; call C<structure> for those.
908  
909 =head1 Installation
910  
911 cpanm Util::Con
912  
913 or, from a copy of the repository:
914  
915 perl Makefile.PL
916 make
917 make test
918 make install
919  
920 =head1 Compatibility
921  
922 C<Util::Con> runs on perl 5.10.1 and later, and is tested there. C<execute> hands its command to the shell, so the command has to be written for the shell of the system it runs on. On Windows, the paths C<dir> returns are joined with C</>, which perl accepts there; recursive listings with C<cpu> rely on C<Parallel::ForkManager>'s emulated C<fork>.
923  
924 =head1 Author
925  
926 David E. Condon L<mailto:dec986@gmail.com>
927  
928 =head1 License
929  
930 This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
↑ Summary